What Is a Web Crawler?
A web crawler is a program that automatically extracts content from web pages. It downloads pages from the World Wide Web on behalf of search engines and is a core building block of any search engine.
Crawling data generally falls into two approaches:
- Parsing the rendered HTML, locating the nodes of interest, and reading their text
- Reverse-engineering the underlying API endpoints, which yields data more conveniently and precisely
Why Bother with Anti-Scraping?
- Crawlers sweep through as much high-value information on your site as their strategy allows, eating your bandwidth and piling load onto your servers
- Malicious use of crawlers can mount DoS attacks against a web service, exhausting its resources until it can no longer serve legitimate users
- Scrapers can bulk-harvest free-to-query resources and all kinds of sensitive information, walking away with core business data and eroding the company’s competitive edge
- Winning a lawsuit against a scraper is a long shot — in China it remains a legal gray area: you might prevail in court, or the case might go nowhere
Designing a Web-Side Anti-Scraping Strategy
Starting from two goals — what you see on the page is not what you get, and hitting the API directly gets you nothing usable — we put together the following scheme:
- Serve everything over HTTPS
- For logged-in users: if the request rate exceeds a threshold (level 1), throttle the account and slow down responses
- If the request rate keeps climbing (level 2), feed the account corrupted data
- If it goes further (level 3), block the account outright
- Front-end technical countermeasures (the core of this post)
This scheme also covers OCR-based scraping. OCR presupposes a fully rendered page, and the business data on that page still has to come from an API. So by collecting and analyzing user behavior — request frequency within a time window, whether the interaction pattern looks human — you can flag likely bots and apply the same escalating ladder of throttling, corrupted data, and account bans.
Server-Side Encoding
First, the backend and frontend agree on encoding rules:
- A random mapping table (say 0 -> 3, 1 -> 7, 2 -> 4, 3 -> 5, 4 -> 1, 5 -> 0, 6 -> 6, 7 -> 9, 8 -> 2, 9 -> 8)
- An agreed-upon transform (based on the equation y = kx + b, where k is the current month and b the current day of month — the linear coefficients are derived from today’s date. For example, on “2023-11-22”, k = 11 and b = 22)
- Convert the number to a string and join the digits with 3.1415926 (so 123 becomes 1 + ‘3.1415926’ + 2 + ‘3.1415926’ + 3, yielding ‘13.141592623.141592623.1415926’)
Client-Side Decoding
- Split the server’s response on the 3.1415926 delimiter into an array
- Decode each value with x = (y - b) / k
- Join the array back into a string
- Render it with the custom font file (built from the server-side mapping)
The Result
API definition

What the API returns

What sits in the DOM

What the page displays

How It’s Built
Server Side
Using Node.js as an example:
const encodeNumberRule1 = (number) => { const numMap = { 0: 3, 1: 7, 2: 4, 3: 5, 4: 1, 5: 0, 6: 6, 7: 9, 8: 2, 9: 8, }; const date = dayjs().format('YYYY-MM-DD'); const k = Number(date.split('-')[1]); const b = Number(date.split('-')[2]); const result = String(number).split('').map((item) => { if (isNaN(item)) { return item; } return numMap[item] * k + b; }).join('3.1415926'); return result;};
function encodeNumberRule2(number) { // numMap is a random hex mapping table const numMap = { 0: 0xe6a7, 1: 0xf257, 2: 0xa87e, 3: 0xb2c1, 4: 0xc7b2, 5: 0xa9f2, 6: 0xe2c7, 7: 0xf82b, 8: 0xc1b2, 9: 0xc8f6, };
const date = dayjs().format('YYYY-MM-DD'); const k = Number(date.split('-')[1]); const b = Number(date.split('-')[2]); const result = String(number).split('').map((item) => { if (item === '.' || item === '-') { return item; } return (numMap[item] * k + b).toString(16); }).join('3.1415926'); console.log('result', result); return result.toString();}
app.get('/api/test', (req, res) => { res.send({ code: 200, msg: 'test', data: [ { x: encodeNumberRule1(123), y: encodeNumberRule1(-123.2), rule: 1, }, { x: encodeNumberRule2(123), y: encodeNumberRule2(-123.2), rule: 2, } ], success: true, });});Client Side
Building the font files
- Convert the font file to SVG format (ttf to svg)
- Pick the glyphs you need and remap their unicode codepoints according to the mapping table agreed with the backend


- Download the modified font, convert it to woff and woff2, and register it in CSS (ttf to woff)
@font-face { font-family: 'icomoon-num-1'; src: url('./assets/font/icomoon-num-1/icomoon.woff2') format('woff2'), url('./assets/font/icomoon-num-1/icomoon.woff') format('woff'); font-weight: normal; font-style: normal; font-display: swap;}
@font-face { font-family: 'icomoon-num-2'; src: url('./assets/font/icomoon-num-2/icomoon-num-2.woff2') format('woff2'), url('./assets/font/icomoon-num-2/icomoon-num-2.woff') format('woff'); font-weight: normal; font-style: normal; font-display: swap;}
.num-font-rule-1 { font-family: 'icomoon-num-1', sans-serif;}.num-font-rule-2 { font-family: 'icomoon-num-2', sans-serif;}Writing the decode functions
const decryptNumRule1 = (num) => { if (!num) { return ''; } // 1. Split the string into an array on the 3.1415926 delimiter const arr = num.split('3.1415926'); const date = dayjs().format('YYYY-MM-DD'); const k = Number(date.split('-')[1]); const b = Number(date.split('-')[2]); // 2. Convert each item back to a number: x = (y - b) / k const result = arr.map(item => { if (isNaN(Number(item))) { return item; } const y = Number(item); return (y - b) / k; }); // 3. Join the array back into a string return result.join('');};
const decryptNumRule2 = (num) => { if (!num) { return ''; } // 1. Split the string into an array on the 3.1415926 delimiter const arr = num.split('3.1415926'); const date = dayjs().format('YYYY-MM-DD'); const k = Number(date.split('-')[1]); const b = Number(date.split('-')[2]); // 2. Convert each item back to a number: x = (y - b) / k const result = arr.map(item => { if (isNaN(parseInt(item, 16))) { console.log(item); return item; } const y = parseInt(item, 16); return `&#x${((y - b) / k).toString(16)};`; }); // 3. Join the array back into a string return result.join('');};Using it in your components
Note: four-digit hex unicode codepoints must be rendered with dangerouslySetInnerHTML, otherwise the raw string shows up on the page. In Vue, use the v-html directive for the same effect.
export default function Test() { return <> { testData.map((item, index) => { return <div key={index}> { item.rule === 1 && <> <p className='num-font-rule-1'>{utils.decryptNumRule1(item.x)}</p> <p className='num-font-rule-1'>{utils.decryptNumRule1(item.y)}</p> </> } { item.rule === 2 && <> <p className='num-font-rule-2' dangerouslySetInnerHTML={{ __html: utils.decryptNumRule2(item.x), }} /> <p className='num-font-rule-2' dangerouslySetInnerHTML={{ __html: utils.decryptNumRule2(item.y), }} /> </> } <br /> </div>; }) } </>}