1043 words
5 minutes
Front-End Anti-Scraping with Font-Face

What Is a Web Crawler?#

A web crawler is a program that automatically extracts content from web pages. It downloads pages from the World Wide Web on behalf of search engines and is a core building block of any search engine.

Crawling data generally falls into two approaches:

  • Parsing the rendered HTML, locating the nodes of interest, and reading their text
  • Reverse-engineering the underlying API endpoints, which yields data more conveniently and precisely

Why Bother with Anti-Scraping?#

  • Crawlers sweep through as much high-value information on your site as their strategy allows, eating your bandwidth and piling load onto your servers
  • Malicious use of crawlers can mount DoS attacks against a web service, exhausting its resources until it can no longer serve legitimate users
  • Scrapers can bulk-harvest free-to-query resources and all kinds of sensitive information, walking away with core business data and eroding the company’s competitive edge
  • Winning a lawsuit against a scraper is a long shot — in China it remains a legal gray area: you might prevail in court, or the case might go nowhere

Designing a Web-Side Anti-Scraping Strategy#

Starting from two goals — what you see on the page is not what you get, and hitting the API directly gets you nothing usable — we put together the following scheme:

  • Serve everything over HTTPS
  • For logged-in users: if the request rate exceeds a threshold (level 1), throttle the account and slow down responses
  • If the request rate keeps climbing (level 2), feed the account corrupted data
  • If it goes further (level 3), block the account outright
  • Front-end technical countermeasures (the core of this post)

This scheme also covers OCR-based scraping. OCR presupposes a fully rendered page, and the business data on that page still has to come from an API. So by collecting and analyzing user behavior — request frequency within a time window, whether the interaction pattern looks human — you can flag likely bots and apply the same escalating ladder of throttling, corrupted data, and account bans.

Server-Side Encoding#

First, the backend and frontend agree on encoding rules:

  1. A random mapping table (say 0 -> 3, 1 -> 7, 2 -> 4, 3 -> 5, 4 -> 1, 5 -> 0, 6 -> 6, 7 -> 9, 8 -> 2, 9 -> 8)
  2. An agreed-upon transform (based on the equation y = kx + b, where k is the current month and b the current day of month — the linear coefficients are derived from today’s date. For example, on “2023-11-22”, k = 11 and b = 22)
  3. Convert the number to a string and join the digits with 3.1415926 (so 123 becomes 1 + ‘3.1415926’ + 2 + ‘3.1415926’ + 3, yielding ‘13.141592623.141592623.1415926’)

Client-Side Decoding#

  1. Split the server’s response on the 3.1415926 delimiter into an array
  2. Decode each value with x = (y - b) / k
  3. Join the array back into a string
  4. Render it with the custom font file (built from the server-side mapping)

The Result#

API definition

Front-end anti-scraping diagram

What the API returns

API response with encoded values

What sits in the DOM

Garbled data in the DOM

What the page displays

Correct data rendered on the page

How It’s Built#

Server Side#

Using Node.js as an example:

const encodeNumberRule1 = (number) => {
const numMap = {
0: 3,
1: 7,
2: 4,
3: 5,
4: 1,
5: 0,
6: 6,
7: 9,
8: 2,
9: 8,
};
const date = dayjs().format('YYYY-MM-DD');
const k = Number(date.split('-')[1]);
const b = Number(date.split('-')[2]);
const result = String(number).split('').map((item) => {
if (isNaN(item)) {
return item;
}
return numMap[item] * k + b;
}).join('3.1415926');
return result;
};
function encodeNumberRule2(number) {
// numMap is a random hex mapping table
const numMap = {
0: 0xe6a7,
1: 0xf257,
2: 0xa87e,
3: 0xb2c1,
4: 0xc7b2,
5: 0xa9f2,
6: 0xe2c7,
7: 0xf82b,
8: 0xc1b2,
9: 0xc8f6,
};
const date = dayjs().format('YYYY-MM-DD');
const k = Number(date.split('-')[1]);
const b = Number(date.split('-')[2]);
const result = String(number).split('').map((item) => {
if (item === '.' || item === '-') {
return item;
}
return (numMap[item] * k + b).toString(16);
}).join('3.1415926');
console.log('result', result);
return result.toString();
}
app.get('/api/test', (req, res) => {
res.send({
code: 200,
msg: 'test',
data: [
{
x: encodeNumberRule1(123),
y: encodeNumberRule1(-123.2),
rule: 1,
},
{
x: encodeNumberRule2(123),
y: encodeNumberRule2(-123.2),
rule: 2,
}
],
success: true,
});
});

Client Side#

Building the font files

  1. Convert the font file to SVG format (ttf to svg)
  2. Pick the glyphs you need and remap their unicode codepoints according to the mapping table agreed with the backend

Editing glyph codepoints

Remapped unicode table

  1. Download the modified font, convert it to woff and woff2, and register it in CSS (ttf to woff)
@font-face {
font-family: 'icomoon-num-1';
src: url('./assets/font/icomoon-num-1/icomoon.woff2') format('woff2'),
url('./assets/font/icomoon-num-1/icomoon.woff') format('woff');
font-weight: normal;
font-style: normal;
font-display: swap;
}
@font-face {
font-family: 'icomoon-num-2';
src: url('./assets/font/icomoon-num-2/icomoon-num-2.woff2') format('woff2'),
url('./assets/font/icomoon-num-2/icomoon-num-2.woff') format('woff');
font-weight: normal;
font-style: normal;
font-display: swap;
}
.num-font-rule-1 {
font-family: 'icomoon-num-1', sans-serif;
}
.num-font-rule-2 {
font-family: 'icomoon-num-2', sans-serif;
}

Writing the decode functions

const decryptNumRule1 = (num) => {
if (!num) {
return '';
}
// 1. Split the string into an array on the 3.1415926 delimiter
const arr = num.split('3.1415926');
const date = dayjs().format('YYYY-MM-DD');
const k = Number(date.split('-')[1]);
const b = Number(date.split('-')[2]);
// 2. Convert each item back to a number: x = (y - b) / k
const result = arr.map(item => {
if (isNaN(Number(item))) {
return item;
}
const y = Number(item);
return (y - b) / k;
});
// 3. Join the array back into a string
return result.join('');
};
const decryptNumRule2 = (num) => {
if (!num) {
return '';
}
// 1. Split the string into an array on the 3.1415926 delimiter
const arr = num.split('3.1415926');
const date = dayjs().format('YYYY-MM-DD');
const k = Number(date.split('-')[1]);
const b = Number(date.split('-')[2]);
// 2. Convert each item back to a number: x = (y - b) / k
const result = arr.map(item => {
if (isNaN(parseInt(item, 16))) {
console.log(item);
return item;
}
const y = parseInt(item, 16);
return `&#x${((y - b) / k).toString(16)};`;
});
// 3. Join the array back into a string
return result.join('');
};

Using it in your components

Note: four-digit hex unicode codepoints must be rendered with dangerouslySetInnerHTML, otherwise the raw string shows up on the page. In Vue, use the v-html directive for the same effect.

export default function Test() {
return <>
{
testData.map((item, index) => {
return <div key={index}>
{
item.rule === 1 && <>
<p className='num-font-rule-1'>{utils.decryptNumRule1(item.x)}</p>
<p className='num-font-rule-1'>{utils.decryptNumRule1(item.y)}</p>
</>
}
{
item.rule === 2 && <>
<p className='num-font-rule-2' dangerouslySetInnerHTML={{
__html: utils.decryptNumRule2(item.x),
}} />
<p className='num-font-rule-2' dangerouslySetInnerHTML={{
__html: utils.decryptNumRule2(item.y),
}} />
</>
}
<br />
</div>;
})
}
</>
}
Front-End Anti-Scraping with Font-Face
https://www.mihouo.com/en/posts/front/implementing-front-end-anti-scraping-with-font-face/
Author
Jia
Published at
2023-11-22
License
CC BY-NC-SA 4.0