paul/picpeak
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

fix(og): broaden social-crawler coverage (Bluesky Cardyb, WeChat-scraper, fediverse, etc.)

Browse Source 
From alexvaltchev's field UA list on #699. Adds CRAWLER-EXCLUSIVE tokens
to both the nginx UA regex and SOCIAL_CRAWLER_PATTERNS (kept in sync):
Cardyb (Bluesky's actual link-card fetcher), facebookcatalog, Signal,
Misskey, Pleroma, Synapse, Nextcloud, Rocket.Chat, kakaotalk-scrap,
Google-PageRenderer, OdklBot, ZoomBot.

Deliberately NOT added: UAs shared with real human in-app browsers
(WeChat MicroMessenger, LINE 'Line/', Zalo) and broad strings
('InAppBrowser', 'preview', 'unfurl', 'XING' → matches 'boxing'). Our OG
response is meta-only with no redirect, so matching those would serve a
human the bare stub. New negative test locks that exclusion in.

Verified: nginx -t passes; live harness confirms the new tokens rewrite
to /og while the in-app-browser UAs still get the SPA. Backend suite 15/15.
This commit is contained in:
Paul Nothaft
2026-07-05 23:37:16 +02:00
parent 0dffe0ce92
commit a0a28a4777
3 changed files with 45 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files
backend/src/__tests__/galleryOgService.shareImage.test.js
+21 -1
View File
@@ -343,14 +343,34 @@ describe('isSocialCrawler — extended bot coverage (#521)', () => {
// 3rd-party preview services used by business-messaging stacks
'LinkPreview/1.0',
'Slack-ImgProxy/1.0',
// Viber link-preview fetcher (#699 follow-up)
// Viber + broader crawler set (#699 follow-up)
'Mozilla/5.0 (compatible; Viber)',
'Mozilla/5.0 (compatible; Bluesky Cardyb/1.1)',
'facebookcatalog/1.0',
'kakaotalk-scrap/1.0',
'Mozilla/5.0 (compatible; Synapse/1.98)',
'Rocket.Chat/6.0',
];
for (const ua of knownBots) {
expect(isSocialCrawler(ua)).toBe(true);
}
});
it('does NOT match human in-app-browser UAs (our OG response is meta-only, no redirect)', () => {
// These share a token with a preview bot but are also sent by real users
// browsing inside the app's webview — matching them would serve a human
// the bare OG stub. Deliberately excluded; guard against re-adding them.
const inAppBrowsers = [
'Mozilla/5.0 (iPhone) AppleWebKit MicroMessenger/8.0.0', // WeChat in-app
'Mozilla/5.0 (iPhone) AppleWebKit Line/13.0.0', // LINE in-app
'Mozilla/5.0 (Linux; Android) Zalo', // Zalo in-app
'Mozilla/5.0 (Macintosh) Chrome/120.0 Safari/537.36 boxing', // "XING" substring trap
];
for (const ua of inAppBrowsers) {
expect(isSocialCrawler(ua)).toBe(false);
}
});
it('does not match a regular browser UA', () => {
const browsers = [
'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0 Safari/537.36',