paul/picpeak
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

fix(og): broaden social-crawler coverage (Bluesky Cardyb, WeChat-scraper, fediverse, etc.)

Browse Source 
From alexvaltchev's field UA list on #699. Adds CRAWLER-EXCLUSIVE tokens
to both the nginx UA regex and SOCIAL_CRAWLER_PATTERNS (kept in sync):
Cardyb (Bluesky's actual link-card fetcher), facebookcatalog, Signal,
Misskey, Pleroma, Synapse, Nextcloud, Rocket.Chat, kakaotalk-scrap,
Google-PageRenderer, OdklBot, ZoomBot.

Deliberately NOT added: UAs shared with real human in-app browsers
(WeChat MicroMessenger, LINE 'Line/', Zalo) and broad strings
('InAppBrowser', 'preview', 'unfurl', 'XING' → matches 'boxing'). Our OG
response is meta-only with no redirect, so matching those would serve a
human the bare stub. New negative test locks that exclusion in.

Verified: nginx -t passes; live harness confirms the new tokens rewrite
to /og while the in-app-browser UAs still get the SPA. Backend suite 15/15.
This commit is contained in:
Paul Nothaft
2026-07-05 23:37:16 +02:00
parent 0dffe0ce92
commit a0a28a4777
3 changed files with 45 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files
backend/src/services/galleryOgService.js
+19 -1
View File
@@ -38,7 +38,25 @@ const SOCIAL_CRAWLER_PATTERNS = [
// Viber's link-preview fetcher — was never detected, so shared links
// showed no rich preview in Viber (#699 follow-up). Keep in sync with the
// UA list in frontend/nginx.conf.
/Viber/i
/Viber/i,
// Broader crawler coverage (#699 follow-up, from alexvaltchev's field list).
// IMPORTANT: only CRAWLER-EXCLUSIVE tokens are added here. Our OG response is
// meta-only (no client redirect), so a UA shared with a real human in-app
// browser would serve that human the bare stub. That rules out WeChat
// (MicroMessenger), LINE (Line/), Zalo, and generic strings like
// "InAppBrowser"/"preview"/"unfurl" — deliberately NOT added.
/Cardyb/i, // Bluesky's link-card service (the actual fetcher UA)
/facebookcatalog/i, // Facebook catalog crawler
/Signal/i, // Signal link preview
/Misskey/i, // fediverse (server-side preview fetch)
/Pleroma/i, // fediverse
/Synapse/i, // Matrix homeserver URL preview
/Nextcloud/i, // Nextcloud Talk/News link crawler
/Rocket\.Chat/i, // Rocket.Chat server preview
/kakaotalk-scrap/i, // KakaoTalk's scraper (NOT the in-app browser UA)
/Google-PageRenderer/i, // Google Chat previews (not Search)
/OdklBot/i, // Odnoklassniki
/ZoomBot/i // Zoom Team Chat
];
function isSocialCrawler(userAgent) {