fix(og): broaden social-crawler coverage (Bluesky Cardyb, WeChat-scraper, fediverse, etc.)

From alexvaltchev's field UA list on #699. Adds CRAWLER-EXCLUSIVE tokens
to both the nginx UA regex and SOCIAL_CRAWLER_PATTERNS (kept in sync):
Cardyb (Bluesky's actual link-card fetcher), facebookcatalog, Signal,
Misskey, Pleroma, Synapse, Nextcloud, Rocket.Chat, kakaotalk-scrap,
Google-PageRenderer, OdklBot, ZoomBot.

Deliberately NOT added: UAs shared with real human in-app browsers
(WeChat MicroMessenger, LINE 'Line/', Zalo) and broad strings
('InAppBrowser', 'preview', 'unfurl', 'XING' → matches 'boxing'). Our OG
response is meta-only with no redirect, so matching those would serve a
human the bare stub. New negative test locks that exclusion in.

Verified: nginx -t passes; live harness confirms the new tokens rewrite
to /og while the in-app-browser UAs still get the SPA. Backend suite 15/15.
This commit is contained in:
Paul Nothaft
2026-07-05 23:37:16 +02:00
parent 0dffe0ce92
commit a0a28a4777
3 changed files with 45 additions and 4 deletions
+5 -2
View File
@@ -251,8 +251,11 @@ server {
# backend/src/services/galleryOgService.js. WhatsAppBot / wa-bot
# and LinkPreview / Slack-ImgProxy added in #521 to catch
# business-API preview fetchers that aren't the main WhatsApp app.
# Viber added in #699 follow-up (its preview fetcher was never detected).
if ($http_user_agent ~* "(facebookexternalhit|facebot|Twitterbot|WhatsApp|WhatsAppBot|wa-bot|Slackbot|Slack-ImgProxy|TelegramBot|SkypeUriPreview|Discordbot|LinkedInBot|Pinterest|vkShare|redditbot|Embedly|iframely|Snapchat|Applebot|Mastodon|Bluesky|OpenGraph|LinkPreview|Viber)") {
# Viber + the broader set below added in #699 follow-up. Only
# CRAWLER-EXCLUSIVE tokens — the backend OG response is meta-only (no
# redirect), so UAs shared with real human in-app browsers (WeChat's
# MicroMessenger, LINE's "Line/", Zalo, "InAppBrowser") are NOT added.
if ($http_user_agent ~* "(facebookexternalhit|facebookcatalog|facebot|Twitterbot|WhatsApp|WhatsAppBot|wa-bot|Slackbot|Slack-ImgProxy|TelegramBot|SkypeUriPreview|Discordbot|LinkedInBot|Pinterest|vkShare|redditbot|Embedly|iframely|Snapchat|Applebot|Mastodon|Bluesky|Cardyb|OpenGraph|LinkPreview|Viber|Signal|Misskey|Pleroma|Synapse|Nextcloud|Rocket\.Chat|kakaotalk-scrap|Google-PageRenderer|OdklBot|ZoomBot)") {
rewrite ^ /og/gallery/$gallery_slug last;
}
try_files $uri $uri/ /index.html;