I doubt it's really Facebook creating fake accounts and using proxies, why would they keep their real UA then?
Anyhow, I occasionally suffer this same problem when happy crawlers find my site and use truesign.ai to block them, so far successfully. It detects residential proxies, fake emails and overall suspicious activity, and - something I really wanted to avoid - there are no captchas.
It had meta in the user-agent headers, it’s trivial to spoof but I’ve read of many others who are seeing the same thing. It’s not just them though, it’s also parallel web systems, and then a bunch of others who aren’t identifying themselves.
That's probably not Meta themselves, but their new agent called Muse. It's likely coming from actual individuals. Not sure why they're scraping your website though. Maybe time to turn off all the SEO stuff to make your site less discoverable?
I doubt it's really Facebook creating fake accounts and using proxies, why would they keep their real UA then?
Anyhow, I occasionally suffer this same problem when happy crawlers find my site and use truesign.ai to block them, so far successfully. It detects residential proxies, fake emails and overall suspicious activity, and - something I really wanted to avoid - there are no captchas.
How do you know it's facebook's crawler if it going through residential proxies?
It had meta in the user-agent headers, it’s trivial to spoof but I’ve read of many others who are seeing the same thing. It’s not just them though, it’s also parallel web systems, and then a bunch of others who aren’t identifying themselves.
That's probably not Meta themselves, but their new agent called Muse. It's likely coming from actual individuals. Not sure why they're scraping your website though. Maybe time to turn off all the SEO stuff to make your site less discoverable?
[flagged]