Skip to content

ProxymaCrawler

Proxyma's crawler. It fetches pages for someone who added your site as a source.

Its requests come from the machine that copy of Proxyma runs on. User agent:

Mozilla/5.0 (compatible; ProxymaCrawler/1.0; +https://proxyma.ai/bot)

A site crawled with JavaScript rendering on is loaded by a browser, with that browser's user agent.

It follows the Disallow rules in robots.txt for User-agent * and ProxymaCrawler. To keep it out:

User-agent: ProxymaCrawler
Disallow: /

Questions about it: contact@proxyma.ai