Bots: How statpx's 3-Layer Bot Detection Keeps Your Stats Clean
Search engine crawlers, AI training scrapers, SEO tools, and outright scripted abuse all hit your site alongside real visitors — and if none of it gets filtered, your session counts, bounce rate, and conversion rate all quietly drift away from reality. The Bots page (bot_traffic.php) is where statpx separates the two, using three independent detection layers before a hit ever reaches your main stats.
Layer 1 — User-Agent Signature Matching
Every request carries a User-Agent string, and a large share of bots — especially the well-behaved ones — identify themselves honestly in it. statpx matches incoming User-Agents against a maintained list of known crawler and bot signatures, including major search engines, SEO crawlers like AhrefsBot, and the newer wave of AI crawlers such as GPTBot and Claude's crawler, which now make up a meaningful share of non-human traffic on most sites. Anything matching a known signature is flagged as a bot immediately, before any of the other checks run.
Layer 2 — IP Rate Limits
Signature matching only catches bots that announce themselves. To catch the ones that spoof a normal browser User-Agent, statpx also watches request velocity per IP address. A single IP making far more requests in a short window than any real human browsing session could plausibly generate gets flagged — this is what catches scripted scraping and unlabeled crawlers that skip the honesty of layer 1 entirely.
Layer 3 — Known Bot CIDR Ranges
The third layer checks the request's source IP against published CIDR ranges belonging to known bot operators and hosting providers commonly used to run automated traffic. This catches bots running from cloud infrastructure ranges that don't bother setting a distinctive User-Agent and haven't yet tripped the rate-limit threshold, closing a gap the first two layers leave open on their own.
Reading the Bots Page
The Bots page shows a dedicated bot-request counter for the selected date range, plus a breakdown of the top bot IP addresses (with ASN/organization where available) and the top bot User-Agent strings by raw request count. This is useful for two very different reasons: confirming that your real traffic numbers elsewhere in statpx aren't inflated by crawler noise, and spotting abusive scraping patterns worth blocking at the server or firewall level rather than just filtering out of your analytics. Compare it against your overall traffic in real-time analytics if you want to see the bot vs human split as it happens rather than after the fact.
Keep your analytics free of bot noise with statpx
statpx filters bots with three independent detection layers automatically, then shows you exactly what got caught on a dedicated Bots page. Privacy-friendly, no cookies required.
Start for free →The Bottom Line
Bot traffic isn't going away — if anything, the rise of AI crawlers has added an entirely new category on top of the search engines and SEO tools that were already hitting your site constantly. Relying on a single detection method (User-Agent matching alone, for example) leaves gaps that scripted traffic slips through easily. statpx's layered approach — signatures, rate limits, and known ranges — catches far more of it, so the numbers you make decisions on reflect actual human visitors, not crawlers.