01
Scraped catalog data
Botnets scraping prices and listings and overloading backend databases.
Scraping, bot abuse, credential stuffing and DDoS against your API
API rate limiting and scraping protection at the edge
Protect your APIs from scraping, brute-force bots and floods with WAF rules at the edge and per-key rate limits in Redis.
Symptoms
Signs your platform has this problem
If several of these sound familiar, the plan below is where we would start.
02
Credential stuffing
Leaked password lists tested against login endpoints, saturating database CPU.
03
Volumetric floods
Sudden request floods overwhelming proxies and taking the API down for real users.
Remediation plan
How we fix it, step by step
Each phase ends with a measurement, so you can see what changed before the next one starts.
01
WAF and bot defense at the edge
Cloudflare WAF and bot rules, including TLS fingerprinting (JA3/JA4) and challenges, across its edge network.
02
Sliding-window rate limits
Atomic Lua scripts in Redis that track allowances per IP and per API key.
03
Tiered customer limits
Limits based on each customer's plan, returned in standard rate-limit headers.
04
Blocking and alerting
Blocking abusive IPs automatically and streaming alerts to your SIEM.
Technical checklist
Remediation checklist
What we check before a change goes to production:
- Deploy WAF rules with managed bot challenges
- Implement sliding-window rate limits in Redis with atomic Lua scripts
- Return rate-limit and Retry-After headers on 429 responses
- Fail open on a short timeout so the limiter can never block valid traffic on its own
What we measure
We take a baseline first and report the same measurements after each change, from your own monitoring — evidence, not promised results.
- Blocked traffic
- Bot and abusive requests stopped at the edge, per day
- Limiter overhead
- Added latency on valid requests at p99
- False positives
- Legitimate clients rate-limited, from support tickets and logs
Related service
Cloud & DevOps
Cloud cost optimization, Kubernetes platforms, and CI/CD that make deploys boring — savings and reliability measured in your dashboards, not our deck.
Explore Cloud & DevOpsQuestions
Questions about this remediation
The limiter fails open: if Redis doesn't answer within a short timeout, requests are allowed rather than blocked, so the limiter can't cause an outage. The edge WAF rules keep working regardless.
We look at authenticated API keys, TLS fingerprints, user-agent anomalies and request frequency over a sliding window.
Related playbooks
Other engineering problem playbooks
RAG retrieval accuracy and chunking optimization
Improve retrieval with better chunking, hybrid BM25 and vector search, and cross-encoder reranking — measured on an evaluation set built from your own questions.
Commercial LLM APIs to self-hosted models on your own GPUs
When token volume is high and steady, serving an open-weight model with vLLM on your own GPUs can cost less than API pricing. We test quality on your prompts first, then move traffic gradually.
JavaScript to strict TypeScript code modernization
Move a JavaScript codebase to strict TypeScript module by module, so data-shape bugs are caught at compile time instead of in production.
Interaction to Next Paint (INP) and main-thread fixes
Find the interactions that block the main thread, break up the long tasks behind them, and confirm the improvement in field INP data.
Want an engineer to look at this with you?
Send us the symptoms and any metrics you have. We'll reply within one business day, set up a call and agree what to measure before anything changes.