The Scraping Stack I Actually Use

• Scraping is not reading pixels. It is finding where the product gets its data.
• The page is usually the loudest layer and the least useful one.
• The edge is fresh data, not clever code.

Workflow

1. Network tab first. Use the product normally, then Command-F through requests for the API shape you need.
2. Charles or MITM for iOS. Mobile clients often expose cleaner endpoints than the web app.
3. Replay the request. Strip junk headers, then understand tokens, payloads, pagination, and failure states.
4. curl-cffi for client shape. Match browser-ish headers, TLS, and HTTP behaviour when normal requests look wrong.
5. Evomi for proxies. Use sticky, region-coherent proxies when continuity matters.
6. Make it boring. Queues, cache, database, alerts. The script is not the product; the monitored pipeline is.

That is the whole game: observe the real client, copy only what matters, and turn messy public information into a pipeline that keeps working.