AI Glossary / Techniques
Web scraping.
Also known as: scraping, scrapers
Web scraping is pulling the actual content off a web page. Claude can search the web on its own, but it can't reach in and extract what's on a page. Scraping fixes that.
You point a tool at a URL and it hands back the page as clean Markdown, raw HTML, or a screenshot. That's the part Claude can then read and work with.
It comes up everywhere. Auditing a landing page, grabbing a competitor's branding, pulling research, feeding a pile of pages into an agent. Anything where you need what's on a site, not just a link to it.
Tools like Firecrawl and Apify do the work. You plug one in and your agent goes from "I can see this exists" to "I have the whole thing in front of me."