Progress will not be saved. This browser is blocking storage (often private browsing). You can still learn, but closing the tab loses your progress.
D4Vinci/Scrapling
🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev
- Stars
- 104,838
- +7 days
- —
- Forks
- 10,967
- Language
- Python
- License
- BSD-3-Clause
- Last push
- 17 Nov 2026
Summary
Summary written by AI from the README; it may be imprecise.This Python web scraping and crawling framework combines HTTP requests, stealth browser automation, and data parsing into a single library. It provides spiders with concurrency controls, pause and resume capabilities, automatic proxy rotation, and built-in exporters for JSON and CSV. Developers can also run an MCP server to give AI agents access to web browsing, Turnstile handling, and Markdown extraction.
Use it when
- Building web crawlers that need stealth browser automation, proxy rotation, and pause-resume support
- Setting up an MCP server so AI coding agents can fetch web pages and bypass Turnstile challenges
- Converting web page content directly into clean Markdown for RAG systems and LLM pipelines
Install
pip install "scrapling[shell]"pip install scraplingpip install "scrapling[fetchers]"Commands quoted from the repo's README. Read the README before running them.
Stars over 90 days
Collecting: 2 of 90 days so far. The chart appears after 7 days.