MCP servers

D4Vinci/Scrapling

🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev

Stars
104,838
+7 days
—
Forks
10,967
Language
Python
License
BSD-3-Clause
Last push
17 Nov 2026

Summary

Summary written by AI from the README; it may be imprecise.

This Python web scraping and crawling framework combines HTTP requests, stealth browser automation, and data parsing into a single library. It provides spiders with concurrency controls, pause and resume capabilities, automatic proxy rotation, and built-in exporters for JSON and CSV. Developers can also run an MCP server to give AI agents access to web browsing, Turnstile handling, and Markdown extraction.

Use it when

  • Building web crawlers that need stealth browser automation, proxy rotation, and pause-resume support
  • Setting up an MCP server so AI coding agents can fetch web pages and bypass Turnstile challenges
  • Converting web page content directly into clean Markdown for RAG systems and LLM pipelines

Install

pip
pip install "scrapling[shell]"
pip
pip install scrapling
pip
pip install "scrapling[fetchers]"

Commands quoted from the repo's README. Read the README before running them.

Stars over 90 days

Collecting: 2 of 90 days so far. The chart appears after 7 days.

Type to search the reference, templates and lessons.