Awesome Web Scraping

List of packages, services, and manuals related to web scraping.

Topics

• Python - Python packages
• PHP - PHP packages
• Ruby - Ruby packages
• JavaScript - JavaScript packages
• Go - Go packages
• Command Line Tools - tools with a command line interface
• Web Scraping Manuals - list of articles and books teaching web scraping
• dhamaniasad / HeadlessBrowsers - list of (almost) all headless web browsers in existence
• DNS over HTTPS providers - list of DNS over HTTPS providers
• Awesome Pastebins - list of pastebin sites

Captcha Solving Services

• https://2captcha.com

Proxy Server Marketplaces

• https://www.blackhatworld.com/forums/proxies-for-sale.112/
• https://forum.antichat.com/forums/147/

Telegram Discussion Groups

• @grablab - talks in English
• @grablab_ru - talks in Russian

How to Contribute to This List

Please see the CONTRIBUTING.md document.

Credits

The list is initially based on partial data from these sources awesome-python, awesome-php, awesome-ruby, ruby-nlp, awesome-javascript