Need help with scrapers?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

153 Stars 80 Forks MIT License 28 Commits 2 Opened issues


Lots and lots of web scrapers

Services available


Need anything else?

Contributors list

# 169,194
27 commits

Practical Webscraping

This repository aims to be a collection of examples for useful web scraping written in Python. All examples scripts are provided as is and are free to use according to the terms laid out in the LICENSE file.

Each subdirectory contains scrapers relevant to a single service. As an example, a subdirectory called "reddit" would contain scrapers for Reddit and nothing else.

In mose cases there will be two scrapers that accomplish the same thing. The only difference between the two scrapers is the set of libraries used in each. Most commonly scrapers will be written using the library Scrapy. If a second scraper exists it will be written using requests and BeautifulSoup.

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.