Need help with scrapers?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

ThaWeatherman
147 Stars 79 Forks MIT License 28 Commits 0 Opened issues

Description

Lots and lots of web scrapers

Services available

!
?

Need anything else?

Contributors list

# 165,588
Python
CSS
27 commits

Practical Webscraping

This repository aims to be a collection of examples for useful web scraping written in Python. All examples scripts are provided as is and are free to use according to the terms laid out in the LICENSE file.

Each subdirectory contains scrapers relevant to a single service. As an example, a subdirectory called "reddit" would contain scrapers for Reddit and nothing else.

In mose cases there will be two scrapers that accomplish the same thing. The only difference between the two scrapers is the set of libraries used in each. Most commonly scrapers will be written using the library Scrapy. If a second scraper exists it will be written using requests and BeautifulSoup.

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.