by ThaWeatherman

ThaWeatherman / scrapers

Lots and lots of web scrapers

136 Stars 74 Forks Last release: Not found MIT License 28 Commits 0 Releases

Available items

No Items, yet!

The developer of this repository has not created any items for sale yet. Need a bug fixed? Help with integration? A different license? Create a request here:

Practical Webscraping

This repository aims to be a collection of examples for useful web scraping written in Python. All examples scripts are provided as is and are free to use according to the terms laid out in the LICENSE file.

Each subdirectory contains scrapers relevant to a single service. As an example, a subdirectory called "reddit" would contain scrapers for Reddit and nothing else.

In mose cases there will be two scrapers that accomplish the same thing. The only difference between the two scrapers is the set of libraries used in each. Most commonly scrapers will be written using the library Scrapy. If a second scraper exists it will be written using requests and BeautifulSoup.

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.