by simapple

simapple / spider

python爬虫 全球网址URL滚动提取

138 Stars 123 Forks Last release: Not found 16 Commits 0 Releases

Available items

No Items, yet!

The developer of this repository has not created any items for sale yet. Need a bug fixed? Help with integration? A different license? Create a request here:


python 爬虫


版本1 功能简述: 以hao123为入口页面,滚动爬取外链,收集网址,并记录网址上的内链和外链数目,记录title等信息

windows7 32位上测试,目前每24个小时,可收集数据为10万左右

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.