Github url


by gocolly

gocolly /colly

Elegant Scraper and Crawler Framework for Golang

11.3K Stars 961 Forks Last release: about 1 month ago (v2.1.0) Apache License 2.0 548 Commits 6 Releases

Available items

No Items, yet!

The developer of this repository has not created any items for sale yet. Need a bug fixed? Help with integration? A different license? Create a request here:


Lightning Fast and Elegant Scraping Framework for Gophers

Colly provides a clean interface to write any kind of crawler/scraper/spider.

With Colly you can easily extract structured data from websites, which can be used for a wide range of applications, like data mining, data processing or archiving.

GoDocBackers on Open Collective Sponsors on Open Collective build statusreport cardview examplesCode CoverageFOSSA StatusTwitter URL


  • Clean API
  • Fast (>1k request/sec on a single core)
  • Manages request delays and maximum concurrency per domain
  • Automatic cookie and session handling
  • Sync/async/parallel scraping
  • Caching
  • Automatic encoding of non-unicode responses
  • Robots.txt support
  • Distributed scraping
  • Configuration via environment variables
  • Extensions


func main() { c := colly.NewCollector() // Find and visit all links c.OnHTML("a[href]", func(e \*colly.HTMLElement) { e.Request.Visit(e.Attr("href")) }) c.OnRequest(func(r \*colly.Request) { fmt.Println("Visiting", r.URL) }) c.Visit("") }

See examples folder for more detailed examples.


Add colly to your



module go 1.14 require ( latest )


Bugs or suggestions? Visit the issue tracker or join


on freenode

Other Projects Using Colly

Below is a list of public, open source projects that use Colly:

If you are using Colly in a project please send a pull request to add it to the list.


This project exists thanks to all the people who contribute. [Contribute].


Thank you to all our backers! ๐Ÿ™ [Become a backer]


Support this project by becoming a sponsor. Your logo will show up here with a link to your website. [Become a sponsor]


FOSSA Status

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.