surfer

by henrylee2cn

henrylee2cn / surfer

Package surfer is a high level concurrency http client. It has surf andphantom download engines, hig...

202 Stars 62 Forks Last release: about 1 year ago (v1.2.3) Apache License 2.0 52 Commits 6 Releases

Available items

No Items, yet!

The developer of this repository has not created any items for sale yet. Need a bug fixed? Help with integration? A different license? Create a request here:

Surfer GitHub release report card github issues github closed issues GoDoc view Go大数据

Package surfer is a high level concurrency http client. It has

surf
and
phantom
download engines, highly simulated browser behavior, the function of analog login and so on.

简体中文

Features

  • Both
    surf
    and
    phantomjs
    engines are supported
  • Support random User-Agent
  • Support cache cookie
  • Support http/https

Usage

package main

import ( "github.com/henrylee2cn/surfer" "io/ioutil" "log" )

func main() { // Use surf engine resp, err := surfer.Download(&surfer.Request{ Url: "http://github.com/henrylee2cn/surfer", }) if err != nil { log.Fatal(err) } b, err := ioutil.ReadAll(resp.Body) log.Println(string(b), err)

// Use phantomjs engine
surfer.SetPhantomJsFilePath("Path to phantomjs.exe")
resp, err = surfer.Download(&surfer.Request{
    Url:          "http://github.com/henrylee2cn",
    DownloaderID: 1,
})
if err != nil {
    log.Fatal(err)
}
b, err = ioutil.ReadAll(resp.Body)
log.Println(string(b), err)

resp.Body.Close()
surfer.DestroyJsFiles()

}

Full example

License

Surfer is under Apache v2 License. See the LICENSE file for the full license text.

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.