Need help with surfer?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

henrylee2cn
207 Stars 62 Forks Apache License 2.0 52 Commits 1 Opened issues

Description

Package surfer is a high level concurrency http client. It has surf andphantom download engines, highly simulated browser behavior, the function of analog login and so on.

Services available

!
?

Need anything else?

Contributors list

# 62,459
Go
Shell
spider
Iris
21 commits
# 4,426
Go
spider
golang
slack
21 commits
# 2,829
rust-la...
golang
wechat
bilibil...
2 commits
# 12,606
Go
go-libr...
golang
Kuberne...
1 commit

Surfer GitHub release report card github issues github closed issues GoDoc view Go大数据

Package surfer is a high level concurrency http client. It has

surf
and
phantom
download engines, highly simulated browser behavior, the function of analog login and so on.

简体中文

Features

  • Both
    surf
    and
    phantomjs
    engines are supported
  • Support random User-Agent
  • Support cache cookie
  • Support http/https

Usage

package main

import ( "github.com/henrylee2cn/surfer" "io/ioutil" "log" )

func main() { // Use surf engine resp, err := surfer.Download(&surfer.Request{ Url: "http://github.com/henrylee2cn/surfer", }) if err != nil { log.Fatal(err) } b, err := ioutil.ReadAll(resp.Body) log.Println(string(b), err)

// Use phantomjs engine
surfer.SetPhantomJsFilePath("Path to phantomjs.exe")
resp, err = surfer.Download(&surfer.Request{
    Url:          "http://github.com/henrylee2cn",
    DownloaderID: 1,
})
if err != nil {
    log.Fatal(err)
}
b, err = ioutil.ReadAll(resp.Body)
log.Println(string(b), err)

resp.Body.Close()
surfer.DestroyJsFiles()

}

Full example

License

Surfer is under Apache v2 License. See the LICENSE file for the full license text.

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.