A headless,standalone webkit server which make grabing dynamic web page easier.
A headless,standalone webkit server which make grabing dynamic web page easier.
cd /dir/of/seimiAgent ./seimiagent -p 8000
SeimiAgent will start and listen on the port that you set.Than you can use any http client tools post a load reqest to SeimiAgent and get back the content which just like chrome do.Http client tools you can use: apache
httpclientof java,
curlof cmd,
httplib2of python including, but not limited to.
Only support post.Request path:
/doload-
urlyour target url
renderTimeHow long time you hope to give seimiAgent to process javascript action and document after load finashed.Milliseconds.
proxyTell SeimiAgent to use proxy.Pattern:
http|https|socket://user:[email protected]:port
postParamJson string only,tell seimiAgent you want to use http post method and pass the parameters in
postParam.
useCookieIf
useCookie==1,seimiAgent deem you want to use cookie.Default 0.
contentTypeDetermine the output format,you can choose
imgor
html.
scriptA javascript script which can operate current html document and just seem like in chrome console to execute.
uaSet your userAgent
resourceTimeoutSet resource request timeout,such as js resource etc.Default resource timeout 20000ms.
It will take a very long time to build,so it is recommended to use the premade binary file in 'Download'.
on ubuntu
sudo apt-get install build-essential g++ flex bison gperf ruby perl libsqlite3-dev libfontconfig1-dev libicu-dev libfreetype6 libssl-dev libpng-dev libjpeg-dev python libx11-dev libxext-dev
on centos
yum -y install gcc gcc-c++ make flex bison gperf ruby openssl-devel freetype-devel fontconfig-devel libicu-devel sqlite-devel libpng-devel libjpeg-devel
python build.py
Then wait or take a cup of tea.
More Doc is on his way...