by redco

Universal scrapping tool, which allows you to extract data using multiple environments

210 Stars 15 Forks Last release: over 1 year ago (v0.6.1) MIT License 418 Commits 10 Releases

Available items

No Items, yet!

The developer of this repository has not created any items for sale yet. Need a bug fixed? Help with integration? A different license? Create a request here:



CircleCI (all branches) Codecov Latest Stable Version Total Downloads NPM downloads

This tool moves routine crawling process to the new level. Now it's possible to parse a web page for a moment. All you need is to specify parsing rules based on css selectors. It's so simple as Goose can do it. This library allows to parse such data types as Grid, Collections, and Simple objects. Parser has support of pagination by extension goose-paginator. Also it offers you following features: actions to interact with the page and transforms to convert parsed data to friendly format.

Goose Starter Kit

Now it's easy to start with Goose, just try to use goose-starter-kit for it.

Key features

  • Declarative approach for definition of parsing rules, actions and transformations.
  • Multi environments to run parser on the browser, PhantomJS, Chrome, JsDOM and more.
  • Clear code with the latest features of ES6.
  • Clear and consistent API with promises all the way.
  • Improved Sizzle format of selectors.
  • Ajax and multi-pages parsing modes.
  • Docker Support.
  • It's easy extendable.


yarn add goose-parser goose-chrome-environment


const Parser = require('goose-parser');
const ChromeEnvironment = require('goose-chrome-environment');

const env = new ChromeEnvironment({ url: '', });

const parser = new Parser({ environment: env });

(async function () { try { const results = await parser.parse({ actions: [ { type: 'wait', timeout: 10 * 1000, scope: '.srg>.g', parentScope: 'body' } ], rules: { scope: '.srg>.g', collection: [[ { name: 'url', scope: 'h3.r>a', attr: 'href', }, { name: 'text', scope: 'h3.r>a', } ]] } }); console.log(results); } catch (e) { console.log('Error occurred:'); console.log(e.stack); } })();


This is a special atmosphere where Parser has to be executed. The main purpose of an environment is to provide a method for evaluating JS on the page. Goose supports following environments: * PhantomJS (executes in NodeJS) * Chrome (executes in NodeJS) * JSDom (executes in NodeJS) * FireFox (coming soon) * Browser (executes in Browser)

Docker usage

For now it's available to run goose-parser as a docker service.


  • url - first param is an url to parser
  • Parsing rules [optional] - Rules to parse. It's optional, if --rules-file specified.


  • -e "DEBUG=*" - to enable debug mode and see all what happens inside the goose-parser. Reed more about debug here.
  • --rules-file - to specify rules file. Be aware that you need to mount a folder with rules as a volume to the docker container.

There are two options to run it:

Process parsing from the user input

docker run -it --rm -e "DEBUG=*,-puppeteer:*" redcode/goose-parser:chrome-1.1.3-parser-0.6.0\\
      "actions": [
          "type": "wait",
          "scope": ".g"
      "rules": {
        "scope": ".g",
        "collection": [
              "scope": ".r>a h3",
              "name": "name"
              "scope": ".r>a:eq(0)",
              "name": "link",
              "attr": "href"

Process parsing from the mounted file with parsing rules

Create a file

which contains parser rules and run following command:
docker run -it --rm --volume="`pwd`/rules:/app/rules:ro" -e "DEBUG=*,-puppeteer:*" redcode/goose-parser:chrome-1.1.3-parser-0.6.0 --rules-file="/app/rules/rules.json" ''


Based on the code you can find detailed documentation about actions and transformations

API reference - coming soon

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.