Israeli Bank Scrapers - Get closer to your own data

What's here?

What you can find here is scrapers for all major Israeli banks and credit card companies. That's the plan at least. Currently only the following banks are supported: - Bank Hapoalim (thanks @sebikaplun) - Leumi Bank (thanks @esakal) - Discount Bank - Mizrahi Bank (thanks @baruchiro) - Otsar Hahayal Bank (thanks @matanelgabsi) - Visa Cal (thanks @nirgin) - Max (Formerly Leumi Card) - Isracard - Amex (thanks @erezd) - Union Bank (Thanks to Intuit FDP OpenSource Team @dratler,@kalinoy,@shanigad,@dudiventura and @NoamGoren) - Beinleumi (Thanks to @dudiventura from the Intuit FDP OpenSource Team) - Massad

Prerequisites

To use this you will need to have Node.js >= 10.x installed.

Getting started

To use these scrapers you'll need to install the package from npm:

sh
npm install israeli-bank-scrapers --save

import { CompanyTypes, createScraper } from 'israeli-bank-scrapers';

(async function() {
  try {
    // read documentation below for available options
    const options = {
      companyId: CompanyTypes.leumi, 
      startDate: new Date('2020-05-01'),
      combineInstallments: false,
      showBrowser: true 
    };
// read documentation below for information about credentials
const credentials = {
  username: 'vr29485',
  password: 'sometingsomething'
};

const scraper = createScraper(options);
const scrapeResult = await scraper.scrape(credentials);

if (scrapeResult.success) {
  scrapeResult.accounts.forEach((account) => {
    console.log(`found ${account.txns.length} transactions for account number ${account.accountNumber}`);
  });
}
else {
  throw new Error(scrapeResult.errorType);
}
  } catch(e) {
    console.error(scraping failed for the following reason: ${e.message});
  }
})();

Check the options declaration here for available options.

Regarding credentials, you should provide the relevant credentials for the chosen company. See this file with list of credentials per company.

The structure of the result object is as follows:
```node
{
  success: boolean,
  accounts: [{
    accountNumber: string,
    balance?: number, // Account balance. Not implemented for all accounts.
    txns: [{
      type: string, // can be either 'normal' or 'installments'
      identifier: int, // only if exists
      date: string, // ISO date string
      processedDate: string, // ISO date string
      originalAmount: double,
      originalCurrency: string,
      chargedAmount: double,
      description: string,
      memo: string, // can be null or empty
      installments: { // only if exists
        number: int, // the current installment number
        total: int, // the total number of installments
      },
      status: string //can either be 'completed' or 'pending'
    }],
  }],
  errorType: "INVALID_PASSWORD"|"CHANGE_PASSWORD"|"ACCOUNT_BLOCKED"|"UNKNOWN_ERROR"|"TIMEOUT"|"GENERIC", // only on success=false
  errorMessage: string, // only on success=false
}

You can also use the

SCRAPERS

node
import { SCRAPERS } from 'israeli-bank-scrapers';

The return value is a list of scraper metadata:

node
{
  : {
    name: string, // the name of the scraper
    loginFields: [ // a list of login field required by this scraper
      '' // the name of the field
    ]
  }
}

Getting deployed version of latest changes in master

This library is currently deployed to NPM manually and not as part of automatic process. You should expect situations when code was pushed to master and wasn't deployed to NPM yet.

If you are a developer and want to access the

next

next

sh
npm install [email protected] --save

Keep in mind that although this

next

version should be stable as it passed our code review, it was deployed automatically using github action workflow without the usual tests we run manually before we deploy the official version.

Israeli-bank-scrapers-core

library

TL;DR this is the same library as the default library. The only difference is that it is using

puppeteer-core

instead of

puppeteer

which is useful if you are using frameworks like Electron to pack your application.

In most cases you will probably want to use the default library (read Getting Started section).

Israeli bank scrapers library is published twice: 1. israeli-bank-scrapers - the default variation, great for common usage as node dependency in server application or cli. 2. israeli-bank-scrapers-core - extremely useful for applications that bundle

node_modules

## Differences between default and core variations

The default variation israeli-bank-scrapers is using puppeteer which handles the installation of local chroumium on its' own. This behavior is very handy since it takes care on all the hard work figuring which chromium to download and manage the actual download process. As a side effect it increases node_modules by several hounded megabytes.

The core variation israeli-bank-scrapers-core is using puppeteer-core which is exactly the same library as

puppeteer

To install

israeli-bank-scrapers-core

sh
npm install israeli-bank-scrapers-core --save

Getting chromium version used by puppeteer-core

When using the

israeli-bank-scrapers-core

puppeteer-core

israeli-bank-scrapers-core

Please read the following to learn more about the process: 1. To get the required chromium revision use the following code: ``` import { getPuppeteerConfig } from 'israeli-bank-scrapers-core';

const chromiumVersion = getPuppeteerConfig().chromiumRevision; ```

1. Once you have the chromium revision, you can either download it manually or use other liraries like download-chromium to fetch that version. The mentioned library is very handy as it caches the download and provide useful helpers like download progress information.

   1. provide the path to chromium to the library using the option key

      executablePath

      .

Specific definitions per scraper

Bank Hapoalim scraper

This scraper expects the following credentials object:

node
const credentials = {
  userCode: ,
  password: 
};

Bank Leumi scraper

This scraper expects the following credentials object:

node
const credentials = {
  username: ,
  password: 
};

Discount scraper

This scraper expects the following credentials object:

node
const credentials = {
  id: ,
  password: ,
  num: 
};

Known Limitations

• Missing memo field

Mizrahi scraper

This scraper expects the following credentials object:

node
const credentials = {
  username: ,
  password: 
};

Beinleumi & Massad

These scrapers are essentially identical and expect the following credentials object:

node
const credentials = {
  username: ,
  password: 
};

Bank Otsar Hahayal scraper

This scraper expects the following credentials object:

node
const credentials = {
  username: ,
  password: 
};

Visa Cal scraper

This scraper expects the following credentials object:

node
const credentials = {
  username: ,
  password: 
};

Max scraper (Formerly Leumi-Card)

This scraper expects the following credentials object:

node
const credentials = {
  username: ,
  password: 
};

Isracard scraper

This scraper expects the following credentials object:

node
const credentials = {
  id: ,
  card6Digits: <6 last digits of card>
  password: 
};

Amex scraper

This scraper expects the following credentials object:

node
const credentials = {
  id: ,
  card6Digits: <6 last digits of card>
  password: 
};

Known projects

These are the projects known to be using this module: - Israeli YNAB updater - A command line tool for exporting banks data to CSVs, formatted specifically for YNAB - Israel Finance Telegram Bot - A simple telegram bot that sends notifications about new transactions and interacts with them - Automated budget tracking app - An app for automatically sending transactions from Israeli banks and credit cards to budget tracking apps - Oshi - Secure desktop app for retriving your transactions from all israeli banks and credit cards

Built something interesting you want to share here? Let me know.

License

The MIT License