Need help with node-libcurl?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

JCMais
451 Stars 75 Forks MIT License 1.1K Commits 10 Opened issues

Description

libcurl bindings for Node.js

Services available

!
?

Need anything else?

Contributors list

node-libcurl<!-- omit in toc -->

Buy Me A Coffee
Patreon Logo
Discord Logo

NPM version license Dependencies

Travis CI Status AppVeyor CI Status Code Quality

The fastest URL transfer library for Node.js.

libcurl bindings for Node.js. libcurl official description:

libcurl is a free and easy-to-use client-side URL transfer library, supporting DICT, FILE, FTP, FTPS, Gopher, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, POP3, POP3S, RTMP, RTSP, SCP, SFTP, SMTP, SMTPS, Telnet and TFTP. libcurl supports SSL certificates, HTTP POST, HTTP PUT, FTP uploading, HTTP form based upload, proxies, cookies, user+password authentication (Basic, Digest, NTLM, Negotiate, Kerberos), file transfer resume, http proxy tunneling and more!

Quick Start

Install

npm i node-libcurl --save

or

shell
yarn add node-libcurl

Simple Request - Async / Await using curly

this API is experimental and is subject to changes without a major version bump

const { curly } = require('node-libcurl');

const { statusCode, data, headers } = await curly.get('http://www.google.com')

Any option can be passed using their

FULLNAME
or a
lowerPascalCase
format: ```javascript const querystring = require('querystring'); const { curly } = require('node-libcurl');

const { statusCode, data, headers } = await curly.post('http://httpbin.com/post', { postFields: querystring.stringify({ field: 'value', }), // can use

postFields
or
POSTFIELDS
}) ```

JSON POST example: ```javascript const { curly } = require('node-libcurl') const { data } = await curly.post('http://httpbin.com/post', { postFields: JSON.stringify({ field: 'value' }), httpHeader: [ 'Content-Type: application/json', 'Accept: application/json' ], })

console.log(data) ```

Simple Request - Using Curl class

const { Curl } = require('node-libcurl');

const curl = new Curl();

curl.setOpt('URL', 'www.google.com'); curl.setOpt('FOLLOWLOCATION', true);

curl.on('end', function (statusCode, data, headers) { console.info(statusCode); console.info('---'); console.info(data.length); console.info('---'); console.info(this.getInfo( 'TOTAL_TIME'));

this.close(); });

curl.on('error', curl.close.bind(curl)); curl.perform();

Setting HTTP headers

Pass an array of strings specifying headers

javascript
curl.setOpt(Curl.option.HTTPHEADER,
  ['Content-Type: application/x-amz-json-1.1'])

Form Submission (Content-Type: application/x-www-form-urlencoded)

const querystring = require('querystring');
const { Curl } = require('node-libcurl');

const curl = new Curl(); const close = curl.close.bind(curl);

curl.setOpt(Curl.option.URL, '127.0.0.1/upload'); curl.setOpt(Curl.option.POST, true) curl.setOpt(Curl.option.POSTFIELDS, querystring.stringify({ field: 'value', }));

curl.on('end', close); curl.on('error', close);

MultiPart Upload / HttpPost libcurl Option (Content-Type: multipart/form-data)

const { Curl } = require('node-libcurl');

const curl = new Curl(); const close = curl.close.bind(curl);

curl.setOpt(Curl.option.URL, '127.0.0.1/upload.php'); curl.setOpt(Curl.option.HTTPPOST, [ { name: 'input-name', file: '/file/path', type: 'text/html' }, { name: 'input-name2', contents: 'field-contents' } ]);

curl.on('end', close); curl.on('error', close);

Binary Data

When requesting binary data make sure to do one of these: - Pass your own

WRITEFUNCTION
(https://curl.haxx.se/libcurl/c/CURLOPT_WRITEFUNCTION.html):
javascript
curl.setOpt('WRITEFUNCTION', (buffer, size, nmemb) => {
  // something
})
- Enable one of the following flags:
javascript
curl.enable(CurlFeature.NoDataParsing)
// or
curl.enable(CurlFeature.Raw)

The reasoning behind this is that by default, the

Curl
instance will try to decode the received data and headers to utf8 strings, as can be seen here: https://github.com/JCMais/node-libcurl/blob/b55b13529c9d11fdcdd7959137d8030b39427800/lib/Curl.ts#L391

For more examples check the examples folder.

API

API documentation for the latest stable version is available at https://node-libcurl-docs.netlify.app/modules/libindex_.html.

Develop branch documentation is available at https://develop--node-libcurl-docs.netlify.app/modules/libindex_.html.

This library provides Typescript type definitions.

Almost all CURL options are supported, if you pass one that is not, an error will be thrown.

For more usage examples check the examples folder.

Special Notes

READFUNCTION
option

The buffer passed as first parameter to the callback set with the

READFUNCTION
option is initialized with the size libcurl is using in their upload buffer (which can be set with
UPLOAD_BUFFERSIZE
), this is initialized using

node::Buffer::Data(buf);
which is basically the same than
Buffer#allocUnsafe
and therefore, it has all the implications as to its correct usage: https://nodejs.org/pt-br/docs/guides/buffer-constructor-deprecation/#regarding-buffer-allocunsafe

So, be careful, make sure to return exactly the amount of data you have written to the buffer on this callback. Only that specific amount is going to be copied and handed over to libcurl.

Common Issues

See COMMON_ISSUES.md

Benchmarks

See ./benchmark

Security

See SECURITY.md

Supported Libcurl Versions

The addon is only tested against libcurl version

7.50.0
and the latest one available.

The code itself is made to compile with any version greater than

7.32.0
, any libcurl version lower than that is not supported.

For Enterprise

node-libcurl
is available as part of the Tidelift Subscription.

The maintainers of node-libcurl and thousands of other packages are working with Tidelift to deliver commercial support and maintenance for the open source dependencies you use to build your applications. Save time, reduce risk, and improve code health, while paying the maintainers of the exact dependencies you use. Learn more.

Detailed Installation

The latest version of this package has prebuilt binaries (thanks to node-pre-gyp) available for: * Node.js: Latest two versions on active LTS (see https://github.com/nodejs/Release) * Electron: Latest 3 major versions * NW.js (node-webkit): Latest 3 major (minor for nw.js case) versions

And on the following platforms: * Linux 64 bits * Mac OS X 64 bits * Windows 32 and 64 bits

Installing with

yarn add node-libcurl
or
npm install node-libcurl
should download a prebuilt binary and no compilation will be needed. However if you are trying to install on
nw.js
or
electron
additional steps will be required, check their corresponding section below.

The prebuilt binary is statically built with the following library versions, features and protocols (library versions may change between Node.js versions):

Version: libcurl/7.73.0 OpenSSL/1.1.1g zlib/1.2.11 brotli/1.0.7 zstd/1.4.9 c-ares/1.16.1 libidn2/2.1.1 libssh2/1.9.0 nghttp2/1.41.0
Protocols: dict, file, ftp, ftps, gopher, http, https, imap, imaps, ldap, ldaps, mqtt, pop3, pop3s, rtsp, scp, sftp, smb, smbs, smtp, smtps, telnet, tftp
Features: AsynchDNS, IDN, IPv6, Largefile, NTLM, NTLM_WB, SSL, libz, brotli, TLS-SRP, HTTP2, UnixSockets, HTTPS-proxy

If there is no prebuilt binary available that matches your system, or if the installation fails, then you will need an environment capable of compiling Node.js addons, which means: - python 2.7 installed - updated C++ compiler able to compile C++11, or if building Electron >= 11 / NW.js >= 0.50, C++17 (see the Electron >= 11 / NW.js >= 0.50 section below).

If you don't want to use the prebuilt binary even if it works on your system, you can pass a flag when installing:

With

npm
sh
npm install node-libcurl --build-from-source
With
yarn
sh
npm_config_build_from_source=true yarn add node-libcurl

Important Notes on Prebuilt Binaries / Direct Installation

Those notes are not important when building on Windows

The prebuilt binaries are statically linked with

brotli
,
libidn2
,
libssh2
,
openLDAP
,
OpenSSL
nghttp2
,
zstd
and
zlib
.

The

brotli
,
nghttp2
,
OpenSSL
and
zlib
versions must match the version Node.js uses, this is necessary to avoid any possible issues by mixing library symbols of different versions, since Node.js also exports some of the symbols of their deps.

In case you are building the addon yourself with the libraries mentioned above, you must make sure their version is ABI compatible with the one Node.js uses, otherwise you are probably going to hit a Segmentation Fault.

If you want to build a statically linked version of the addon yourself, you need to pass the

curl_static_build=true
flag when calling install.

If using

npm
:
sh
npm install node-libcurl --build-from-source --curl_static_build=true
If using
yarn
:
sh
npm_config_build_from_source=true npm_config_curl_static_build=true yarn add node-libcurl

The build process will use

curl-config
available on path, if you want to overwrite it to your own libcurl installation one, you can set the
curl_config_bin
variable, like mentioned above for
curl_static_build
.

And if you don't want to use

curl-config
, you can pass two extra variables to control the build process: -
curl_include_dirs
Space separated list of directories to search for header files -
curl_libraries
Space separated list of flags to pass to the linker

Missing Packages

The statically linked version currently does not have support for

GSS-API
,
SPNEGO
,
KERBEROS
,
RTMP
,
Metalink
,
PSL
and
Alt-svc
.

The scripts to build Kerberos exists on the

./scripts/ci
folder, but it was removed for two reasons: - If built with Heimdal, the addon becomes too big - If built with MIT Kerberos, the addon would be bound to their licensing terms.

Electron / NW.js

If building for a

Electron
or
NW.js
you need to pass additional parameters to the install command.

If you do not want to use the prebuilt binary, pass the

npm_config_build_from_source=true
/
--build-from-source
flag to the install command.

NW.js (aka node-webkit)

For building from source on NW.js you first need to make sure you have nw-gyp installed globally:

yarn global add nw-gyp
or
npm i -g nw-gyp

If on Windows, you also need addition steps, currently the available windelayloadhook.cc on

nw-gyp
is not working with this addon, so it's necessary to apply a patch to it. The patch can be found on `./scripts/ci/patches/windelayloadhook.cc.patch

, and should be applied to the file on
/src/windelayload_hook.cc`.

Then:

yarn

npm_config_runtime=node-webkit npm_config_target=0.38.2 yarn add node-libcurl
npm
bash
npm install node-libcurl --runtime=node-webkit --target=0.38.2 --save

where

--target
is the current version of NW.js you are using

Electron (aka atom-shell)

yarn

bash
npm_config_runtime=electron npm_config_target=$(yarn --silent electron --version) npm_config_disturl=https://www.electronjs.org/headers yarn add node-libcurl

npm

bash
npm install node-libcurl --runtime=electron --target=$(yarn --silent electron --version) --disturl=https://www.electronjs.org/headers --save

Where

--target
is the version of electron you are using, in our case, we are just using the version returned by the locally installed
electron
binary.

You can also put those args in a .npmrc file, like so:

runtime = electron
target = 5.0.1
target_arch = x64
dist_url = https://atom.io/download/atom-shell

Electron >= 11 / NW.js >= 0.50

If you are building for Electron >= 11 or NW.js >= 0.50 you need to set the build process to use the C++17 std, you can do that by passing the variable

node_libcurl_cpp_std=c++17
. The way you do that depends if you are using
npm
or
yarn
:

If using

npm
:
sh
npm install node-libcurl --build-from-source --node_libcurl_cpp_std=c++17
If using
yarn
:
sh
npm_config_build_from_source=true npm_config_node_libcurl_cpp_std=c++17 yarn add node-libcurl

Building on Linux

To build the addon on linux based systems you must have: - gcc >= 4.8 - libcurl dev files - python 2.7

If you are on a debian based system, you can get those by running:

bash
sudo apt-get install python libcurl4-openssl-dev build-essential

If you don't want to use the libcurl version shipped with your system, since it's probably very old, you can install libcurl from source, for the addon to use that libcurl version you can use the variable mentioned above,

curl_config_bin
.

In case you want some examples check the CI configuration files (

.travis.yml
,
.circleci/config.yml
) and the
scripts/ci/
folder.

Building on macOS

On macOS you must have: - macOS >= 10.12 (Sierra) - Xcode Command Line Tools

You can check if you have Xcode Command Line Tools be running:

sh
xcode-select -p

It should return their path, in case it returns nothing, you must install it by running:

sh
xcode-select --install

Xcode >= 10 | macOS >= Mojave

In case you have errors installing the addon from source, and you are using macOS version >= Mojave, check if the error you are receiving is the following one:

  CXX(target) Release/obj.target/node_libcurl/src/node_libcurl.o
  clang: error: no such file or directory: '/usr/include'

If that is the case, it's because newer versions of the Command Line Tools does not add the

/usr/include
folder by default. Check Xcode 10 release notes for details.

The

/usr/include
is now available on
$(xcrun --show-sdk-path)/usr/include
. To correctly build libcurl you then need to pass that path to the
npm_config_curl_include_dirs
environment variable:
npm_config_curl_include_dirs="$(xcrun --show-sdk-path)/usr/include" yarn add node-libcurl

Building on Windows

If installing using a prebuilt binary you only need to have the visual c++ 2017 runtime library.

If building from source, you must have: - Python 2.7 - Visual Studio >= 2017 - nasm

Python 2.7 and the Visual Studio compiler can be installed by running:

sh
npm install --global --production windows-build-tools

nasm
can be obtained from their website, which is linked above, or using chocolatey:
cinst nasm

Currently there is no support to use other libcurl version than the one provided by the curl-for-windows submodule (help is appreciated on adding this feature).

An important note about building the addon on Windows is that we have to do some "hacks" with the header files included by

node-gyp
/
nw-gyp
. The reason for that is because as we are using a standalone version of OpenSSL, we don't want to use the OpenSSL headers provided by Node.js, which are by default added to
/include/node/openssl
, so what we do is that before compilation that folder is renamed to
openssl.disabled
. After a successful installation the folder is renamed back to their original name, however if any error happens during compilation the folder will stay renamed until the addon is compiled successfully. More info on why that was needed and some context can be found on issue #164.

Getting Help

If your question is directly related to the addon or their usage, you can get help the following ways: - Post a question on

stack-overflow
and use the
node-libcurl
tag. - Join our Discord server and send your question there.

Contributing

Read CONTRIBUTING.md

Donations / Patreon

Some people have been asking if there are any means to support my work, I've created a patreon page for that: https://www.patreon.com/jonathancardoso

If you want to donate via PayPal, use the same e-mail that is available on my GitHub profile: https://github.com/JCMais

And thanks for reading till here! 😄

Originally this addon was based on the work from jiangmiao/node-curl, things have changed and most if not all code has been rewritten.

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.