Fiber/coroutine support for v8 and node.
NOTE OF OBSOLESCENCE -- The author of this project recommends you avoid its use if possible. The original version of this module targeted nodejs v0.1.x in early 2011 when JavaScript on the server looked a lot different. Since then async/await, Promises, and Generators were standardized and the ecosystem as a whole has moved in that direction.
I'll continue to support newer versions of nodejs as long as possible but v8 and nodejs are extraordinarily complex and dynamic platforms. It is inevitable that one day this library will abruptly stop working and no one will be able to do anything about it.
I'd like to say thank you to all the users of fibers, your support over the years has meant a lot to me.
Fibers, sometimes called coroutines, are a powerful tool which expose an API to jump between multiple call stacks from within a single thread. This can be useful to make code written for a synchronous library play nicely in an asynchronous environment.
npm install fibers
git clone git://github.com/laverdet/node-fibers.git
cd node-fibers
npm install
Note: node-fibers uses node-gyp for building. To manually invoke the build process, you can use
node-gyp rebuild. This will put the compiled extension in
build/Release/fibers.node. However, when you do
require('fibers'), it will expect the module to be in, for example,
bin/linux-x64-v8-3.11/fibers.node. You can manually put the module here every time you build, or you can use the included build script. Either
npm installor
node build -fwill do this for you. If you are going to be hacking on node-fibers, it may be worthwhile to first do
node-gyp configureand then for subsequent rebuilds you can just do
node-gyp buildwhich will be faster than a full
npm installor
node-gyp rebuild.
If you're trying to get meteor running and you ended up at this page you're probably doing something wrong. Please uninstall all versions of NodeJS and Meteor, then start over. See meteor#5124 for more information.
If you are running 64-bit nodejs version 12.x or 14.x on Linux, OS X, or Windows (7 or later) then you should be able to install fibers from npm just fine. If you are running nodejs v10.x then you will need to use
npm install [email protected]. Older versions of nodejs will require older and older version of fibers.
(special thanks to Jeroen Janssen for his work on fibers in Windows)
If you do end up needing to compile fibers first make sure you have node-gyp installed as a global dependency (
npm install -g node-gyp), and that you have setup your build environment by following the instructions at node-gyp. Ubuntu-flavored Linux users may need to run
sudo apt-get install g++as well.
The examples below describe basic use of
Fiber, but note that it is not recommended to use
Fiberwithout an abstraction in between your code and fibers. See "FUTURES" below for additional information.
This is a quick example of how you can write sleep() with fibers. Note that while the sleep() call is blocking inside the fiber, node is able to handle other events.
$ cat sleep.js
var Fiber = require('fibers');function sleep(ms) { var fiber = Fiber.current; setTimeout(function() { fiber.run(); }, ms); Fiber.yield(); }
Fiber(function() { console.log('wait... ' + new Date); sleep(1000); console.log('ok... ' + new Date); }).run(); console.log('back in main');
$ node sleep.js wait... Fri Jan 21 2011 22:42:04 GMT+0900 (JST) back in main ok... Fri Jan 21 2011 22:42:05 GMT+0900 (JST)
Yielding execution will resume back in the fiber right where you left off. You can also pass values back and forth through yield() and run(). Again, the node event loop is never blocked while this script is running.
$ cat generator.js
var Fiber = require('fibers');var inc = Fiber(function(start) { var total = start; while (true) { total += Fiber.yield(total); } });
for (var ii = inc.run(1); ii <= 10; ii = inc.run(1)) { console.log(ii); }
$ node generator.js 1 2 3 4 5 6 7 8 9 10
Expanding on the incremental generator above, we can create a generator which returns a new Fibonacci number with each invocation. You can compare this with the ECMAScript Harmony Generator Fibonacci example.
$ cat fibonacci.js
var Fiber = require('fibers');// Generator function. Returns a function which returns incrementing // Fibonacci numbers with each call. function Fibonacci() { // Create a new fiber which yields sequential Fibonacci numbers var fiber = Fiber(function() { Fiber.yield(0); // F(0) -> 0 var prev = 0, curr = 1; while (true) { Fiber.yield(curr); var tmp = prev + curr; prev = curr; curr = tmp; } }); // Return a bound handle to
run
on this fiber return fiber.run.bind(fiber); }// Initialize a new Fibonacci sequence and iterate up to 1597 var seq = Fibonacci(); for (var ii = seq(); ii <= 1597; ii = seq()) { console.log(ii); }
$ node fibonacci.js 0 1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987 1597
Fibers are exception-safe; exceptions will continue travelling through fiber boundaries:
$ cat error.js
var Fiber = require('fibers');var fn = Fiber(function() { console.log('async work here...'); Fiber.yield(); console.log('still working...'); Fiber.yield(); console.log('just a little bit more...'); Fiber.yield(); throw new Error('oh crap!'); });
try { while (true) { fn.run(); } } catch(e) { console.log('safely caught that error!'); console.log(e.stack); } console.log('done!');
$ node error.js async work here... still working... just a little bit more... safely caught that error! Error: oh crap! at error.js:11:9 done!
Using the
Fiberclass without an abstraction in between your code and the raw API is not recommended.
Fiberis meant to implement the smallest amount of functionality in order make possible many different programming patterns. This makes the
Fiberclass relatively lousy to work with directly, but extremely powerful when coupled with a decent abstraction. There is no right answer for which abstraction is right for you and your project. Included with
node-fibersis an implementation of "futures" which is fiber-aware. Usage of this library is documented below. There are several other externally-maintained options which can be found on the wiki. You should feel encouraged to be creative with fibers and build a solution which works well with your project. For instance,
Futureis not a good abstraction to use if you want to build a generator function (see Fibonacci example above).
Using
Futureto wrap existing node functions. At no point is the node event loop blocked:
$ cat ls.js
var Future = require('fibers/future'); var fs = Future.wrap(require('fs'));Future.task(function() { // Get a list of files in the directory var fileNames = fs.readdirFuture('.').wait(); console.log('Found '+ fileNames.length+ ' files');
// Stat each file var stats = []; for (var ii = 0; ii < fileNames.length; ++ii) { stats.push(fs.statFuture(fileNames[ii])); } stats.map(function(f) { f.wait() }); // Print file size for (var ii = 0; ii < fileNames.length; ++ii) { console.log(fileNames[ii]+ ': '+ stats[ii].get().size); }
}).detach();
$ node ls.js Found 11 files bin: 4096 fibers.js: 1708 .gitignore: 37 README.md: 8664 future.js: 5833 .git: 4096 LICENSE: 1054 src: 4096 ls.js: 860 Makefile: 436 package.json: 684
The future API is designed to make it easy to move between classic callback-style code and fiber-aware waiting code:
$ cat sleep.js
var Future = require('fibers/future'), wait = Future.wait;// This function returns a future which resolves after a timeout. This // demonstrates manually resolving futures. function sleep(ms) { var future = new Future; setTimeout(function() { future.return(); }, ms); return future; }
// You can create functions which automatically run in their own fiber and // return futures that resolve when the fiber returns (this probably sounds // confusing.. just play with it to understand). var calcTimerDelta = function(ms) { var start = new Date; sleep(ms).wait(); return new Date - start; }.future(); //
$ node sleep.js Set timer for 2000ms, waited 2009msAPI DOCUMENTATION
Fiber's definition looks something like this:
/** * Instantiate a new Fiber. You may invoke this either as a function or as * a constructor; the behavior is the same. * * When run() is called on this fiber for the first time, `fn` will be * invoked as the first frame on a new stack. Execution will continue on * this new stack until `fn` returns, or Fiber.yield() is called. * * After the function returns the fiber is reset to original state and * may be restarted with another call to run(). */ function Fiber(fn) { [native code] }/**
Fiber.current
will contain the currently-running Fiber. It will beundefined
if there is no fiber (i.e. the main stack of execution).Fiber.current
./**
Fiber.yield()
will halt execution of the current fiber and return controlyield
is a reserved word in Javascript. This is normally/**
/**
/**
Future's definition looks something like this:
/**
/**
/**
err
. For example, `child_process.exec()` returns [err, stdout, stderr]
/**
wait
on stuff here/**
/**
/**
/**
/**
/**
/**
/**
/**
/**
/**
If you intend to build generators, iterators, or "lazy lists", you should be aware that all fibers must eventually unwind. This is implemented by causing yield() to throw unconditionally when the library is trying to unwind your fiber-- either because reset() was called, or all handles to the fiber were lost and v8 wants to delete it.
Something like this will, at some point, cause an infinite loop in your application:
var fiber = Fiber(function() { while (true) { try { Fiber.yield(); } catch(e) {} } }); fiber.run();
If you either call reset() on this fiber, or the v8 garbage collector decides it is no longer in use, the fiber library will attempt to unwind the fiber by causing all calls to yield() to throw. However, if you catch these exceptions and continue anyway, an infinite loop will occur.
There are other garbage collection issues that occur with misuse of fiber handles. If you grab a handle to a fiber from within itself, you should make sure that the fiber eventually unwinds. This application will leak memory:
var fiber = Fiber(function() { var that = Fiber.current; Fiber.yield(); } fiber.run(); fiber = undefined;
There is no way to get back into the fiber that was started, however it's impossible for v8's garbage collector to detect this. With a handle to the fiber still outstanding, v8 will never garbage collect it and the stack will remain in memory until the application exits.
Thus, you should take care when grabbing references to
Fiber.current.