msgflo

by msgflo

msgflo /msgflo

Distributed Flow-Based Programming via message queues

130 Stars 13 Forks Last release: Not found MIT License 784 Commits 117 Releases

Available items

No Items, yet!

The developer of this repository has not created any items for sale yet. Need a bug fixed? Help with integration? A different license? Create a request here:

MsgFlo - Flow-Based Programming with Message Queues Build Status

Implementation of the Flow-Based Programming using message queues as the communications layer between different processes. Initial message queue transports targeted are AMQP and MQTT.

MsgFlo lets you build robust polyglot FBP systems spanning multiple computers/devices. A node can be implemented in any language, to reuse existing code, libraries and developer know-how.

In FBP each component is a black-box that processes and produces data, without knowledge about where the input data comes from, or where the output data goes. This ensures that a service is easy to change, and facilitates automated testing.

MsgFlo is designed to enable partial and gradual integration into existing systems; by using standard broker/transports, not placing restrictions on message payloads, allowing to use existing queue names, and integrating non-MsgFlo nodes seamlessly.

Status

In Production

  • Used in production at TheGrid website builder, with AMQP/RabbitMQ. 20 roles, 1'000'000 jobs/weekly+
  • Used in production in imgflo image processing server. 4 roles, 200'000 jobs/weekly+
  • Used for IoT networks at hackerspaces c-base and Bitraf, using MQTT/Mosquitto.

Client support

Tooling

  • msgflo
    executable implements the FBP runtime protocol.
  • Initial support for automated testing using fbp-spec
  • Experimental support for visually building networks using Flowhub
  • guv provides autoscaling of workers when using Heroku/AMQP.

Licence

MIT, see ./LICENSE

Documentation

Please refer to https://msgflo.org

Support

Flowhub logo

MsgFlo is a part of Flowhub, a platform for building robust IoT systems and web services.
We offer an Integrated Development Environment and consulting services.

Debugging

The msgflo executable, as well as the transport/participant library uses the debug NPM module. You can enable (all) logging using:

export DEBUG=msgflo*

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.