Stream

Event-driven readable and writable streams for non-blocking I/O in ReactPHP.

In order to make the EventLoop easier to use, this component introduces the powerful concept of "streams". Streams allow you to efficiently process huge amounts of data (such as a multi Gigabyte file download) in small chunks without having to store everything in memory at once. They are very similar to the streams found in PHP itself, but have an interface more suited for async, non-blocking I/O.

Table of contents

• Stream usage
  • ReadableStreamInterface
  • data event
  • end event
  • error event
  • close event
  • isReadable()
  • pause()
  • resume()
  • pipe()
  • close()
  • WritableStreamInterface
  • drain event
  • pipe event
  • error event
  • close event
  • isWritable()
  • write()
  • end()
  • close()
  • DuplexStreamInterface
• Creating streams
  • ReadableResourceStream
  • WritableResourceStream
  • DuplexResourceStream
  • ThroughStream
  • CompositeStream
• Usage
• Install
• Tests
• License
• More

Stream usage

ReactPHP uses the concept of "streams" throughout its ecosystem to provide a consistent higher-level abstraction for processing streams of arbitrary data contents and size. While a stream itself is a quite low-level concept, it can be used as a powerful abstraction to build higher-level components and protocols on top.

If you're new to this concept, it helps to think of them as a water pipe: You can consume water from a source or you can produce water and forward (pipe) it to any destination (sink).

Similarly, streams can either be

• readable (such as

  STDIN

  terminal input) or
• writable (such as

  STDOUT

  terminal output) or
• duplex (both readable and writable, such as a TCP/IP connection)

Accordingly, this package defines the following three interfaces

• ReadableStreamInterface

• WritableStreamInterface

• DuplexStreamInterface

ReadableStreamInterface

The

ReadableStreamInterface

Besides defining a few methods, this interface also implements the

EventEmitterInterface

The event callback functions MUST be a valid

callable

Exception

Every implementation of this interface MUST follow these event semantics in order to be considered a well-behaving stream.

Note that higher-level implementations of this interface may choose to define additional events with dedicated semantics not defined as part of this low-level stream specification. Conformance with these event semantics is out of scope for this interface, so you may also have to refer to the documentation of such a higher-level implementation.

data event

The

data

$stream->on('data', function ($data) {
    echo $data;
});

This event MAY be emitted any number of times, which may be zero times if this stream does not send any data at all. It SHOULD not be emitted after an

end

close

The given

$data

string

string

Many common streams (such as a TCP/IP connection or a file-based stream) will emit the raw (binary) payload data that is received over the wire as chunks of

string

Due to the stream-based nature of this, the sender may send any number of chunks with varying sizes. There are no guarantees that these chunks will be received with the exact same framing the sender intended to send. In other words, many lower-level protocols (such as TCP/IP) transfer the data in chunks that may be anywhere between single-byte values to several dozens of kilobytes. You may want to apply a higher-level protocol to these low-level data chunks in order to achieve proper message framing.

end event

The

end

$stream->on('end', function () {
    echo 'END';
});

This event SHOULD be emitted once or never at all, depending on whether a successful end was detected. It SHOULD NOT be emitted after a previous

end

close

error

After the stream is ended, it MUST switch to non-readable mode, see also

isReadable()

This event will only be emitted if the end was reached successfully, not if the stream was interrupted by an unrecoverable error or explicitly closed. Not all streams know this concept of a "successful end". Many use-cases involve detecting when the stream closes (terminates) instead, in this case you should use the

close

end

close

Many common streams (such as a TCP/IP connection or a file-based stream) will emit this event if either the remote side closes the connection or a file handle was successfully read until reaching its end (EOF).

Note that this event should not be confused with the

end()

end()

error event

The

error

Exception

$server->on('error', function (Exception $e) {
    echo 'Error: ' . $e->getMessage() . PHP_EOL;
});

This event SHOULD be emitted once the stream detects a fatal error, such as a fatal transmission error or after an unexpected

data

end

error

end

close

After the stream errors, it MUST close the stream and SHOULD thus be followed by a

close

close()

isReadable()

Many common streams (such as a TCP/IP connection or a file-based stream) only deal with data transmission and do not make assumption about data boundaries (such as unexpected

data

end

If this stream is a

DuplexStreamInterface

error

close event

The

close

$stream->on('close', function () {
    echo 'CLOSED';
});

This event SHOULD be emitted once or never at all, depending on whether the stream ever terminates. It SHOULD NOT be emitted after a previous

close

After the stream is closed, it MUST switch to non-readable mode, see also

isReadable()

Unlike the

end

end

Many common streams (such as a TCP/IP connection or a file-based stream) will likely choose to emit this event after reading a successful

end

error

If this stream is a

DuplexStreamInterface

close

isWritable()

end

isReadable()

The

isReadable(): bool

This method can be used to check if the stream still accepts incoming data events or if it is ended or closed already. Once the stream is non-readable, no further

data

end

assert($stream->isReadable() === false);

$stream->on('data', assertNeverCalled());
$stream->on('end', assertNeverCalled());

A successfully opened stream always MUST start in readable mode.

Once the stream ends or closes, it MUST switch to non-readable mode. This can happen any time, explicitly through

close()

If this stream is a

DuplexStreamInterface

isWritable()

pause()

The

pause(): void

Removes the data source file descriptor from the event loop. This allows you to throttle incoming data.

Unless otherwise noted, a successfully opened stream SHOULD NOT start in paused state.

Once the stream is paused, no futher

data

end

$stream->pause();

$stream->on('data', assertShouldNeverCalled());
$stream->on('end', assertShouldNeverCalled());

This method is advisory-only, though generally not recommended, the stream MAY continue emitting

data

You can continue processing events by calling

resume()

Note that both methods can be called any number of times, in particular calling

pause()

See also

resume()

resume()

The

resume(): void

Re-attach the data source after a previous

pause()

$stream->pause();

$loop->addTimer(1.0, function () use ($stream) {
    $stream->resume();
});

Note that both methods can be called any number of times, in particular calling

resume()

pause()

See also

pause()

pipe()

The

pipe(WritableStreamInterface $dest, array $options = [])

Automatically sends all incoming data to the destination. Automatically throttles the source based on what the destination can handle.

$source->pipe($dest);

Similarly, you can also pipe an instance implementing

DuplexStreamInterface

$connection->pipe($connection);

This method returns the destination stream as-is, which can be used to set up chains of piped streams:

$source->pipe($decodeGzip)->pipe($filterBadWords)->pipe($dest);

By default, this will call

end()

end

$source->pipe($dest, array('end' => false));

Note that this only applies to the

end

error

close

$source->pipe($dest);
$source->on('close', function () use ($dest) {
    $dest->end('BYE!');
});

If the source stream is not readable (closed state), then this is a NO-OP.

$source->close();
$source->pipe($dest); // NO-OP

If the destinantion stream is not writable (closed state), then this will simply throttle (pause) the source stream:

$dest->close();
$source->pipe($dest); // calls $source->pause()

Similarly, if the destination stream is closed while the pipe is still active, it will also throttle (pause) the source stream:

$source->pipe($dest);
$dest->close(); // calls $source->pause()

Once the pipe is set up successfully, the destination stream MUST emit a

pipe

close()

The

close(): void

This method can be used to (forcefully) close the stream.

$stream->close();

Once the stream is closed, it SHOULD emit a

close

After calling this method, the stream MUST switch into a non-readable mode, see also

isReadable()

data

end

$stream->close();
assert($stream->isReadable() === false);

$stream->on('data', assertNeverCalled());
$stream->on('end', assertNeverCalled());

If this stream is a

DuplexStreamInterface

close()

isWritable()

end()

WritableStreamInterface

The

WritableStreamInterface

Besides defining a few methods, this interface also implements the

EventEmitterInterface

The event callback functions MUST be a valid

callable

Exception

Every implementation of this interface MUST follow these event semantics in order to be considered a well-behaving stream.

Note that higher-level implementations of this interface may choose to define additional events with dedicated semantics not defined as part of this low-level stream specification. Conformance with these event semantics is out of scope for this interface, so you may also have to refer to the documentation of such a higher-level implementation.

drain event

The

drain

$stream->on('drain', function () use ($stream) {
    echo 'Stream is now ready to accept more data';
});

This event SHOULD be emitted once every time the buffer became full previously and is now ready to accept more data. In other words, this event MAY be emitted any number of times, which may be zero times if the buffer never became full in the first place. This event SHOULD NOT be emitted if the buffer has not become full previously.

This event is mostly used internally, see also

write()

pipe event

The

pipe

pipe()

ReadableStreamInterface

$stream->on('pipe', function (ReadableStreamInterface $source) use ($stream) {
    echo 'Now receiving piped data';

// explicitly close target if source emits an error
$source->on('error', function () use ($stream) {
    $stream->close();
});
});
$source->pipe($stream);

This event MUST be emitted once for each readable stream that is successfully piped into this destination stream. In other words, this event MAY be emitted any number of times, which may be zero times if no stream is ever piped into this stream. This event MUST NOT be emitted if either the source is not readable (closed already) or this destination is not writable (closed already).

This event is mostly used internally, see also

pipe()

error event

The

error

Exception

$stream->on('error', function (Exception $e) {
    echo 'Error: ' . $e->getMessage() . PHP_EOL;
});

This event SHOULD be emitted once the stream detects a fatal error, such as a fatal transmission error. It SHOULD NOT be emitted after a previous

error

close

After the stream errors, it MUST close the stream and SHOULD thus be followed by a

close

close()

isWritable()

Many common streams (such as a TCP/IP connection or a file-based stream) only deal with data transmission and may choose to only emit this for a fatal transmission error once and will then close (terminate) the stream in response.

If this stream is a

DuplexStreamInterface

error

close event

The

close

$stream->on('close', function () {
    echo 'CLOSED';
});

This event SHOULD be emitted once or never at all, depending on whether the stream ever terminates. It SHOULD NOT be emitted after a previous

close

After the stream is closed, it MUST switch to non-writable mode, see also

isWritable()

This event SHOULD be emitted whenever the stream closes, irrespective of whether this happens implicitly due to an unrecoverable error or explicitly when either side closes the stream.

Many common streams (such as a TCP/IP connection or a file-based stream) will likely choose to emit this event after flushing the buffer from the

end()

end

error

If this stream is a

DuplexStreamInterface

close

isReadable()

end

isWritable()

The

isWritable(): bool

This method can be used to check if the stream still accepts writing any data or if it is ended or closed already. Writing any data to a non-writable stream is a NO-OP:

assert($stream->isWritable() === false);

$stream->write('end'); // NO-OP
$stream->end('end'); // NO-OP

A successfully opened stream always MUST start in writable mode.

Once the stream ends or closes, it MUST switch to non-writable mode. This can happen any time, explicitly through

end()

close()

If this stream is a

DuplexStreamInterface

isReadable()

write()

The

write(mixed $data): bool

A successful write MUST be confirmed with a boolean

true

Many common streams (such as a TCP/IP connection or file-based stream) may choose to buffer all given data and schedule a future flush by using an underlying EventLoop to check when the resource is actually writable.

If a stream cannot handle writing (or flushing) the data, it SHOULD emit an

error

close()

If the internal buffer is full after adding

$data

write()

false

drain

Similarly, if the the stream is not writable (already in a closed state) it MUST NOT process the given

$data

false

The given

$data

string

string

Many common streams (such as a TCP/IP connection or a file-based stream) will only accept the raw (binary) payload data that is transferred over the wire as chunks of

string

Due to the stream-based nature of this, the sender may send any number of chunks with varying sizes. There are no guarantees that these chunks will be received with the exact same framing the sender intended to send. In other words, many lower-level protocols (such as TCP/IP) transfer the data in chunks that may be anywhere between single-byte values to several dozens of kilobytes. You may want to apply a higher-level protocol to these low-level data chunks in order to achieve proper message framing.

end()

The

end(mixed $data = null): void

This method can be used to successfully end the stream, i.e. close the stream after sending out all data that is currently buffered.

$stream->write('hello');
$stream->write('world');
$stream->end();

If there's no data currently buffered and nothing to be flushed, then this method MAY

close()

If there's still data in the buffer that needs to be flushed first, then this method SHOULD try to write out this data and only then

close()

close

Note that this interface gives you no control over explicitly flushing the buffered data, as finding the appropriate time for this is beyond the scope of this interface and left up to the implementation of this interface.

Many common streams (such as a TCP/IP connection or file-based stream) may choose to buffer all given data and schedule a future flush by using an underlying EventLoop to check when the resource is actually writable.

You can optionally pass some final data that is written to the stream before ending the stream. If a non-

null

$data

write($data)

// shorter version
$stream->end('bye');

// same as longer version
$stream->write('bye');
$stream->end();

After calling this method, the stream MUST switch into a non-writable mode, see also

isWritable()

write()

end()

$stream->end();
assert($stream->isWritable() === false);

$stream->write('nope'); // NO-OP
$stream->end(); // NO-OP

If this stream is a

DuplexStreamInterface

isReadable()

data

end

pause()

resume()

Note that this method should not be confused with the

close()

close()

The

close(): void

This method can be used to forcefully close the stream, i.e. close the stream without waiting for any buffered data to be flushed. If there's still data in the buffer, this data SHOULD be discarded.

$stream->close();

Once the stream is closed, it SHOULD emit a

close

After calling this method, the stream MUST switch into a non-writable mode, see also

isWritable()

write()

end()

$stream->close();
assert($stream->isWritable() === false);

$stream->write('nope'); // NO-OP
$stream->end(); // NO-OP

Note that this method should not be confused with the

end()

end()

end()

$stream->end();
$loop->addTimer(1.0, function () use ($stream) {
    $stream->close();
});

If this stream is a

DuplexStreamInterface

close()

isReadable()

DuplexStreamInterface

The

DuplexStreamInterface

It builds on top of the existing interfaces for readable and writable streams and follows the exact same method and event semantics. If you're new to this concept, you should look into the

ReadableStreamInterface

WritableStreamInterface

Besides defining a few methods, this interface also implements the

EventEmitterInterface

ReadbleStreamInterface

WritableStreamInterface

The event callback functions MUST be a valid

callable

Exception

Every implementation of this interface MUST follow these event semantics in order to be considered a well-behaving stream.

Note that higher-level implementations of this interface may choose to define additional events with dedicated semantics not defined as part of this low-level stream specification. Conformance with these event semantics is out of scope for this interface, so you may also have to refer to the documentation of such a higher-level implementation.

See also

ReadableStreamInterface

WritableStreamInterface

Creating streams

ReactPHP uses the concept of "streams" throughout its ecosystem, so that many higher-level consumers of this package only deal with stream usage. This implies that stream instances are most often created within some higher-level components and many consumers never actually have to deal with creating a stream instance.

• Use react/socket if you want to accept incoming or establish outgoing plaintext TCP/IP or secure TLS socket connection streams.
• Use react/http if you want to receive an incoming HTTP request body streams.
• Use react/child-process if you want to communicate with child processes via process pipes such as STDIN, STDOUT, STDERR etc.
• Use experimental react/filesystem if you want to read from / write to the filesystem.
• See also the last chapter for more real-world applications.

However, if you are writing a lower-level component or want to create a stream instance from a stream resource, then the following chapter is for you.

Note that the following examples use

fopen()

and

stream_socket_client()

for illustration purposes only. These functions SHOULD NOT be used in a truly async program because each call may take several seconds to complete and would block the EventLoop otherwise. Additionally, the

fopen()

call will return a file handle on some platforms which may or may not be supported by all EventLoop implementations. As an alternative, you may want to use higher-level libraries listed above.

ReadableResourceStream

The

ReadableResourceStream

ReadableStreamInterface

This can be used to represent a read-only resource like a file stream opened in readable mode or a stream such as

STDIN

$stream = new ReadableResourceStream(STDIN, $loop);
$stream->on('data', function ($chunk) {
    echo $chunk;
});
$stream->on('end', function () {
    echo 'END';
});

See also

ReadableStreamInterface

The first parameter given to the constructor MUST be a valid stream resource that is opened in reading mode (e.g.

fopen()

r

InvalidArgumentException

// throws InvalidArgumentException
$stream = new ReadableResourceStream(false, $loop);

See also the

DuplexResourceStream

Internally, this class tries to enable non-blocking mode on the stream resource which may not be supported for all stream resources. Most notably, this is not supported by pipes on Windows (STDIN etc.). If this fails, it will throw a

RuntimeException

// throws RuntimeException on Windows
$stream = new ReadableResourceStream(STDIN, $loop);

Once the constructor is called with a valid stream resource, this class will take care of the underlying stream resource. You SHOULD only use its public API and SHOULD NOT interfere with the underlying stream resource manually.

This class takes an optional

int|null $readChunkSize

null

-1

$stream = new ReadableResourceStream(STDIN, $loop, 8192);

PHP bug warning: If the PHP process has explicitly been started without a

STDIN

stream, then trying to read from

STDIN

may return data from another stream resource. This does not happen if you start this with an empty stream like

php test.php < /dev/null

instead of

php test.php .
  See #81 for more details.

WritableResourceStream

The

WritableResourceStream

WritableStreamInterface

This can be used to represent a write-only resource like a file stream opened in writable mode or a stream such as

STDOUT

STDERR

$stream = new WritableResourceStream(STDOUT, $loop);
$stream->write('hello!');
$stream->end();

See also

WritableStreamInterface

The first parameter given to the constructor MUST be a valid stream resource that is opened for writing. Otherwise, it will throw an

InvalidArgumentException

// throws InvalidArgumentException
$stream = new WritableResourceStream(false, $loop);

See also the

DuplexResourceStream

Internally, this class tries to enable non-blocking mode on the stream resource which may not be supported for all stream resources. Most notably, this is not supported by pipes on Windows (STDOUT, STDERR etc.). If this fails, it will throw a

RuntimeException

// throws RuntimeException on Windows
$stream = new WritableResourceStream(STDOUT, $loop);

Once the constructor is called with a valid stream resource, this class will take care of the underlying stream resource. You SHOULD only use its public API and SHOULD NOT interfere with the underlying stream resource manually.

Any

write()

This class takes an optional

int|null $writeBufferSoftLimit

null

$stream = new WritableResourceStream(STDOUT, $loop, 8192);

This class takes an optional

int|null $writeChunkSize

null

-1

$stream = new WritableResourceStream(STDOUT, $loop, null, 8192);

See also

write()

DuplexResourceStream

The

DuplexResourceStream

DuplexStreamInterface

This can be used to represent a read-and-write resource like a file stream opened in read and write mode mode or a stream such as a TCP/IP connection:

$conn = stream_socket_client('tcp://google.com:80');
$stream = new DuplexResourceStream($conn, $loop);
$stream->write('hello!');
$stream->end();

See also

DuplexStreamInterface

The first parameter given to the constructor MUST be a valid stream resource that is opened for reading and writing. Otherwise, it will throw an

InvalidArgumentException

// throws InvalidArgumentException
$stream = new DuplexResourceStream(false, $loop);

See also the

ReadableResourceStream

WritableResourceStream

Internally, this class tries to enable non-blocking mode on the stream resource which may not be supported for all stream resources. Most notably, this is not supported by pipes on Windows (STDOUT, STDERR etc.). If this fails, it will throw a

RuntimeException

// throws RuntimeException on Windows
$stream = new DuplexResourceStream(STDOUT, $loop);

Once the constructor is called with a valid stream resource, this class will take care of the underlying stream resource. You SHOULD only use its public API and SHOULD NOT interfere with the underlying stream resource manually.

This class takes an optional

int|null $readChunkSize

null

-1

$conn = stream_socket_client('tcp://google.com:80');
$stream = new DuplexResourceStream($conn, $loop, 8192);

Any

write()

This class takes another optional

WritableStreamInterface|null $buffer

null

If you want to change the write buffer soft limit, you can pass an instance of

WritableResourceStream

$conn = stream_socket_client('tcp://google.com:80');
$buffer = new WritableResourceStream($conn, $loop, 8192);
$stream = new DuplexResourceStream($conn, $loop, null, $buffer);

See also

WritableResourceStream

ThroughStream

The

ThroughStream

DuplexStreamInterface

$through = new ThroughStream();
$through->on('data', $this->expectCallableOnceWith('hello'));

$through->write('hello');

Similarly, the

end()

end

close()

close()

close

pipe()

$through = new ThroughStream();
$source->pipe($through)->pipe($dest);

Optionally, its constructor accepts any callable function which will then be used to filter any data written to it. This function receives a single data argument as passed to the writable side and must return the data as it will be passed to its readable end:

$through = new ThroughStream('strtoupper');
$source->pipe($through)->pipe($dest);

Note that this class makes no assumptions about any data types. This can be used to convert data, for example for transforming any structured data into a newline-delimited JSON (NDJSON) stream like this:

$through = new ThroughStream(function ($data) {
    return json_encode($data) . PHP_EOL;
});
$through->on('data', $this->expectCallableOnceWith("[2, true]\n"));

$through->write(array(2, true));

The callback function is allowed to throw an

Exception

error

close()

$through = new ThroughStream(function ($data) {
    if (!is_string($data)) {
        throw new \UnexpectedValueException('Only strings allowed');
    }
    return $data;
});
$through->on('error', $this->expectCallableOnce()));
$through->on('close', $this->expectCallableOnce()));
$through->on('data', $this->expectCallableNever()));

$through->write(2);

CompositeStream

The

CompositeStream

DuplexStreamInterface

ReadableStreamInterface

WritableStreamInterface

This is useful for some APIs which may require a single

DuplexStreamInterface

$stdin = new ReadableResourceStream(STDIN, $loop);
$stdout = new WritableResourceStream(STDOUT, $loop);

$stdio = new CompositeStream($stdin, $stdout);
$stdio->on('data', function ($chunk) use ($stdio) {
    $stdio->write('You said: ' . $chunk);
});

This is a well-behaving stream which forwards all stream events from the underlying streams and forwards all streams calls to the underlying streams.

If you

write()

write()

If you

end()

end()

pause()

If you

close()

close

close()

Usage

The following example can be used to pipe the contents of a source file into a destination file without having to ever read the whole file into memory:

$loop = new React\EventLoop\StreamSelectLoop;

$source = new React\Stream\ReadableResourceStream(fopen('source.txt', 'r'), $loop);
$dest = new React\Stream\WritableResourceStream(fopen('destination.txt', 'w'), $loop);
$source->pipe($dest);
$loop->run();

Note that this example uses

fopen()

for illustration purposes only. This should not be used in a truly async program because the filesystem is inherently blocking and each call could potentially take several seconds. See also creating streams for more sophisticated examples.

Install

The recommended way to install this library is through Composer. New to Composer?

This project follows SemVer. This will install the latest supported version:

$ composer require react/stream:^1.1.1

See also the CHANGELOG for details about version upgrades.

This project aims to run on any platform and thus does not require any PHP extensions and supports running on legacy PHP 5.3 through current PHP 7+ and HHVM. It's highly recommended to use PHP 7+ for this project due to its vast performance improvements.

Tests

To run the test suite, you first need to clone this repo and then install all dependencies through Composer:

$ composer install

To run the test suite, go to the project root and run:

$ php vendor/bin/phpunit

The test suite also contains a number of functional integration tests that rely on a stable internet connection. If you do not want to run these, they can simply be skipped like this:

$ php vendor/bin/phpunit --exclude-group internet

License

MIT, see LICENSE file.

More

• See creating streams for more information on how streams are created in real-world applications.
• See our users wiki and the dependents on Packagist for a list of packages that use streams in real-world applications.