video-stream-merger

by t-mullen

Merge multiple video MediaStreams into one composite.

170 Stars 44 Forks Last release: Not found MIT License 199 Commits 0 Releases

Available items

No Items, yet!

The developer of this repository has not created any items for sale yet. Need a bug fixed? Help with integration? A different license? Create a request here:

video-stream-merger

JavaScript Style Guide

Merges the video of multiple MediaStreams. Also merges the audio via the WebAudio API.

  • Send multiple videos over a single WebRTC MediaConnection.
  • Hotswap streams without worrying about renegotation or delays.
  • Crop, scale, and rotate live video.
  • Add creative effects through the canvas API.

Demo

P2P Demo

Existing Files Demo

Check out WBS, which uses this package.

install

npm install video-stream-merger

or


usage

Let's first get two media streams. One from the webcam, and another a screen capture. ```javascript const getusermedia = require('getusermedia') const screenRecord = require('screen-record')

getusermedia({video: true, audio:true}, (err, webcamStream) => { screenRecord(window, (err, sourceId, constraints) => { getusermedia(constraints, (err, screenStream) => { // We now have 2 streams: webcamStream, screenStream }) }) }) ```

We want to overlay the webcam stream in the corner of the screen stream.
```javascript var VideoStreamMerger = require('video-stream-merger')

var merger = new VideoStreamMerger()

// Add the screen capture. Position it to fill the whole stream (the default) merger.addStream(screenStream, { x: 0, // position of the topleft corner y: 0, width: merger.width, height: merger.height, mute: true // we don't want sound from the screen (if there is any) })

// Add the webcam stream. Position it on the bottom left and resize it to 100x100. merger.addStream(webcamStream, { x: 0, y: merger.height - 100, width: 100, height: 100, mute: false })

// Start the merging. Calling this makes the result available to us merger.start()

// We now have a merged MediaStream! merger.result ```

API

merger = new VideoStreamMerger([opts])

Create a new video merger.

Optional

opts
defaults to the below:
{
  width: 400,   // Width of the output video
  height: 300,  // Height of the output video
  fps: 25,       // Video capture frames per second
  clearRect: true, // Clear the canvas every frame
  audioContext: null, // Supply an external AudioContext (for audio effects)
}

merger.addStream(mediaStream|id, [opts])

Add a MediaStream to be merged. Use an

id
string if you only want to provide an effect.

The order that streams are added matters. Streams placed earlier will be behind later streams (use the

index
option to change this behaviour.)

Optional

opts
defaults to the below:
{
  x: 0, // position of the top-left corner
  y: 0,
  width: ,     // size to draw the stream
  height: ,
  index: 0, // Layer on which to draw the stream (0 is bottom, 1 is above that, and so on)
  mute: false,  // if true, any audio tracks will not be merged
  draw: null,    // A custom drawing function (see below)
  audioEffect: null // A custom WebAudio effect (see below)
}

merger.removeStream(mediaStream|id)

Remove a MediaStream from the merging. You may also use the ID of the stream.

If you have added the same MediaStream multiple times, all instances will be removed.

merger.addMediaElement(id, mediaElement, opts)

A convenience function to merge a HTML5 MediaElement instead of a MediaStream.

id
is a string used to remove or update the index of the stream later.

mediaElement
is a playing HTML5 Audio or Video element.

opts
are identical to the opts for
addStream
.

Streams from MediaElements can be removed via

merger.removeStream(id)
.

merger.updateIndex(mediaStream|id, newIndex)

Update the z-index (draw order) of an already added stream or data object. Identical to the

index
option.

If you have added the same MediaStream multiple times, all instances will be updated.

merger.setOutputSize(width, height)

Change the size of the canvas and the output video track.

Automatically updates

merger.width
and
merger.height
.

merger.start()

Start the merging and create

merger.result
.

You can call this any time, but you only need to call it once.

You will still be able to add/remove streams and the result stream will automatically update.

merger.result

The resulting merged MediaStream. Only available after calling

merger.start()

Never has more than one Audio and one Video track.

merger.destroy()

Clean up everything and destroy the result stream.

merger.getAudioContext()

Get the WebAudio AudioContext being used by the merger.

merger.getAudioDestination()

Get the MediaStreamDestination node that is used by the merger.

Hot-Swapping Streams (P2P Streaming)

This library makes it easy to change streams in a WebRTC connection without needing to renegotiate.

The result MediaStream will appear to be constant and stable, no matter what streams you add/remove!

P2P Streaming Demo

getusermedia({video: true, audio:true}, (err, webcamStream) => {
  const merger = new VideoStreamMerger()
  merger.start()
  players[0].srcObject = merger.result
  players[0].play()

const peer1 = new SimplePeer({initiator: true, stream:merger.result}) const peer2 = new SimplePeer()

peer1.on('signal', (data) => { peer2.signal(data) }) peer2.on('signal', (data) => { peer1.signal(data) })

peer2.on('stream', (stream) => { players[1].srcObject = stream })

const clones = []

shareWebCamStream.addEventListener('click', () => { clones.push(webcamStream.clone()) merger.addStream(clones[clones.length-1]) }) removeWebCamStream.addEventListener('click', () => { merger.removeStream(clones.pop()) }) })

Custom Draw Function

If sizing and positioning aren't enough, you can directly draw the video frames by passing a function to the

draw
option.
merger.addStream(mediaStream, {
  draw: (ctx, frame, done) => {
    // You can do whatever you want with this canvas context
    ctx.drawImage(frame, 0, 0, merger.width, merger.height)
    done()
  })
})

See the bottom example of the Live Demo to see this in action.

Custom WebAudio Effects

You can also take direct control over how audio streams are merged, and apply effects.

merger.addStream(mediaStream, {
  audioEffect: (sourceNode, destinationNode) => {
    // sourceNode is the input streams audio (microphone, etc)
    // destinationNode is the output streams audio

sourceNode.connect(destinationNode) // The default effect, simply merges audio

}) })

Both the

draw
and
audioEffect
options can be used without any MediaStream at all. Just pass a string instead.

Sponsors

Support this project by becoming a sponsor. Your logo will appear here with a link to your website. [Become a sponsor]

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.