tfObjWebrtc

by webrtcHacks

webrtcHacks /tfObjWebrtc

Tensorflow Object Detection API Web Service wrapper that works on any <video> tag and WebRTC streams

138 Stars 45 Forks Last release: Not found MIT License 6 Commits 0 Releases

Available items

No Items, yet!

The developer of this repository has not created any items for sale yet. Need a bug fixed? Help with integration? A different license? Create a request here:

Tensorflow Object Detection API Web Service

This is an example of how to turn the TensorFlow Object API into a web service. A Python Flask web server is used to interact with a JavaScript a client library. The example shows how you can extract frames from WebRTC's getUserMedia, upload them to the API, and then use the canvas to display them. This allows use of the TensorFlow Object API on any HTML

element.

Please see the Computer Vision on the Web with WebRTC and TensorFlow post on webrtcHacks for a walkthrough and more details.

Example image: Detecting Cats

TensorFlow Object Detection API on a WebRTC getUserMedia stream demo video:

TensorFlow Object Detection API on a video element

TensorFlow Object Detection API on a video element demo video:

TensorFlow Object Detection API on a video element

Quick start with Docker

docker run -it -p 5000:5000 chadhart/tensorflow-object-detection:runserver

Installation

Docker Install

git clone https://github.com/webrtcHacks/tfObjWebrtc.git
cd tfObjWebrtc
docker run -it -p 5000:5000 --name tf-webrtchacks -v $(pwd):/code chadhart/tensorflow-object-detection:webrtchacks
python setup.py install
python server.py

Manual install

Follow the TensorFlow Object API install instructions. Then run the the instructions above.

Example web apps

Point your browser to: -

https://localhost:5000/local
- shows a mirrored video from a webcam -
https://localhost:5000/video
- shows object detection running on a HTML
tag

Browser support

WebRTC browsers have secure origin restrictions: - Chrome will only work on

localhost
unless you add TLS certificates to your server - Firefox will work on any domain as long as you allow it - Safari will work, but you will need to "Allow Media Capture on Insecure Domains"

These should all work fine with any other video source.

Edge is currently not supported (polyfill for

canvas.toBlob
needed)

See the webrtcHacks link for details.

API Details

Point to a

 tag to 
objDetect.js
with an id of
objDetect
. Include
data-source="myVideo"
and other optional
data-
tags to set parameters.

Example:

$xslt

Data tags:

  • data-source
    - the ID of the source
    to use. Must be specified.
  • data-uploadWidth
    - the width of the upload file. Height will automatically be calculated based on the source video's aspect ratio. Default is
    640
    .
  • data-mirror
    - mirror the boundary boxes. Used is the image is mirrored (as is usual with a local getUserMedia view). Default is
    false
    .
  • data-scoreThreshold
    - only show objects above this confidence threshold. Default is
    0.5
    • data-apiServer
      - the full URL of the TensorFlow Object Detection Web API server location. Default is
      /image
      off of the current domain - i.e.
      http://localhost:5000/image

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.