Need help with project-bodypix?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

google-coral
133 Stars 25 Forks Apache License 2.0 6 Commits 12 Opened issues

Description

BodyPix model demo application for Google Coral

Services available

!
?

Need anything else?

Contributors list

Coral BodyPix

BodyPix is an open-source machine learning model which allows for person and body-part segmentation. This has previously been released as a Tensorflow.Js project.

This repo contains a set of pre-trained BodyPix Models (with both MobileNet v1 and ResNet50 backbones) that are quantized and optimized for the Coral Edge TPU. Example code is provided to enable inferencing on generic platforms as well as an optimized version for the Coral Dev Board.

Body-Part Segmentation

Anonymous Population Flow

The above images show two possible applications of BodyPix. The left shows body-part segmentation (on an example video) with bounding boxes and PoseNet-style skeletons. The right shows anonymous population flow. Both are running on the Coral Dev Board; see below for information on enabling these modes on the Dev Board or on a generic platform.

What is Person/Body-Part Segmentation?

Image segmentation refers to grouping pixels of an image into semantic areas, typically to locate objects and boundaries. For example, the Coral DeepLab model (available on the Coral Models Page) segments based on 20 objects. In this example, as with all segmentation examples, pixels are classified as one of those objects or background.

BodyPix extends this concept and segments for people as well as twenty-four body parts (such as "right hand" or "torso front"). More information can be found on the Tensorflow.Js page. This model and post-processing (contained as a custom OP in the Edge TPU TFLite Interpreter) has been optimized for the Edge TPU.

Examples in this repo

NOTE: BodyPix relies on the latest version of the Coral API and for the Dev Board the latest Mendel system image.

To install all the requirements, simply run

sh install_requirements.sh

bodypix.py

A generic BodyPix example intended to be run on multiple platforms, which has not been optimized. Note that this is not recommended for the Coral Dev Board, where the performance is poor compared to the bodypixglimx example. This example allows segmentation of a person, segmentation of body parts, as well as an anonymizer option which lets you remove the person from the camera image.

Run the base demo (using the MobileNet v1 backbone with 640x480 input) like this:

python3 bodypix.py

To segment body parts (grouped as regions as opposed to displaying all 24) instead of the entire person, pass the

--bodyparts
flag:
python3 bodypix.py --bodyparts

In this repo we have included 11 BodyPix model files using different backbone networks and supporting different input resolutions. There are significant trade-offs in these versions, MobileNet will be faster than ResNet but less accurate; larger resolutions are slower but allow a wider field of view (allowing further-away people to be processed correctly).

This can be changed with the

--model
flag. The following models are provided:
models/bodypix_mobilenet_v1_075_1024_768_16_quant_edgetpu_decoder.tflite
models/bodypix_mobilenet_v1_075_1280_720_16_quant_edgetpu_decoder.tflite
models/bodypix_mobilenet_v1_075_480_352_16_quant_edgetpu_decoder.tflite
models/bodypix_mobilenet_v1_075_640_480_16_quant_edgetpu_decoder.tflite
models/bodypix_mobilenet_v1_075_768_576_16_quant_edgetpu_decoder.tflite
models/bodypix_resnet_50_416_288_16_quant_edgetpu_decoder.tflite
models/bodypix_resnet_50_640_480_16_quant_edgetpu_decoder.tflite
models/bodypix_resnet_50_768_496_32_quant_edgetpu_decoder.tflite
models/bodypix_resnet_50_864_624_32_quant_edgetpu_decoder.tflite
models/bodypix_resnet_50_928_672_16_quant_edgetpu_decoder.tflite
models/bodypix_resnet_50_960_736_32_quant_edgetpu_decoder.tflite

You can change the camera resolution by using the

--width
and
--height
parameter. Note that in general the camera resolution should equal or exceed the input resolution of the network to get the full advantage of the higher resolution inference:
python3 bodypix.py --width 480 --height 360  # fast but low res
python3 bodypix.py --width 640 --height 480  # default
python3 bodypix.py --width 1280 --height 720 # slower but high res

If the camera and monitor are both facing you, consider adding the

--mirror
flag:
python3 bodypix.py --mirror

If your input camera supports encoded frames (h264 or JPEG) you can provide the corresponding flags to increase performance. Note these modes are mutually exclusive:

python3 bodypix.py --h264
python3 bodypix.py --jpeg

You can enable Anonymizer mode (which anonymizes the person, similar to in the Coral PoseNet Project. As opposed to the PoseNet example, instead of indicating the pose skeleton the entire outline of the person is indicated.

python3 bodypix.py --anonymize

bodypixglimx.py

This example is optimized specifically for the iMX8MQ GPU and VPU found on the Coral Dev Board. It is intended to allow real time processing and rendering on the platform (able to achieve 30 FPS even at 1280x720 resolution). The flags for input (models, camera configuration) are the same but we enable toggling between display modes with key presses instead of a flag:

python3 bodypix_gl_imx.py

The following key presses can be used to toggle various modes:

Toggle PoseNet-style Skeletons: 's'
Toggle Bounding Boxes: 'b'
Toggle Anonymizer: 'a'
Toggle Aggregated Heatmap Generation: 'h'
Toggle Body Part Segmentation: 'p'
Reset: 'r'

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.