Need help with soccerontable?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

466 Stars 95 Forks BSD 2-Clause "Simplified" License 33 Commits 3 Opened issues


Upconverting YouTube soccer videos in 3D for viewing in AR/VR devices.Soccer On Your Tabletop

Services available


Need anything else?

Contributors list

Soccer On Your Tabletop [Project Page]

Konstantinos Rematas, Ira Kemelmacher-Shlizerman, Brian Curless, Steve Seitz, in CVPR 2018

Teaser Image

Warning: I am in the process of transferring the repo, so some things probably will not work.


This repository contains:

Upconversion of YouTube Soccer videos to 3D - Camera Calibration - Player Analysis (detection/segmentation/tracking) - Player Depth estimation - Temporal Game Reconstruction

Scripts for getting training data from video games - Electronic Arts FIFA 2016 RenderDoc Depth buffer capture - Depth buffer to point cloud

Visualization tools - Example Unity project - Hololens VS Solution


These are the non "pip3 install" dependencies: - Detectron for detection and (instance) segmentation. - OpenPose for the players' pose estimation. - OpenCV 3.1 + OpenCV_contrib for image loading/edge estimation etc. I followed this guide and it worked fine. - Eigen3 for the instance segmentation - Cocoapi for mask utilities - Boost - CMake

Upconversion of YouTube Soccer videos to 3D

The pipeline contains parts written or depending on python2, python3, cython, C/C++, which makes it a bit difficult to combine everything in one system. Therefore we break it into individual parts that have specific inputs and outputs (eg png to pickle) and communicate through a python3 class that reads, processes and writes the intermediate and final results.

First, download the repo and install its dependencies ```


git clone $SOCCERCODE pip3 install -r requirements.txt

Let's start by downloading an example dataset
wget unzip


barcelona ├── images ├── 00000.jpg ├── 00001.jpg ├── ...

The original video was cropped from [YouTube](
and frames were extracted with avconv.

Run Detectron to get bounding boxes and segmentation masks

mkdir $DATADIR/detectron


cp utils/thirdpartyscripts/ ./$DETECTRON/tools/ cd $DETECTRON python2 tools/ --cfg configs/122017baselines/e2emaskrcnnR-101-FPN2x.yaml --output-dir $DATADIR/detectron --image-ext jpg --wts $DATADIR/images/ ```

Now we can run the calibration step. In the first frame we give 4 manual correspondences and afterwards the camera parameters are optimized to fit a synthetic 3D field to the lines in the image.

python3 demo/ --path_to_data $DATADIR

Next, we estimate poses, near the bounding boxes that Mask-RCNN gave. ```


export LDLIBRARYPATH=$LDLIBRARYPATH:/usr/local/cuda/lib64/ python3 demo/ --pathtodata $DATADIR --openposedir $OPENPOSEDIR ```

The estimated poses cover very well the players in terms of localization/extend/etc. We use them to make individual crops of players for every frame for further processing. We use also the poses to refine the instance segmentation.

python3 demo/ --path_to_data $DATADIR
./soccer3d/instancesegm/instancesegm --path_to_data $DATADIR/players/ --thresh 1.5 --path_to_model ./soccer3d/instancesegm/model.yml.gz

We combine the masks from Mask-RCNN and our pose-based optimization and we prepare the data for the network.

The model weights can be found here ```


python3 demo/ --pathtodata $DATADIR python3 soccer3d/soccerdepth/ --pathto_data $DATADIR/players --modelpath $MODELPATH ```

Next, we convert tje estimated depthmaps to pointclouds.

python3 demo/ --path_to_data $DATADIR

Finally we generate one mesh per frame, with the smooth position of the players, based on tracking. Note that the resolution of the mesh is reduced, so later it can easily fit into Hololens.

python3 demo/ --path_to_data $DATADIR
python3 demo/ --path_to_data $DATADIR

Just to be sure that everything is fine, we can have a simple opengl visualization ``` python3 demo/ --pathto_data $DATADIR



The unity project and the hololens visual studio solution can be downloaded below (click on the image). The project was done on Unity version 2018.1 and to run the Hololens VS solution you will need a Hololens (with a bluetooth keyboard to place the hologram properly).

Drawing Drawing
Unity project (2018.1) Hololens Visual Studio solution (VS2017 Community)

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.