nerf-pytorch

by yenchenlin

yenchenlin /nerf-pytorch

A PyTorch implementation of NeRF (Neural Radiance Fields) that reproduces the results.

537 Stars 69 Forks Last release: Not found MIT License 21 Commits 0 Releases

Available items

No Items, yet!

The developer of this repository has not created any items for sale yet. Need a bug fixed? Help with integration? A different license? Create a request here:

NeRF-pytorch

NeRF (Neural Radiance Fields) is a method that achieves state-of-the-art results for synthesizing novel views of complex scenes. Here are some videos generated by this repository (pre-trained models are provided below):

This project is a faithful PyTorch implementation of NeRF that reproduces the results while running 1.3 times faster. The code is based on authors' Tensorflow implementation here, and has been tested to match it numerically.

Installation

git clone https://github.com/yenchenlin/nerf-pytorch.git
cd nerf-pytorch
pip install -r requirements.txt
Dependencies (click to expand)

Dependencies

  • PyTorch 1.4
  • matplotlib
  • numpy
  • imageio
  • imageio-ffmpeg
  • configargparse

The LLFF data loader requires ImageMagick.

You will also need the LLFF code (and COLMAP) set up to compute poses if you want to run on your own real data.

How To Run?

Quick Start

Download data for two example datasets:

lego
and
fern
bash download_example_data.sh

To train a low-res

lego
NeRF:
python run_nerf.py --config configs/lego.txt
After training for 100k iterations (~4 hours on a single 2080 Ti), you can find the following video at
logs/lego_test/lego_test_spiral_100000_rgb.mp4
.


To train a low-res

fern
NeRF:
python run_nerf.py --config configs/fern.txt
After training for 200k iterations (~8 hours on a single 2080 Ti, ~21 hours on a single GTX Titan X), you can find the following video at
logs/fern_test/fern_test_spiral_200000_rgb.mp4
and
logs/fern_test/fern_test_spiral_200000_disp.mp4


More Datasets

To play with other scenes presented in the paper, download the data here. Place the downloaded dataset according to the following directory structure:

├── configs
│   ├── ...
│  
├── data
│   ├── nerf_llff_data
│   │   └── fern
│   │   └── flower  # downloaded llff dataset
│   │   └── horns   # downloaded llff dataset
|   |   └── ...
|   ├── nerf_synthetic
|   |   └── lego
|   |   └── ship    # downloaded synthetic dataset
|   |   └── ...

To train NeRF on different datasets:

python run_nerf.py --config configs/{DATASET}.txt

replace

{DATASET}
with
trex
|
horns
|
flower
|
fortress
|
lego
| etc.

To test NeRF trained on different datasets:

python run_nerf.py --config configs/{DATASET}.txt --render_only

replace

{DATASET}
with
trex
|
horns
|
flower
|
fortress
|
lego
| etc.

Pre-trained Models

You can download the pre-trained models here. Place the downloaded directory in

./logs
in order to test it later. See the following directory structure for an example:
├── logs
│   ├── fern_test
│   ├── flower_test  # downloaded logs
│   ├── trex_test    # downloaded logs

Reproducibility

Tests that ensure the results of all functions and training loop match the official implementation are contained in a different branch

reproduce
. One can check it out and run the tests:
git checkout reproduce
py.test

Method

NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis Ben Mildenhall*1, Pratul P. Srinivasan*1, Matthew Tancik*1, Jonathan T. Barron2, Ravi Ramamoorthi3, Ren Ng1
1UC Berkeley, 2Google Research, 3UC San Diego *denotes equal contribution

A neural radiance field is a simple fully connected network (weights are ~5MB) trained to reproduce input views of a single scene using a rendering loss. The network directly maps from spatial location and viewing direction (5D input) to color and opacity (4D output), acting as the "volume" so we can use volume rendering to differentiably render new views

Citation

Kudos to the authors for their amazing results:

@misc{mildenhall2020nerf,
    title={NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis},
    author={Ben Mildenhall and Pratul P. Srinivasan and Matthew Tancik and Jonathan T. Barron and Ravi Ramamoorthi and Ren Ng},
    year={2020},
    eprint={2003.08934},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}

However, if you find this implementation or pre-trained models helpful, please consider to cite:

@misc{lin2020nerfpytorch,
  title={NeRF-pytorch},
  author={Yen-Chen, Lin},
  howpublished={\url{https://github.com/yenchenlin/nerf-pytorch/}},
  year={2020}
}

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.