BYU-PCCL / holodeck

High Fidelity Simulator for Reinforcement Learning and Robotics Research.

489 Stars 27 Forks Last release: 7 months ago (v0.3.1) MIT License 827 Commits 6 Releases

Available items

No Items, yet!

The developer of this repository has not created any items for sale yet. Need a bug fixed? Help with integration? A different license? Create a request here:


Holodeck Video

Read the docs badge Build Status

Holodeck is a high-fidelity simulator for reinforcement learning built on top of Unreal Engine 4.


  • 7+ rich worlds for training agents in, and many scenarios for those worlds
  • Linux and Windows support
  • Easily extend and modify training scenarios
  • Train and control more than one agent at once
  • Simple, OpenAI Gym-like Python interface
  • High performance - simulation speeds of up to 2x real time are possible. Performance penalty only for what you need
  • Run headless or watch your agents learn

Questions? Join our Discord!


pip install holodeck

(requires >= Python 3.5)

See Installation for complete instructions (including Docker).


Usage Overview

Holodeck's interface is similar to OpenAI's Gym.

We try and provide a batteries included approach to let you jump right into using Holodeck, with minimal fiddling required.

To demonstrate, here is a quick example using the

import holodeck

Load the environment. This environment contains a UAV in a city.

env = holodeck.make("UrbanCity-MaxDistance")

You must call .reset() on a newly created environment before ticking/stepping it


The UAV takes 3 torques and a thrust as a command.

command = [0, 0, 0, 100]

for i in range(30): state, reward, terminal, info = env.step(command)

  • state
    : dict of sensor name to the sensor's value (nparray).
  • reward
    : the reward received from the previous action
  • terminal
    : indicates whether the current state is a terminal state.
  • info
    : contains additional environment specific information.

If you want to access the data of a specific sensor, import sensors and retrieving the correct value from the state dictionary:


Multi Agent-Environments

Holodeck supports multi-agent environments.

Calls to

only provide an action for the main agent, and then tick the simulation.

provides a persistent action for a specific agent, and does not tick the simulation. After an action has been provided,
will advance the simulation forward. The action is persisted until another call to

provides a different action.
import holodeck
import numpy as np

env = holodeck.make("CyberPunkCity-Follow") env.reset()

Provide an action for each agent

env.act('uav0', np.array([0, 0, 0, 100])) env.act('nav0', np.array([0, 0, 0]))

Advance the simulation

for i in range(300):

The action provided above is repeated

states = env.tick()

You can access the reward, terminal and location for a multi agent environment as follows:

task = states["uav0"]["FollowTask"]

reward = task[0] terminal = task[1] location = states["uav0"]["LocationSensor"]


comes from the scenario configuration file)

Running Holodeck Headless

Holodeck can run headless with GPU accelerated rendering. See Using Holodeck Headless


  Author = {Joshua Greaves and Max Robinson and Nick Walton and Mitchell Mortensen and Robert Pottorff and Connor Christopherson and Derek Hancock and Jayden Milne and David Wingate},
  Title = {Holodeck: A High Fidelity Simulator},
  Year = {2018},

Holodeck is a project of BYU's Perception, Cognition and Control Lab (https://pcc.cs.byu.edu/).

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.