Need help with tdw?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

threedworld-mit
150 Stars 15 Forks BSD 2-Clause "Simplified" License 746 Commits 0 Opened issues

Description

ThreeDWorld simulation environment

Services available

!
?

Need anything else?

Contributors list

ThreeDWorld (TDW)

A Platform for Interactive Multi-Modal Physical Simulation

ThreeDWorld (TDW) is a platform for interactive multi-modal physical simulation. With TDW, users can simulate high-fidelity sensory data and physical interactions between mobile agents and objects in a wide variety of rich 3D environments.

Getting Started (Read this first!)

Code of Conduct

Changelog

Website

C# Code

License

How to upgrade from TDW v1.7 to v1.8

API

Commands and Output Data

| Document | Description | | ----------------------------------------------------------- | ------------------------------------------------------------ | | Command API | API for every command a controller can send to the build. | | Command API Guide | Overview of how to send commands to the build. | | Output Data | API for all output data a controller can receive from the build. |

High-Level APIs

| API | Description | | ------------------------------------------------------------ | ---------------------------------------------- | | Image dataset | Generate 1.3M photorealistic images. | | tdw_physics | Generate physics datasets. | | Magnebot | High-level Magnebot robotics API. | | Transport Challenge | A higher-level API that uses the Magnebot API. |

Python
tdw
module

Frontend

| Document | Description | | ------------------------------------------------------------ | ------------------------------------------------------------ | |

tdw
module | Overview of the Python

tdw
module. | | Controller | Base class for all controllers. | | TDWUtils | Utility class. | | AssetBundleCreator | Covert 3D models into TDW-compatible asset bundles. | | PyImpact | Generate impact sounds at runtime. | | DebugController | Child class of
Controller
that has useful debug features. | | KeyboardController | Child class of
Controller
that can listen for keyboard input. | | FloorplanController | Child class of
Controller
that creates an interior environment and populates it with objects. | | Librarian | "Librarians" hold asset bundle metadata records. | | FluidTypes | Access different NVIDIA Flex fluid types. | | Object Init Data | Wrapper classes for storing object initialization data. |
Backend

| Document | Description | | ------------------------------------------------------------ | ------------------------------------------------------------ | | Build | Helper functions for downloading the build. | | PyPi | Helper functions for checking the version of the

tdw
module on PyPi. |

Audio and Video

| Document | Description | | ------------------------------------------------------------ | ------------------------------------------------- | | Impact Sounds | Generate impact sounds at runtime using PyImpact. | | PyImpact | PyImpact API. | | Audio/Video Recording | Record audio, video, or audio+video. | | Remote rendering | How to render using xpra. |

Avatars (Agents)

| Document | Description | | ------------------------------------------------------------ | ------------------------------------------------------------ | | Observation Data | Different means of determining what an agent is observing. | | Depth Maps | How to use depth maps. | | Avatar Movement | Different techniques for moving agents. | | Magnebot | High-level Magnebot API. |

Benchmarks and Speed

| Document | Description | | ------------------------------------------------------------ | -------------------------- | | Benchmarks | Performance benchmarks. | | Performance Optimizations | Increase simulation speed. |

Examples

| Document | Description | | ------------------------------------------------------------ | ---------------------------------------------------------- | | Example Controllers | Index of all example controllers in the repo. | | Use Cases (see below) | The "use cases" section showcases "advanced" usage of TDW. |

Misc.

| Document | Description | | ------------------------------------------------------------ | ------------------------------------ | | VR | VR in TDW. | | Humanoids | Add "humanoids" and play animations. |

Physics (PhysX and Flex)

| Document | Description | | ------------------------------------------------------------ | ------------------------------------------------------------ | | Physics | Common physics problems and solutions. | | NVIDIA Flex | Add soft bodies, cloth, and fluids to TDW. | | FluidTypes | Access different NVIDIA Flex fluid types. | | Physics Determinism | Benchmark of PhysX physics determinism. | | tdw_physics | Generate a physics dataset. | | Rube Goldberg (demo) | Demo of complex physical interactions between objects, with PyImpact generation of impact sounds, set in a photorealistic scene. |

Releases

| Document | Description | | --- | --- | | C# code | Access to C# backend source code | | Releases | Release versioning in TDW. | | Freezing your code | "Freeze" your controller into a compiled executable. | | v1.6 to v1.7 | How to upgrade from TDW v1.6 to TDW v1.7 |

Remote Server

| Document | Description | | ------------------------------------------------------- | ----------------------------------------------------------- | | Docker | Create a Docker container for TDW. | | Remote rendering | How to render using xpra. | | BinaryManager | Manage multiple instances of TDW builds on a remote server. | | bash scripts | Useful bash scripts for Linux. |

Rendering and Photorealism

| Document | Description | | ------------------------------------------------------------ | ------------------------------------------------------------ | | Asset Bundle Librarians | Overview of what asset bundles are, how to add objects, scenes, materials, HDRI skyboxes, and humanoids, and how to access each asset bundle's metadata. | | Model Screenshotter | Generate images of every model in TDW. | | Material Screenshotter | Generate images of every material in TDW. | | Materials, textures, and colors | Defines materials, textures, and colors. | | Depth of Field | Prevent blurry images and increase realism. | | Depth Maps | How to use depth maps. | | Remote rendering | How to render using xpra. | | Observation Data | Different means of determining what an agent is observing. |

Robotics

| Document | Description | | ------------------------------------------------------------ | --------------------------------------------- | | Robots | Overview of robotics and the Magnebot in TDW. | | Robot Librarian | API for accessing robot metadata. | | Magnebot | High-level Magnebot API. |

Scene Setup

| Document | Description | | ------------------------------------------------------------ | ------------------------------------------------------------ | | Scene Setup | Overview of how to set up a scene. | | Asset Bundle Librarians | Overview of what asset bundles are, how to add objects, scenes, materials, HDRI skyboxes, and humanoids, and how to access each asset bundle's metadata. | | Model Librarian | Overview of how to add objects and access metadata. | | Rotation | Different means of rotating objects and agents in a scene. | | Scene Reset | How to reset a scene. | | FloorplanController | Child class of

Controller
that creates an interior environment and populates it with objects. |

TDW and 3D Objects

| Document | Description | | ------------------------------------------------------------ | ------------------------------------------------------------ | | Model Librarian | Overview of how to add objects and access metadata. | | Model Screenshotter | Generate images of every model in TDW. | | Non-free models | Access the TDW "full model library". | | Local 3D models | Add your own objects to TDW. | | ShapeNet models | Convert ShapeNET models into TDW objects. | | Composite Objects
Creating Composite Objects | Use and create "composite objects". | | AssetBundleCreator | API for the

AssetBundleCreator
class (used to convert 3D models into TDW-compatible asset bundles). | | Rotation | Different means of rotating objects and agents in a scene. |

Troubleshooting TDW

| Document | Description | | ------------------------------------------------------------ | ----------------------------------------------------- | | Debug TDW | Several strategies for debugging errors in your code. | | DebugController | API for the

DebugController
class. | | Depth of Field | Prevent blurry images and increase realism. | | Performance Optimizations | Increase simulation speed. | | OS X | Common OS X problems and solutions. | | Physics | Common physics problems and solutions. |

Use Cases

| Use Case | Description | | ------------------------------------------------------------ | ------------------------------------------ | | Image dataset | Generate 1.3M photorealistic images. | | IntPhys (demo) | Demo of how to simulate IntPhys in TDW. | | Humanoid videos | Generate a dataset of humanoid animations. | | tdw_sound20k | Generate an audio dataset. | | tdw_physics | Generate a physics dataset. | | Rube Goldberg (demo) | Demo of complex physical interactions between objects, with PyImpact generation of impact sounds, set in a photorealistic scene. | | Magnebot | High-level Magnebot API. |

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.