ThreeDWorld simulation environment
ThreeDWorld (TDW) is a platform for interactive multi-modal physical simulation. With TDW, users can simulate high-fidelity sensory data and physical interactions between mobile agents and objects in a wide variety of rich 3D environments.
| Document | Description | | ----------------------------------------------------------- | ------------------------------------------------------------ | | Command API | API for every command a controller can send to the build. | | Command API Guide | Overview of how to send commands to the build. | | Output Data | API for all output data a controller can receive from the build. |
| API | Description | | ------------------------------------------------------------ | ---------------------------------------------- | | Image dataset | Generate 1.3M photorealistic images. | | tdw_physics | Generate physics datasets. | | Magnebot | High-level Magnebot robotics API. | | Transport Challenge | A higher-level API that uses the Magnebot API. |
tdwmodule
| Document | Description | | ------------------------------------------------------------ | ------------------------------------------------------------ | |
tdwmodule | Overview of the Python
tdwmodule. | | Controller | Base class for all controllers. | | TDWUtils | Utility class. | | AssetBundleCreator | Covert 3D models into TDW-compatible asset bundles. | | PyImpact | Generate impact sounds at runtime. | | DebugController | Child class of
Controllerthat has useful debug features. | | KeyboardController | Child class of
Controllerthat can listen for keyboard input. | | FloorplanController | Child class of
Controllerthat creates an interior environment and populates it with objects. | | Librarian | "Librarians" hold asset bundle metadata records. | | FluidTypes | Access different NVIDIA Flex fluid types. | | Object Init Data | Wrapper classes for storing object initialization data. |
| Document | Description | | ------------------------------------------------------------ | ------------------------------------------------------------ | | Build | Helper functions for downloading the build. | | PyPi | Helper functions for checking the version of the
tdwmodule on PyPi. |
| Document | Description | | ------------------------------------------------------------ | ------------------------------------------------- | | Impact Sounds | Generate impact sounds at runtime using PyImpact. | | PyImpact | PyImpact API. | | Audio/Video Recording | Record audio, video, or audio+video. | | Remote rendering | How to render using xpra. |
| Document | Description | | ------------------------------------------------------------ | ------------------------------------------------------------ | | Observation Data | Different means of determining what an agent is observing. | | Depth Maps | How to use depth maps. | | Avatar Movement | Different techniques for moving agents. | | Magnebot | High-level Magnebot API. |
| Document | Description | | ------------------------------------------------------------ | -------------------------- | | Benchmarks | Performance benchmarks. | | Performance Optimizations | Increase simulation speed. |
| Document | Description | | ------------------------------------------------------------ | ---------------------------------------------------------- | | Example Controllers | Index of all example controllers in the repo. | | Use Cases (see below) | The "use cases" section showcases "advanced" usage of TDW. |
| Document | Description | | ------------------------------------------------------------ | ------------------------------------ | | VR | VR in TDW. | | Humanoids | Add "humanoids" and play animations. |
| Document | Description | | ------------------------------------------------------------ | ------------------------------------------------------------ | | Physics | Common physics problems and solutions. | | NVIDIA Flex | Add soft bodies, cloth, and fluids to TDW. | | FluidTypes | Access different NVIDIA Flex fluid types. | | Physics Determinism | Benchmark of PhysX physics determinism. | | tdw_physics | Generate a physics dataset. | | Rube Goldberg (demo) | Demo of complex physical interactions between objects, with PyImpact generation of impact sounds, set in a photorealistic scene. |
| Document | Description | | --- | --- | | C# code | Access to C# backend source code | | Releases | Release versioning in TDW. | | Freezing your code | "Freeze" your controller into a compiled executable. | | v1.6 to v1.7 | How to upgrade from TDW v1.6 to TDW v1.7 |
| Document | Description | | ------------------------------------------------------- | ----------------------------------------------------------- | | Docker | Create a Docker container for TDW. | | Remote rendering | How to render using xpra. | | BinaryManager | Manage multiple instances of TDW builds on a remote server. | | bash scripts | Useful bash scripts for Linux. |
| Document | Description | | ------------------------------------------------------------ | ------------------------------------------------------------ | | Asset Bundle Librarians | Overview of what asset bundles are, how to add objects, scenes, materials, HDRI skyboxes, and humanoids, and how to access each asset bundle's metadata. | | Model Screenshotter | Generate images of every model in TDW. | | Material Screenshotter | Generate images of every material in TDW. | | Materials, textures, and colors | Defines materials, textures, and colors. | | Depth of Field | Prevent blurry images and increase realism. | | Depth Maps | How to use depth maps. | | Remote rendering | How to render using xpra. | | Observation Data | Different means of determining what an agent is observing. |
| Document | Description | | ------------------------------------------------------------ | --------------------------------------------- | | Robots | Overview of robotics and the Magnebot in TDW. | | Robot Librarian | API for accessing robot metadata. | | Magnebot | High-level Magnebot API. |
| Document | Description | | ------------------------------------------------------------ | ------------------------------------------------------------ | | Scene Setup | Overview of how to set up a scene. | | Asset Bundle Librarians | Overview of what asset bundles are, how to add objects, scenes, materials, HDRI skyboxes, and humanoids, and how to access each asset bundle's metadata. | | Model Librarian | Overview of how to add objects and access metadata. | | Rotation | Different means of rotating objects and agents in a scene. | | Scene Reset | How to reset a scene. | | FloorplanController | Child class of
Controllerthat creates an interior environment and populates it with objects. |
| Document | Description |
| ------------------------------------------------------------ | ------------------------------------------------------------ |
| Model Librarian | Overview of how to add objects and access metadata. |
| Model Screenshotter | Generate images of every model in TDW. |
| Non-free models | Access the TDW "full model library". |
| Local 3D models | Add your own objects to TDW. |
| ShapeNet models | Convert ShapeNET models into TDW objects. |
| Composite Objects
Creating Composite Objects | Use and create "composite objects". |
| AssetBundleCreator | API for the
AssetBundleCreatorclass (used to convert 3D models into TDW-compatible asset bundles). | | Rotation | Different means of rotating objects and agents in a scene. |
| Document | Description | | ------------------------------------------------------------ | ----------------------------------------------------- | | Debug TDW | Several strategies for debugging errors in your code. | | DebugController | API for the
DebugControllerclass. | | Depth of Field | Prevent blurry images and increase realism. | | Performance Optimizations | Increase simulation speed. | | OS X | Common OS X problems and solutions. | | Physics | Common physics problems and solutions. |
| Use Case | Description | | ------------------------------------------------------------ | ------------------------------------------ | | Image dataset | Generate 1.3M photorealistic images. | | IntPhys (demo) | Demo of how to simulate IntPhys in TDW. | | Humanoid videos | Generate a dataset of humanoid animations. | | tdw_sound20k | Generate an audio dataset. | | tdw_physics | Generate a physics dataset. | | Rube Goldberg (demo) | Demo of complex physical interactions between objects, with PyImpact generation of impact sounds, set in a photorealistic scene. | | Magnebot | High-level Magnebot API. |