Need help with COCO-GAN?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

259 Stars 30 Forks MIT License 21 Commits 2 Opened issues


COCO-GAN: Generation by Parts via Conditional Coordinating (ICCV 2019 oral)

Services available


Need anything else?

Contributors list

# 337,341
20 commits


Authors' implementation of "COCO-GAN: Generation by Parts via Conditional Coordinating" in TensorFlow.

[Project Page] [Paper] [Paper (Full Resolution)]

0. Pre-requirements

# Install the major required packages with
conda install tensorflow-gpu==1.13.1 scipy==1.1.0 pyyaml==5.3 pillow==6.0.0 tqdm


Recommended Tensorflow versions: 1.9.~1.13.

Other Tensorflow versions may be applicable, but you may face some errors during the FID calculation.

You may directly disable FID calculation, or try to comment/uncomment different variants of

setting codes I used for different TF versions at

You may need to write your own workaround if you face errors with other TF versions.

1. Data Download and Preprocessing


You may directly run the following command to setup everythong:

python scripts/

LSUN (Bedroom)

You may directly run the following command to setup everythong:

sh scripts/


  1. You must first contact the authors of Matterport3D dataset, sign an user license agreement, then obtain the downaload script from the authors.
  2. After you obtained the
    , download the
    category and place it under
    . Please refer to the official document:
  3. Install cube2sphere library with
    pip install cube2sphere
  4. Execute the preprocessing script with
    python scripts/
    , which outputs the final images at

Note: I don't have access to the original dataset now, so I'm not entirely sure about the detailed filenames and paths. You may need to make some small modifications in



A synthesis variant of CelebA that mess-ups the alignment of faces in CelebA dataset.

python ./scripts/generate_celeba_syn --data_dir="./data/CelebA/*.jpg" --syn_type="inward"
python ./scripts/generate_celeba_syn --data_dir="./data/CelebA/*.jpg" --syn_type="outward"


I forgot how I downloaded it. I believe there was a script somewhere though...

Please manually download it from the official site:

2. Convert TF-Records

To speed up training, we process the images into tfrecords: ```

Run the one(s) you need

python ./scripts/ --dataset celeba --resolution 64 python ./scripts/ --dataset celeba --resolution 128 python ./scripts/ --dataset lsun --resolution 64 python ./scripts/ --dataset lsun --resolution 256 python ./scripts/ --dataset matterport3d python ./scripts/ --dataset celeba-hq ```

If your data is in a directory other than

, another way to do the conversion is:
python ./scripts/ --dataset celeba --resolution 64 --img_paths="./path/to/images/*.jpg"

3. Pre-calculate FID statistics

Calculation of FID requires some statistic from real data, you may disable FID calcuation in the config files.

python ./fid_utils/ --dataset lsun --data_path "./data/lsun/*" 
This will generate statistic files under

4. Training

We provide the configurations for standard COCO-GAN training on CelebA, LSUN and Matterport3D:

python --config="./configs/CelebA_128x128_N2M2S64.yaml"
python --config="./configs/LSUN_256x256_N2M2S128.yaml"
python --config="./configs/MP3D_258x768_N2M2S128.yaml"
is the number of micro patches in a macro patch,
is the macro patch size.

CelebA-HQ is not well-tested (actually, we only tune the hyperparameters for CelebA):

python --config="./configs/CelebAHQ_1024x1024_N2M2S512.yaml"

Also the extrapolation experiment (this requires a well-pretrained

python --config="./configs/LSUN_256x256_N2M2S128_Extrapolation.yaml"
Note: This experiment is a little unstable, I usually run 3 to 5 epochs and pick the best model by personal preference.

With the Patch-Guided Image Generation, we need to train an additional critic

by setting
. Here's an exmple config:
python --config="./configs/CelebA_128x128_N2M2S64_PatchGuidedGeneration.yaml"

5. Running Testing Only

(Not well-tested) (You can skip step 1~5)

Suppose you have successfully adopted either the default path (specified in the yaml files) to save checkpoints or use

to find checkpoints, you can run the following commands:
python --config="./configs/CelebA_128x128_N2M2S64.yaml" --test
The testing codes also provides two arguments:

Pretrained checkpoints

We provide following pretrained checkpoints: - CelebA64x64N2M2S32 - CelebA128x128N2M2S64 - CelebA128x128N2M2S64PatchGuidedGeneration - LSUN64x64N2M2S32 - LSUN256x256_N2M2S128

Please download them from:

Then, you can use

force_load_from_dir: "path/to/the/pretrained/directory"
argument in each of the config file to load the parameters (this does not override the hyperparameters).

Note: 1. These models are trained with Tensorflow 1.13.0, you may potentially face some loading errors if you use other Tensorflow versions. 2. The structure of the checkpoint directory is:

  +  (name that matches the `exp_name` in the yaml file, e.g., `CelebA_64x64_N2M2S32`)
  |   +---ckpt/
  |   |     +--- (place the downloaded checkpoint files here)
  |   +---images/
  |   +---(event logs)

Some Tuning Tricks While Training on Custom Datasets

(In cases you received NaN exceptions after running with the default hyperparameters)

As you have seen, the adversarial loss values are quite magnificent in COCO-GAN training (from 1e4 to 1e8, depending on the complexity of the images in the dataset). The training is sometimes unstable. And gradient penalty plays an important role in stabilizing the training. And yes, you may need to tune the gradient penalty strength a bit if there are evidences of training instability, for instance, NaNs LOL.

In order to determine the appropriate gradient penalty strength, please open the tensorboard and look into your previous training logs. Go to histogram and open the

pane. Long story short, we would like the constrain the mean of the distribution to near 1, which is also known as the 1-Lipchitz constraint. You may start to increase the
in the config to pull the distribution toward zero a bit, just a bit, otherwise, your training will saturate when the gradients are way too small. Note that, in most cases, you only need to make sure the mean of
is around 1~5 after 10K iterations, as the early stage is expectedly quite unstable.

Some Additional Notes

  • Some computers take forever to calculate FID, it is usually an unknown problem in the CPU, please switch a computer or disable FID calculation!
  • Though our implementation theretically supports (a) patch size H!=W, and (b) micro patch N!=M cases, but these functions are never tested. Please let me know (or create a pull request with a fix if you are willing) if you face any problem while using this feature!
  • In general, with smaller micro patches, as the computation graph becomes too complex for Tensorflow, it will take lots of time an GPU memory to build the graph. The performance may be improved with Pyotrch and using
  • This implementation is different to our private codebase, please kindly let me know if you found anything bizzard.
  • The coordinate generation part looks complicated since I made it generic to different coordinate designs.

Special Thanks

The following open-source repositories largely facilitate our research! - Spectral Normalization: - cGANs with Projection Discriminator (we replicate its architecture): - FID: - Gradient Penalty: - Some GAN-Utilities: - Cube to Sphere transform:


  author    = {Lin, Chieh Hubert and
               Chang, Chia{-}Che and
               Chen, Yu{-}Sheng and
               Juan, Da{-}Cheng and
               Wei, Wei and
               Chen, Hwann{-}Tzong},
  title     = {{COCO-GAN:} Generation by Parts via Conditional Coordinating},
  booktitle = {IEEE International Conference on Computer Vision (ICCV)},
  year      = {2019},

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.