Need help with FDA?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

YanchaoYang
243 Stars 41 Forks 18 Commits 13 Opened issues

Description

Fourier Domain Adaptation for Semantic Segmentation

Services available

!
?

Need anything else?

Contributors list

No Data

FDA: Fourier Domain Adaptation for Semantic Segmentation.

This is the Pytorch implementation of our FDA paper published in CVPR 2020.

Domain adaptation via style transfer made easy using Fourier Transform. FDA needs no deep networks for style transfer, and involves no adversarial training. Below is the diagram of the proposed Fourier Domain Adaptation method:

Step 1: Apply FFT to source and target images.

Step 2: Replace the low frequency part of the source amplitude with that from the target.

Step 3: Apply inverse FFT to the modified source spectrum.

Image of FDA

Usage

  1. FDA Demo

python3 FDA_demo.py

An example of FDA for domain adaptation. (source: GTA5, target: CityScapes, with beta 0.01)

Image of Source

  1. Sim2Real Adaptation Using FDA (single beta)

python3 train.py --snapshot-dir='../checkpoints/FDA' --init-weights='../checkpoints/FDA/initweight/DeepLabinit.pth' --LB=0.01 --entW=0.005 --ita=2.0 --switch2entropy=0

Important: use the original images for FDA, then do mean subtraction, normalization, etc. Otherwise, will be numerical artifacts.

DeepLab initialization can be downloaded through this link.

LB: beta in the paper, controls the size of the low frequency window to be replaced.

entW: weight on the entropy term.

ita: coefficient for the robust norm on entropy.

switch2entropy: entropy minimization kicks in after this many steps.

  1. Evaluation of the Segmentation Networks Adapted with Multi-band Transfer (multiple betas)

python3 evaluationmulti.py --model='DeepLab' --save='../results' --restore-opt1="../checkpoints/FDA/gta2citydeeplab/gta2cityLB001" --restore-opt2="../checkpoints/FDA/gta2citydeeplab/gta2cityLB005" --restore-opt3="../checkpoints/FDA/gta2citydeeplab/gta2cityLB0_09"

Pretrained models on the GTA5 -> CityScapes task using DeepLab backbone can be downloaded here.

The above command should output: ===> mIoU19: 50.45 ===> mIoU16: 54.23 ===> mIoU13: 59.78

  1. Get Pseudo Labels for Self-supervised Training

python3 getSudoLabelmulti.py --model='DeepLab' --data-list-target='./dataset/cityscapeslist/train.txt' --set='train' --restore-opt1="../checkpoints/FDA/gta2citydeeplab/gta2cityLB001" --restore-opt2="../checkpoints/FDA/gta2citydeeplab/gta2cityLB005" --restore-opt3="../checkpoints/FDA/gta2citydeeplab/gta2cityLB009"

  1. Self-supervised Training with Pseudo Labels

python3 SStrain.py --model='DeepLab' --snapshot-dir='../checkpoints/FDA' --init-weights='../checkpoints/FDA/initweight/DeepLabinit.pth' --label-folder='cspseudolabel' --LB=0.01 --entW=0.005 --ita=2.0

  1. Other Models

VGG initializations can be downloaded through this link.

Pretrained models on the Synthia -> CityScapes task using DeepLab backbone link.

Pretrained models on the GTA5 -> CityScapes task using VGG backbone link.

Pretrained models on the Synthia -> CityScapes task using VGG backbone link.

Acknowledgment

Code adapted from BDL.

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.