neural_best_buddies

by kfiraberman

kfiraberman / neural_best_buddies

An algorithm for sparse semantic correspondence between images of different categories [SIGGRAPH 201...

147 Stars 27 Forks Last release: Not found MIT License 33 Commits 0 Releases

Available items

No Items, yet!

The developer of this repository has not created any items for sale yet. Need a bug fixed? Help with integration? A different license? Create a request here:

Neural Best-Buddies in PyTorch

This is our PyTorch implementation for the Neural-Best Buddies paper.

The code was written by Kfir Aberman and supported by Mingyi Shi.

Neural Best-Buddies: Project | Paper

If you use this code for your research, please cite:

Neural Best-Buddies: Sparse Cross-Domain Correspondence Kfir Aberman, Jing Liao, Mingyi Shi, Dani Lischinski, Baoquan Chen, Daniel Cohen-Or, SIGGRAPH 2018.

Prerequisites

  • Linux or macOS
  • Python 2 or 3
  • CPU or NVIDIA GPU + CUDA CuDNN
  • Pytorch > (1.x.x)

Run

  • Run the algorithm (demo example)
    bash
    #!./script.sh
    python3 main.py --datarootA ./images/original_A.png --datarootB ./images/original_B.png --name lion_cat --k_final 10
    
    The option
    --k_final
    dictates the final number of returned points. The results will be saved at
    ../results/
    . Use
    --results_dir {directory_path_to_save_result}
    to specify the results directory.

Output

Sparse correspondence: - correspondenceA.txt, correspondenceB.txt - correspondenceAtopk.txt, correspondenceBtopk.txt

Dense correspondence (densifying based on MLS): - BtoA.npy, AtoB.npy

Warped images (aligned to their middle geometry): - warpAtoM.png, warpBtoM.png

Tips

  • If you are running the algorithm on a bunch of pairs, we recommend to stop it at the second layer to reduce runtime (comes at the expense of accuracy), use the option
    --fast
    .
  • If the images are very similar (e.g, two frames extracted from a video), many corresponding points might be found, resulting in long runtime. In this case we suggest to limit the number of corresponding points per level by setting
    --k_per_level 20
    (or any other desired number)

Image Morphing

The morphing results in the paper are based on the paper Automating image morphing using structural similarity on a halfway domain by Liao et al.

Here are detailed instructions for how to combine the output of our code with the image morphing code:

First, please download the exe file from here and test whether it works on your machine with the given demo case as described in the UserGuide.dox or simply run “runme.bat”. Alternatively, the source code can be found here. When it works:

  1. Make a folder e.g. “case1”, containing three files in it: the two input images (should be with the same size) called image1.png and image2.png respectively, and a setting file called “settings.xml”. You can copy the setting.xml form “baby” and modify it to the points that were extracted by our code. For example:



where the format is:



  1. Make a subfolder called “0” under the main folder “case1”. Put two mask images “mask1.png” “mask2.png” that have the same size as the input images and set all the 3 channels to zero.

  2. Run

    ImageMorphing.exe case1\settings.xml -auto

Citation

If you use this code for your research, please cite our paper:

@article{aberman2018neural,
  title={Neural best-buddies: Sparse cross-domain correspondence},
  author={Aberman, Kfir and Liao, Jing and Shi, Mingyi and Lischinski, Dani and Chen, Baoquan and Cohen-Or, Daniel},
  journal={ACM Transactions on Graphics (TOG)},
  volume={37},
  number={4},
  pages={69},
  year={2018},
  publisher={ACM}
}

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.