Countering Adversarial Image using Input Transformations.
This package implements the experiments described in the paper Countering Adversarial Images Using Input Transformations. It contains implementations for adversarial attacks, defenses based image transformations, training, and testing convolutional networks under adversarial attacks using our defenses. We also provide pre-trained models.
If you use this code, please cite our paper:
The code implements the following four defenses against adversarial images, all of which are based on image transformations: - Image quilting - Total variation minimization - JPEG compression - Pixel quantization
Please refer to the paper for details on these defenses. A detailed description of the original image quilting algorithm can be found here; a detailed description of our solver for total variation minimization can be found here.
To use this code, first install Python, PyTorch, and Faiss (to perform image quilting). We tested the code using Python 2.7, PyTorch v0.2.0, and scikit-image 0.11; your mileage may vary when using other versions.
Pytorch can be installed using the instructions here. Faiss is required to run the image quilting algorithm; it is not automatically included because faiss does not have a pip support and because it requires configuring BLAS and LAPACK flags, as described here. Please install faiss using the instructions given here.
The code uses several other external dependencies (for training Inception models, performing Bregman iteration, etc.). These dependencies are automatically downloaded and installed when you install this package via
cd adversarialimagedefenses pip install .
To import the package in Python:
```python import adversarial
The functionality implemented in this package is demonstrated in this example. Run the example via:
bash python adversarial/examples/demo.py
The full functionality of the package is exposed via several runnable Python scripts. All these scripts require the user to specify the path to the Imagenet dataset, the path to pre-trained models, and the path to quilted images (once they are computed) in
lib/path_config.json. Alternatively, the paths can be passed as input arguments into the scripts.
import adversarial from indexpatches import createfaisspatches, parseargs
args = parse_args()
args.patchsize = 5 createfaiss_patches(args) ```
python index_patches.py. The following arguments are supported: -
--patch_sizePatch size (square) that will be used in quilting (default: 5). -
--num_patchesNumber of patches to generate (default: 1000000). -
--pca_dimsPCA dimension for faiss (default: 64). -
--patches_fileFile in which patches are saved. -
--index_fileFile in which faiss index of patches is saved.
Code example: ```python import adversarial from gentransformedimages import generatetransformedimages from lib import opts
args = opts.parseargs(opts.OptType.TRANSFORMATION) args.operation = "transformationonraw" args.defenses = ["tvm"] args.partitionsize = 1 # Number of samples to generate
python gen_transformed_images.py. In addition to the common arguments and adversarial arguments, the following arguments are supported: -
--operationOperation to run. Supported operations are:
transformation_on_raw: Apply transformations on raw images.
transformation_on_adv: Apply transformations on adversarial images.
cat_data: Concatenate output from distributed
--data_typeData type (
--out_dirDirectory path for output of
--partition_dirDirectory path to output transformed data. -
--data_batchesNumber of data batches to generate. Used for random crops for ensembling. -
--partitionDistributed data partition (default: 0). -
--partition_sizeThe size of each data partition.
transformation_on_raw, partitionsize represents number of classes for each process.
--n_threadsNumber of threads for
Many file systems perform poorly when dealing with millions of small files (such as images). Therefore, we generally TAR our image datasets (obtained by running
generate_transformed_images). Next, we use
gen_tar_index.pyto generate a file index for the TAR file. The file index facilitates fast, random-access reading of the TAR file; it is much faster and requires less memory than untarring the data or using
import adversarial from gentarindex import generatetarindex, parse_args
args = parseargs() generatetar_index(args) ```
python gen_tar_index.py. The following arguments are supported: -
--tar_pathPath for TAR file or directory. -
--index_rootDirectory in which to store TAR index file. -
--path_prefixPrefix to identify TAR member names to be indexed.
Code example: ```python import adversarial from genadversarialimages import generateadversarialimages from lib import opts
args = opts.parseargs(opts.OptType.ADVERSARIAL) args.model = "resnet50" args.adversarytogenerate = "fgs" args.partitionsize = 1 # Number of samples to generate args.datatype = "val" # input dataset type args.normalize = True # apply normalization on input data args.attacktype = "blackbox" # For attack, use transformed models args.pretrained = True # Use pretrained model from model-zoo
python gen_adversarial_images.py. For a list of the supported arguments, see common arguments and adversarial arguments.
Code example: ```python import adversarial from trainmodel import trainmodel from lib import opts
args = opts.parse_args(opts.OptType.TRAIN) args.defenses = None # defense=<(raw, tvm, quilting, jpeg, quantization)> args.model = "resnet50" args.normalize = True # apply normalization on input data
python train_model.py. In addition to the common arguments, the following arguments are supported: -
--resumeResume training from checkpoint (if available). -
--lrInitial learning rate defined in constants.py. -
--lr_decayExponential learning rate decay defined in constants.py. -
--lr_decay_stepsizeDecay learning rate after every stepsize epochs defined in constants.py. -
--momentumMomentum (default: 0.9). -
--weight_decayAmount of weight decay (default: 1e-4). -
--start_epochIndex of first epoch (default: 0). -
--end_epochIndex of last epoch (default: 90). -
--preprocessed_epoch_dataAugmented and transformed data for each epoch is pre-generated (default:
Code exammple: ```python import adversarial from classifyimages import classifyimages from lib import opts
args = opts.parse_args(opts.OptType.CLASSIFY)
python classify_images.py. In addition to the common arguments, the following arguments are supported: -
--ncropsList of number of crops for each defense to use for ensembling (default:
--crop_fracList of crop fraction for each defense to use for ensembling (default:
--crop_typeList of crop type(
sliding(hardset for 9 crops)) for each defense to use for ensembling (default:
We provide pre-trained models that were trained on ImageNet images that were processed using total variation minimization (TVM) or image quilting can be downloaded from the following links (set the
models_rootargument to the path that contains these model model files):
The following arguments are used by multiple scripts, including
--data_rootMain data directory to save and read data.
--models_rootDirectory path to store/load models.
--tar_dirDirectory path for transformed images(train/val) stored in TAR files.
--tar_index_dirDirectory path for index files for transformed images in TAR files.
--quilting_index_rootDirectory path for quilting index files.
--quilting_patch_rootDirectory path for quilting patch files.
--modelModel to use (default:
--deviceDevice to use: cpu or gpu (default:
--normalizeNormalize image data.
--batchsizeBatch size for training and testing (default: 256).
--preprocessed_dataTransformations/Defenses are already applied on saved images (default:
--defensesList of defenses to apply:
--pretrainedUse pretrained model from PyTorch model zoo (default:
--tvm_weightRegularization weight for total variation minimization (TVM).
--pixel_drop_ratePixel drop rate to use in TVM.
--tvm_methodReconstruction method to use in TVM (default:
--quilting_patch_sizePatch size to use in image quilting.
--quilting_neighborsNumber of nearest patches to sample from in image quilting (default: 1).
--quantize_depthBit depth for quantization defense (default: 8).
The following arguments are used whem generating adversarial images with
--n_samplesMaximum number of samples to test on.
--adversaryAdversary to use:
--adversary_modelModel to use for generating adversarial images (default:
--learning_rateLearning rate for iterative adversarial attacks (default: read from constants).
--adv_strengthAdversarial strength for non-iterative adversarial attacks (default: read from constants).
--adversarial_rootPath containing adversarial images.