Paper: https://arxiv.org/abs/2010.13993
This directory contains OGB submissions. All hyperparameters were tuned on the validation set with optuna, except for products, which was hand tuned. All experiments were run with a RTX 2080 TI with 11GB.
In general, the more complex and "smooth" your GNN is, the less likely it'll be that applying the "Correct" portion helps performance. In those cases, you may consider just applying the "smooth" portion, like we do on the GAT. In almost all cases, applying the "smoothing" component will improve performance. For Linear/MLP models, applying the "Correct" portion is almost always essential for obtaining good performance.
In a similar vein, an improvement of performance of your model may not correspond to an improvement after applying C&S. Considering that C&S learns no parameters over your data, our intuition is that C&S "levels" the playing field, allowing models that learn interesting features to shine (as opposed to learning how to be smooth).
In general, autoscale works more reliably than fixedscale, even though fixedscale may make more sense...
python run_experiments.py --dataset arxiv --method lpValid acc: 0.7013658176448874 Test acc: 0.6832294302820814
python gen_models.py --dataset arxiv --model plain --epochs 1000 python run_experiments.py --dataset arxiv --method plainValid acc -> Test acc Args []: 73.00 ± 0.01 -> 71.26 ± 0.01
python gen_models.py --dataset arxiv --model linear --use_embeddings --epochs 1000 python run_experiments.py --dataset arxiv --method linearValid acc -> Test acc Args []: 73.68 ± 0.04 -> 72.22 ± 0.02;
python gen_models.py --dataset arxiv --model mlp --use_embeddings python run_experiments.py --dataset arxiv --method mlpValid acc -> Test acc Args []: 73.91 ± 0.15 -> 73.12 ± 0.12
cd gat && python gat.py --use-norm cd .. && python run_experiments.py --dataset arxiv --method gatValid acc -> Test acc Args []: 74.84 ± 0.07 -> 73.86 ± 0.14
As opposed to the paper's results, which only use spectral embeddings, here we use spectral and diffusion embeddings, which we find improves Arxiv performance.
python run_experiments.py --dataset products --method lpValid acc: 0.9090608549703736 Test acc: 0.7434145274640762
python gen_models.py --dataset products --model plain --epochs 1000 --lr 0.1 python run_experiments.py --dataset products --method plainValid acc -> Test acc Args []: 91.03 ± 0.01 -> 82.54 ± 0.03
python gen_models.py --dataset products --model linear --use_embeddings --epochs 1000 --lr 0.1 python run_experiments.py --dataset products --method linearValid acc -> Test acc Args []: 91.34 ± 0.01 -> 83.01 ± 0.01
python gen_models.py --dataset products --model mlp --hidden_channels 200 --use_embeddings python run_experiments.py --dataset products --method mlpValid acc -> Test acc Args []: 91.47 ± 0.09 -> 84.18 ± 0.07