knowledge-graph-embeddings

by mana-ysh

Implementations of Embedding-based methods for Knowledge Base Completion tasks

219 Stars 60 Forks Last release: Not found 31 Commits 0 Releases

Available items

No Items, yet!

The developer of this repository has not created any items for sale yet. Need a bug fixed? Help with integration? A different license? Create a request here:

knowledge-graph-embeddings

Python Implementations of Embedding-based methods for Knowledge Base Completion tasks, mainly inspired by scikit-kge and complex.

List of methods

  • RESCAL [Nickel+. 2011]
  • TransE [Bordes+. 2013]
  • DistMult [Yang+. 2015]
  • HolE [Nicklel+. 2016]
    • This model is equivalent to ComplEx[Hayashi and Shimbo. 2018], and the computation cost of ComplEx is lower than of HolE.
  • ComplEx [Trouillon+. 2016]
  • ANALOGY [Liu+. 2017]
    • This model can be regarded as a hybrid between DistMult and ComplEx.

Run to train and test

For training...

▶  python train.py -h
usage: Link prediction models [-h] [--mode MODE] [--ent ENT] [--rel REL]
                              [--train TRAIN] [--valid VALID]
                              [--method METHOD] [--epoch EPOCH]
                              [--batch BATCH] [--lr LR] [--dim DIM]
                              [--margin MARGIN] [--negative NEGATIVE]
                              [--opt OPT] [--l2_reg L2_REG]
                              [--gradclip GRADCLIP] [--save_step SAVE_STEP]
                              [--cp_ratio CP_RATIO] [--metric METRIC]
                              [--nbest NBEST] [--filtered]
                              [--graphall GRAPHALL] [--log LOG]

optional arguments: -h, --help show this help message and exit --mode MODE training mode ["pairwise", "single"] --ent ENT entity list --rel REL relation list --train TRAIN training data --valid VALID validation data --method METHOD method ["complex", "distmult", "transe", "hole", "rescal", "analogy"] --epoch EPOCH number of epochs --batch BATCH batch size --lr LR learning rate --dim DIM dimension of embeddings --margin MARGIN margin in max-margin loss for pairwise training --negative NEGATIVE number of negative samples for pairwise training --opt OPT optimizer ["sgd", "adagrad"] --l2_reg L2_REG L2 regularization --gradclip GRADCLIP gradient clipping --save_step SAVE_STEP epoch step for saving model --cp_ratio CP_RATIO ratio of complex's dimention in ANALOGY --metric METRIC evaluation metrics ["mrr", "hits"] --nbest NBEST n-best for hits metric --filtered use filtered metric --graphall GRAPHALL all graph file for filtered evaluation --log LOG output log dir

For testing...

▶  python test.py -h
usage: Link prediction models [-h] [--ent ENT] [--rel REL] [--data DATA]
                              [--filtered] [--graphall GRAPHALL]
                              [--method METHOD] [--model MODEL]

optional arguments: -h, --help show this help message and exit --ent ENT entity list --rel REL relation list --data DATA test data --filtered use filtered metric --graphall GRAPHALL all graph file for filtered evaluation --method METHOD method ["complex", "distmult", "transe", "hole", "rescal", "analogy"] --model MODEL trained model path

Experiments

WordNet (WN18)

| Models | MRR (flt) | MRR (raw) | [email protected] (flt) | [email protected] (flt) | [email protected] (flt) | |:-----------:|:------------:|:------------:|:------------:|:------------:|:------------:| | ComplEx* | 94.1 | 58.7 | 93.6 | 94.5 | 94.7 | | ComplEx | 94.3 | 58.2 | 94.0 | 94.6 | 94.8 |

hyper parameters

  • mode : single
  • epoch : 500
  • batch : 128
  • lr : 0.05
  • dim : 200
  • negative : 5
  • opt : adagrad
  • l2_reg : 0.001
  • gradclip : 5

FreeBase (FB15k)

| Models | MRR (flt) | MRR (raw) | [email protected] (flt) | [email protected] (flt) | [email protected] (flt) | |:-----------:|:------------:|:------------:|:------------:|:------------:|:------------:| | ComplEx* | 69.2 | 24.2 | 59.9 | 75.9 | 84.0 | | ComplEx | 69.5 | 24.2 | 59.8 | 76.9 | 85.0 |

hyper parameters

  • mode : single
  • epoch : 500
  • batch : 128
  • lr : 0.05
  • dim : 200
  • negative : 10
  • opt : adagrad
  • l2_reg : 0.0001
  • gradclip : 5

* means the results reported from the original papers

Dependencies

  • numpy
  • scipy

References

  • Bordes, A.; Usunier, N.; Garcia-Duran, A.; Weston, J.; and Yakhnenko, O. 2013. Translating embeddings for modeling multi-relational data. In Advances in Neural Information Processing Systems (NIPS).

  • Liu, H.; Wu, Y.; and Yang, Y. 2017. Analogical inference for multi-relational embeddings. In Proceedings of the 34th International Conference on Machine Learning (ICML).

  • Nickel, M.; Rosasco, L.; and Poggio, T. 2016. Holographic embeddings of knowledge graphs. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, AAAI’16.

  • Nickel, M.; Tresp, V.; and Kriegel, H.-P. 2011. A threeway model for collective learning on multi-relational data. In International Conference on Machine Learning (ICML-11), ICML ’11,

  • Trouillon, T.; Welbl, J.; Riedel, S.; Gaussier, E.; and Bouchard, G. 2016. Complex embeddings for simple link prediction. In International Conference on Machine Learning (ICML).

  • Yang, B.; Yih, W.; He, X.; Gao, J.; and Deng, L. 2015. Embedding entities and relations for learning and inference in knowledge bases. International Conference on Learning Representations 2015.

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.