Need help with SDNet?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

microsoft
164 Stars 94 Forks MIT License 6 Commits 8 Opened issues

Description

SDNet

Services available

!
?

Need anything else?

Contributors list

# 277,967
Python
6 commits

SDNet

This is the official code for the Microsoft's submission of SDNet model to CoQA leaderboard. It is implemented under PyTorch framework. The related paper to cite is:

SDNet: Contextualized Attention-based Deep Network for Conversational Question Answering, by Chenguang Zhu, Michael Zeng and Xuedong Huang, at https://arxiv.org/abs/1812.03593.

For usage of this code, please follow Microsoft Open Source Code of Conduct.

Directory structure:

  • main.py: the starter code

  • Models/

    • BaseTrainer.py: Base class for trainer
    • SDNetTrainer.py: Trainer for SDNet, including training and predicting procedures
    • SDNet.py: The SDNet network structure
    • Layers.py: Related network layer functions
    • Bert/
    • Bert.py: Customized class to compute BERT contextualized embedding
      • modeling.py, optimization.py, tokenization.py: From Huggingface's PyTorch implementation of BERT
  • Utils/

    • Arguments.py: Process argument configuration file
    • Constants.py: Define constants used
    • CoQAPreprocess.py: preprocess CoQA raw data into intermediate binary/json file, including tokenzation, history preprending
    • CoQAUtils.py, General Utils.py: utility functions used in SDNet
    • Timing.py: Logging time

How to run

Requirement: PyTorch 0.4.1, spaCy 2.0.16. The docker we used is available at dockerhub: https://hub.docker.com/r/zcgzcgzcg/squadv2/tags. Please use v3.0 or v4.0. 1. Create a folder (e.g. coqa) to contain data and running logs; 2. Create folder coqa/data to store CoQA raw data: coqa-train-v1.0.json and coqa-dev-v1.0.json; 3. Copy the file conf from the repo into folder coqa; 4. If you want to use BERT-Large, download their model into coqa/bert-large-uncased; if you want to use BERT-base, download their model into coqa/bert-base-cased; * The models can be downloaded from Huggingface: * 'bert-base-uncased': "https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-uncased.tar.gz", * 'bert-large-uncased': "https://s3.amazonaws.com/models.huggingface.co/bert/bert-large-uncased.tar.gz" * bert-large-uncased-vocab.txt can be downloaded from Google's BERT repository 5. Create a folder glove in the same directory of coqa and download GloVe embedding glove.840B.300d.txt into the folder.

Your directory should look like this: * coqa/ * data/ * coqa-train-v1.0.json * coqa-dev-v1.0.json * bert-large-uncased/ * bert-large-uncased-vocab.txt * bertconfig.json * pytorchmodel.bin * conf
* glove/ * glove.840B.300d.txt

Then, execute

python main.py train path_to_coqa/conf
.

If you run for the first time, CoQAPreprocess.py will automatically create folders conf~/spacyintermediatefeatures~ inside coqa to store intermediate tokenization results, which will take a few hours.

Every time you run the code, a new running folder run_idx will be created inside coqa/conf~, which contains running logs, prediction result on dev set, and best model.

Contact

If you have any questions, please contact Chenguang Zhu, [email protected]

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.