Need help with BERT-for-RRC-ABSA?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

howardhsu
364 Stars 88 Forks Apache License 2.0 72 Commits 10 Opened issues

Description

code for our NAACL 2019 paper: "BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysis"

Services available

!
?

Need anything else?

Contributors list

# 102,462
Jupyter...
reading...
sentime...
C++
69 commits

BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysis

code for our NAACL 2019 paper "BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysis", COLING 2020 paper "Understanding Pre-trained BERT for Aspect-based Sentiment Analysis" and (draft code of) Findings of EMNLP 2020 "DomBERT: Domain-oriented Language Model for Aspect-based Sentiment Analysis".

We found that BERT domain post-training (e.g, 1 day of training) is an economic way to boost the performance of BERT, because it is much harder (e.g., 10 days of training) to learn a general knowledge shared across domains and, meanwhile, loosing the long-tailed domain-specific knowledge.

News

Code base for "Understanding Pre-trained BERT for Aspect-based Sentiment Analysis" is released.
Code base on huggingface

transformers
is under
transformers
, with more cross-domain models.
Preprocessing ABSA xmls organized into a separate rep.
Want to have post-trained models for other domains in reviews ? checkout a cross-domain review BERT or download from HERE.
A conversational dataset of RRC can be found here.
If you only care about ASC, a more formal code base can be found in a similar rep focusing on ASC. *feedbacks are welcomed for missing instructions *

Problem to Solve

We focus on 3 review-based tasks: review reading comprehension (RRC), aspect extraction (AE) and aspect sentiment classification (ASC).

RRC: given a question ("how is the retina display ?") and a review ("The retina display is great.") find an answer span ("great") from that review;

AE: given a review sentence ("The retina display is great."), find aspects("retina display");

ASC: given an aspect ("retina display") and a review sentence ("The retina display is great."), detect the polarity of that aspect (positive).

E2E-ABSA: the combination of the above two tasks as a sequence labeling task.

And how a pre-trained BERT model on reviews be prepared for those tasks.

Code Base

For post-training of NAACL 2019 paper, the code base is splited into two versions:

transformers/
(instructions) and
pytorch-pretrained-bert/
(instructions).

For analysis of pre-trained BERT model for ABSA (COLING 2020), see this instructions.

Please check corresponding instructions for details.

Citation

If you find this work useful, please cite as following.

@inproceedings{xu_bert2019,
    title = "BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysis",
    author = "Xu, Hu and Liu, Bing and Shu, Lei and Yu, Philip S.",
    booktitle = "Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics",
    month = "jun",
    year = "2019",
}
@inproceedings{xu_understanding2020,
    title = "Understanding Pre-trained BERT for Aspect-based Sentiment Analysis",
    author = "Xu, Hu and Shu, Lei and Yu, Philip S. and Liu, Bing",
    booktitle = "The 28th International Conference on Computational Linguistics",
    month = "Dec",
    year = "2020",
}

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.