retext-keywords

by retextjs

plugin to extract keywords and key-phrases

219 Stars 26 Forks Last release: 8 months ago (6.0.0) MIT License 151 Commits 22 Releases

Available items

No Items, yet!

The developer of this repository has not created any items for sale yet. Need a bug fixed? Help with integration? A different license? Create a request here:

retext-keywords

Build Coverage Downloads Size Sponsors Backers Chat

retext plugin to extract keywords and key-phrases.

Install

npm:

npm install retext-keywords

Use

Say we have the following file,

example.txt
, with the first four paragraphs on term Extraction from Wikipedia:
Terminology mining, term extraction, term recognition, or glossary extraction, is a subtask of information extraction. The goal of terminology extraction is to automatically extract relevant terms from a given corpus.

In the semantic web era, a growing number of communities and networked enterprises started to access and interoperate through the internet. Modeling these communities and their information needs is important for several web applications, like topic-driven web crawlers, web services, recommender systems, etc. The development of terminology extraction is essential to the language industry.

One of the first steps to model the knowledge domain of a virtual community is to collect a vocabulary of domain-relevant terms, constituting the linguistic surface manifestation of domain concepts. Several methods to automatically extract technical terms from domain-specific document warehouses have been described in the literature.

Typically, approaches to automatic term extraction make use of linguistic processors (part of speech tagging, phrase chunking) to extract terminological candidates, i.e. syntactically plausible terminological noun phrases, NPs (e.g. compounds "credit card", adjective-NPs "local tourist information office", and prepositional-NPs "board of directors" - in English, the first two constructs are the most frequent). Terminological entries are then filtered from the candidate list using statistical and machine learning methods. Once filtered, because of their low ambiguity and high specificity, these terms are particularly useful for conceptualizing a knowledge domain or for supporting the creation of a domain ontology. Furthermore, terminology extraction is a very useful starting point for semantic similarity, knowledge management, human translation and machine translation, etc.

…and our script,

example.js
, looks as follows:
var vfile = require('to-vfile')
var retext = require('retext')
var pos = require('retext-pos')
var keywords = require('retext-keywords')
var toString = require('nlcst-to-string')

retext() .use(pos) // Make sure to use retext-pos before retext-keywords. .use(keywords) .process(vfile.readSync('example.txt'), done)

function done(err, file) { if (err) throw err

console.log('Keywords:') file.data.keywords.forEach(function(keyword) { console.log(toString(keyword.matches[0].node)) })

console.log() console.log('Key-phrases:') file.data.keyphrases.forEach(function(phrase) { console.log(phrase.matches[0].nodes.map(stringify).join('')) function stringify(value) { return toString(value) } }) }

Now, running

node example
yields:
Keywords:
term
extraction
Terminology
web
domain

Key-phrases: terminology extraction terms term extraction knowledge domain communities

API

retext().use(keywords[, options])

Extract keywords and key-phrases from the document.

The results are stored on

file.data
: keywords at
file.data.keywords
and key-phrases at
file.data.keyphrases
. Both are lists.

A single keyword looks as follows:

{
  stem: 'term',
  score: 1,
  matches: [
    {node: Node, index: 5, parent: Node},
    // …
  ],
  // …
}

…and a key-phrase:

{
  score: 1,
  weight: 11,
  stems: ['terminolog', 'extract'],
  value: 'terminolog extract',
  matches:  [
    {nodes: [Node, Node, Node], parent: Node},
    // …
  ]
}
options.maximum

Try to detect at most

maximum
words
and
phrases
(
number
, default:
5
).

Note that actual counts may differ. For example, when two words have the same score, both will be returned. Or when too few words exist, less will be returned. the same goes for phrases.

Contribute

See

contributing.md
in
retextjs/.github
for ways to get started. See
support.md
for ways to get help.

This project has a code of conduct. By interacting with this repository, organization, or community you agree to abide by its terms.

License

MIT © Titus Wormer

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.