Help: stemming and stem completion with package tm in R

I came across a problem below when doing stemming and stem completion with package tm in R. Word “mining” was stemmed to “mine” with stemDocument(), and then completed to “miners”with stemCompletion(). However, I prefer to keep “mining” intact.

For stemCompletion(), the default type of completion is “prevalent”, which takes the most frequent match as completion. Although “mining” is much more frequent than “miners” in my text, it still completed “mine” to “miners”.

An example is shown below.

############################################
> library(tm)
> (a <- c(“mining”, “miners”, “mining”))
[1] “mining” “miners” “mining”
> (b <- stemDocument(a))
[1] “mine”  “miner” “mine”
> (d <- stemCompletion(b, dictionary=a))
mine    miner     mine
“miners” “miners” “miners”
############################################

Some possible solutions are:
1) to change the options or dictionary in stemDocument(), so that “mining” is not stemmed to “mine”, which I think is the best way;
2) to change the options or dictionary in stemCompletion(), so that “mine” is completed to “mining”;
3) to manually correct this after stem completion, which is the last option.

I am looking for a solution for above 1) or 2), but cannot find the way to do it with stemDocument() in package tm. Any help will be appreciated.

Thanks,
Yanchang Zhao
Email: yanchangzhao(at)gmail.com

RDataMining: http://www.rdatamining.com
Twitter: http://twitter.com/RDataMining
Group on Linkedin: http://group.rdatamining.com
Group on Google: http://group2.rdatamining.com

About these ads

About Yanchang Zhao

I am a data miner, using R for data mining applications. My work on R and data mining: RDataMining.com; Twitter; Group on Linkedin; and Group on Google.
This entry was posted in Data Mining, R and tagged , . Bookmark the permalink.

2 Responses to Help: stemming and stem completion with package tm in R

  1. Pingback: Elektrische Zahnbuerste

  2. Pingback: Help: stemming and stem completion with package tm in R | Statistics with R | Scoop.it

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s