by Yanchang Zhao, RDataMining.com

Compared with many other programming languages, such as C/C++ and Java, R is less efficient and consumes much more memory. Fortunately, there are some packages that enables parallel computing in R and also packages for processing big data in R without loading all data into RAM. I have collected some links to online documents and slides on handling big data and parallel computing in R, which are listed below. Many online resources on other topics related to data mining with R can be found at http://www.rdatamining.com/resources/onlinedocs.

- State of the Art in Parallel Computing with R

It provides an excellent overview and comparison of R packages for parallel computing, including packages for computer cluster, packages for grid computing, and packages for multi-core systems.

http://www.jstatsoft.org/v31/i01/paper

- Taking R to the Limit, Part I – Parallelization in R

http://www.bytemining.com/2010/07/taking-r-to-the-limit-part-i-parallelization-in-r/

- Taking R to the Limit, Part II – Large Datasets in R

http://www.bytemining.com/2010/08/taking-r-to-the-limit-part-ii-large-datasets-in-r/

- Massive data, shared and distributed memory,and concurrent programming: bigmemory and foreach

http://sites.google.com/site/bigmemoryorg/research/documentation/bigmemorypresentation.pdf

- High Performance Computing with R

http://igmcs.utk.edu/sites/igmcs/files/Patel-High-Performance-Computing-with-R-2011-10-20.pdf

- R with High Performance Computing: Parallel processing and large memory

http://files.meetup.com/1781511/HighPerformanceComputingR-Szczepanski.pdf

- Parallel Computing in R

http://tinyurl.com/R-Bioc-slides

- Parallel Computing with R using snow and snowfall

http://www.ics.uci.edu/~vqnguyen/talks/ParallelComputingSeminaR.pdf

- Interacting with Data using the filehash Package for R

http://cran.r-project.org/web/packages/filehash/vignettes/filehash.pdf

- Tutorial: Parallel computing using R package snowfall

http://www.imbi.uni-freiburg.de/parallel/docs/Reisensburg2009_TutParallelComputing_Knaus_Porzelius.pdf

- Easier Parallel Computing in R with snowfall and sfCluster

http://journal.r-project.org/2009-1/RJournal_2009-1_Knaus+et+al.pdf

- Distributed Data Analysis with Hadoop and R

http://www.infoq.com/presentations/Distributed-Data-Analysis-with-Hadoop-and-R

- A tutorial on RHadoop

https://github.com/RevolutionAnalytics/RHadoop/wiki/Tutorial

Advertisements

Pingback: Online resources for handling big data and parallel computing in R | Estadística y R | Scoop.it

Pingback: Online resources for handling big data and parallel computing in R | RDataMining | Statistics with R | Scoop.it

Pingback: Online resources for handling big data and parallel computing in R | Health and Biomedical Informatics | Scoop.it

Pingback: Online resources for handling big data and parallel computing in R | EEDSP | Scoop.it

Pingback: Useful for referring–5-20-2012 « Honglang Wang's Blog

Pingback: Learn R, Forecast in R, Handle Big Data in R « socialempirics

Pingback: 13 R Online Resources for Big Data and Parallel Computing « Blog de xkortazar

Pingback: Online resources for handling big data and parallel computing in R | EEDSP | Scoop.it