R Programming Intermediate: Parallel Processing

Name: R Programming Intermediate: Parallel Processing
Author: Tom Rosenwinkel

Learn how to leverage parallel computing resources to scale your R projects

Preview the first lesson free — get full access to all 3 lessons.

Course: On-Demand

Hello, my name is Tom Rosenwinkle. In this series of lessons, we are going to see examples of parallel processing in R. Parallel processing is an important concept for writing R code that will scale to large data sets, especially when parallel computing resources are available. And those parallel computing resources are a lot more available now than when R first became popular as a programming language. It used to be that you needed to use systems or clusters or even cloud resources to be able to take advantage of parallel processing routines. But now a lot of even local machines are multi-core machines, meaning that there is some parallelization that will be possible. When solving a lot of programming problems with R code, a common technique that is needed is to write a function and that function needs to be applied to every element of a large data set. And so when those data sets become large and parallel processing resources are available, it is important to know that your R code will take advantage of those parallel processing resources so that sections of the data set will be assigned out to resources to execute in parallel so that the end result can be achieved faster. In this series of lessons, we're going to use a few packages. And so you'll see that I'm working in RStudio. It's important to remember how to install and source the packages that are needed. RStudio has an install function that will help to find and install packages and it will run the install.packages command that is necessary. Also, in this lesson, for example, we will use the parallel package. We can use RStudio to run the library command. And so once the package is installed, it is important to source it with the library command. In this introductory lesson, we will look at how many cores I have available on my local machine. So in this parallel package, there is a function called detect cores. And when I run that, I see that on my machine, I have 16 cores available to me. You will see different results on your machine because it is checking your local resources. And again, today, most local machines will have more than one core available. So I could run a for loop that will create a sequence of integers from one to 10, basically in order to make sure that we repeat this code block 10 times. So if I execute this loop that will sleep 10 times for one second each, it will run for about 10 seconds because it is doing that in sequence for the integer one and then two and then three and so on and finally up to 10. Now I can break that down a little bit and use R's L apply function, which takes three arguments first the sequence for which we are looping. And then the second argument is the function, which I'm defining to be the sleep function as an anonymous function. So if I run the same function, the sleep function using R's L apply command on the sequence one through 10, it is applying this function 10 times, but it will be actually a little bit faster because it is taking advantage of those 16 cores. Though in this series of lessons, we should see some packages that are compiled in ways that will take fuller advantage of the available processing cores. In the next lesson, we will see examples of the MCL apply function. Thank you for watching.

Intermediate Provider : Tom Rosenwinkel 3 Lessons · 12m in English

Course Description

How do you save time running R code, or scale your R code to incorporate more data? How can you apply available parallel computing resources to solve your problem more efficiently? The parallel package in R provides access to these resources and the ability to run your functions in multiple processes in parallel. Learning how to use these tools is an essential skill for writing R code that scales with the parallel computing resources available, allowing your projects to run with more data in less time. The topics discussed in these lessons will build on your understanding of applying functions in R and provide a foundation for scaling your projects with parallel computing resources.

In these lessons on parallel processing, you will learn how to leverage parallel computing resources to scale your R projects. We will explore the parallel package, how to apply functions using multiple cores, and creating and using socket clusters. By the end of these lessons, you will be able to scale your R projects by taking advantage of parallel computing resources.

What You'll Learn

Describe the tools available through R's parallel package
Apply R functions using multiple computational cores
Create and use socket clusters following best practices
Leverage parallel computing resources to scale your R projects
Use functions such as mclapply and parLapply for parallel processing

Key Takeaways

The parallel package in R provides access to parallel computing resources and the ability to run functions in multiple processes in parallel.
Using parallel tools allows R code to scale with available parallel computing resources, running projects with more data in less time.
These lessons cover the parallel package, applying functions using multiple cores, and creating and using socket clusters.
This material builds on understanding of applying functions in R and provides a foundation for scaling projects with parallel computing.
By the end of the lessons, learners are able to scale their R projects by taking advantage of parallel computing resources.

Frequently Asked Questions

What does this course cover?

It covers R's parallel package, how to apply functions using multiple cores, and creating and using socket clusters, through lessons on the Parallel Package, Mclapply, and ParLapply.

What skills will I gain from this course?

You will gain skills in parallel computing, parallel processing, and resource utilization, enabling you to scale your R projects with parallel computing resources.

Who is this course for?

This intermediate R programming course builds on your understanding of applying functions in R, so it suits learners who want to scale their R projects using parallel computing resources.

What will I be able to do by the end?

By the end of these lessons, you will be able to scale your R projects by taking advantage of parallel computing resources.

Why use parallel processing in R?

Parallel processing helps you save time running R code and scale your code to incorporate more data by applying available parallel computing resources to solve problems more efficiently.