The parallel library

The parallel library

The parallel library is the easiest way to make use of parallelisation for speeding up calculations. Assuming we have the following situation where we would like to calculate something in a for-loop

RES.for <- c()
for(i in 1:10){
# Simulate some calculations, so let the system sleep for a second
  Sys.sleep(1)
  RES.for[i] <- i^2
}

Here, we iterate over the numbers from 1 to 10, do something that lasts a bit and as toy output we store the square of the index as result into RES.for.

The first step would now be a vectorization. For that we first identify the inner part of the for-loop and transform it into a function and feed it into a lapply function. That way we have

innerFunction <- function(x){
  Sys.sleep(1)
  out <- x^2
  out
}

RES.lapply <- lapply(1:10,innerFunction)
RES.lapply <- unlist(RES.lapply)

That function does the same as above, but instead of using the for-lop, it uses the vectorized lapply-command.

Once we have done this work, the application of the parallel library is very easy. All it needs is to change the lapply to the parallel mclapply command and to define how many cores are requested.

library(parallel)

innerFunction <- function(x){
  Sys.sleep(0.1)
  out <- x^2
  out
}

RES.mclapply <- mclapply(1:10,innerFunction, mc.cores=8)
RES.mclapply <- unlist(RES.mclapply)

That’s it, with that command we can run now the loop in group of eight parallel runs. Please note, on a windows computer, this approach is not possible and one needs to set mc.cores=1.

If the function has instead of a single value e.g. a matrix as output, the results could be stored e.g. into an array. Very little is needed to adjust above code to this, see the following example:

library(parallel)

innerFunction <- function(x){
  Sys.sleep(1)
# Simulate some matrix output
  out <- matrix(x,ncol=3, nrow=3)
  out
}

RES.for <- array(dim=c(3,3,10))
for(i in 1:10){
  RES.for[,,i] <- innerFunction(i)
}


RES.lapply <- lapply(1:10,innerFunction)
# Retransform the output into an array
RES.lapply <- array(unlist(RES.lapply), dim=c(3,3,10))

RES.mclapply <- mclapply(1:10,innerFunction, mc.cores=8)
# Retransform the output into an array
RES.mclapply <- array(unlist(RES.mclapply), dim=c(3,3,10))

Leave a Reply

%d bloggers like this: