Compiling R with Intel's ICC and MKL

6/1/2015

I can count on one hand the number of non-free pieces of software I use regularly, but there are some that are just too good for me to resist. For a while now, I've been predominantly using Intel's ICC compiler for compiling C programs, and their MKL for BLAS and LAPACK functions. Intel makes these resources free for students (they also have discounted academic pricing for non-students), and I recommend that, if you haven't already, you look into trying them. Since I spend most of my time in R, I compile new versions of R from source using ICC and MKL, but it took me a little time to get it working, so I thought I'd share how I succeeded on my personal computer (running Ubuntu 14.04, and this worked with both R 3.2.0 and the latest development version of R 3.2.1). The main resource I used to do this was this post on Intel's website, but it took a little bit extra to get things working for me.

Installation

Before starting, make sure that you have the environmental variables set related to linking ICC and MKL. Assuming your operating system knows where to find everything, these are the commands I use when compiling R:

source /opt/intel/bin/compilervars.sh intel64
export CC="icc -std=c99"
export CXX="icpc"
export AR="xiar"
export LD="xild"
export CFLAGS="-openmp -O3 -ipo -xHost -multiple-processes"
export CXXFLAGS="-openmp -O3 -ipo -xHost -multiple-processes"
export FFLAGS="-fopenmp -O3 -march=native"
export FCFLAGS="-fopenmp -O3 -march=native"
export MKL="-lmkl_gf_lp64 -lmkl_intel_thread -lmkl_core -liomp5 -lpthread"
export FLIBS="-lgfortran"
export MAIN_LDFLAGS="-openmp"
./configure --with-blas="$MKL" --with-lapack
make
sudo -E env "PATH=$PATH" make install

A couple of things that tripped me up in my previous attempts were ignoring the -lgfortran flag, which is necessary because I am using gfortran as the fortran compiler instead of ifort (Intel's fortran compiler, which is not free for students, and I don't use enough to justify buying it). Also, the actual linking of the MKL may be done in a shorthand way now, using -lmkl_rt, but the above method has always worked for me. I had originally ommitted the -std=c99 flag, but there are some packages in which authors define loop variables within the for statement, which will cause an error without the flag. The last line gives the sudo command access to the user's PATH variable.

After line 13 you should make sure that everything is how you'd like it in the output of ./configure

Before running line 15, it's advisable to test your newly compiled program by running it through ./bin/R before fully installing it and replacing any old versions.

In case you were wondering if all of this trouble is worth the time, I ran a small test comparing R compiled from source using the default settings with R compiled using the above procedure (I used the latest development version for this). I tested 3 common BLAS/LAPACK functions on 2000 x 2000 matrices. The results show the average time over 10 trials.

	tcrossprod()	chol2inv(chol())	svd()
Plain Config	3.2	2.6	20.8
ICC + MKL (Single Core)	0.16	0.20	4.06
ICC + MKL (6 Cores)	0.039	0.060	1.61

Hyperthreading

Intel's MKL is smart when it comes to hyperthreading since running on all possible threads would be significantly slower if hyperthreading is enabled (hyperthreading is mainly useful if each thread on a core could potentially be performing different types of operations at a given time). On a 6-core processor, MKL will (when it determines it's necessary) only start 6 threads, but these threads may not be run on separate physical cores. To make sure that threads are run on separate cores, use:

export KMP_AFFINITY=granularity=fine,compact,1,0

Either set this in your .profile or put it in a startup script for R.

Compiling Packages

If you want to be able to use the same settings you used to compile R later when compiling additional packages, you need to put all of the export commands from above in a file that R will be able to find. The location I use is ~/.R/Makevars. Remember that this file exists because if you later compile a new version of R, this will be used during the installation of recommended packages.

General Comments

Even if you're not going to use the MKL, don't use the default shared BLAS and LAPACK libraries with which R comes packaged; they are sooooo slow. An easier option to get up and running, and free, is OpenBLAS. Also, be very careful using any optimized, multithreaded computational libraries on laptops! You could fry your fancy new laptop without too much difficulty if you're running all cores at 100% for long simulation runs. Also, Intel's Haswell cores, and newer cores, can use AVX2 instructions, which can make the processor run even hotter when it is given more voltage.

R has a lot of room for customization, and its potential is not fully utilized by just downloading the pre-compiled binaries. Using an optimized BLAS/LAPACK library should be the first step for any serious R user. I hope that this post can help someone else speed up their computation as much as others have helped me!

Update - 9/11/2015

Intel has now released the 2016 version of their tools, and, at some point previously, they began to include their Fortran compiler, ifort, in their free offering for students. I thought I would share the commands that I used to compile the latest version of R.

source /opt/intel/bin/compilervars.sh intel64
export CC="icc -std=c99"
export CXX="icpc"
export F77="ifort"
export FC="ifort"
export AR="xiar"
export LD="xild"
export CFLAGS="-openmp -O3 -ipo -xHost -multiple-processes"
export CXXFLAGS="-openmp -O3 -ipo -xHost -multiple-processes"
export FFLAGS="-openmp -O3 -ipo -xHost -multiple-processes"
export FCFLAGS="-openmp -O3 -ipo -xHost -multiple-processes"
export MKL="-lmkl_rt"
export MAIN_LDFLAGS="-openmp"
./configure --with-blas="$MKL" --with-lapack --with-tcltk
make
sudo -E env "LD_LIBRARY_PATH=$LD_LIBRARY_PATH" env "PATH=$PATH" make install

This worked on the latest version of R (3.2.2), and the developement version on my system. It required additionally giving the sudo command access to my LD_LIBRARY_PATH environmental variable compared to previously. This variable includes the path to the Intel libraries.