Compiling R with Intel's ICC and MKL
I can count on one hand the number of non-free pieces of software I use regularly, but there are some that are just too good for me to resist. For a while now, I've been predominantly using Intel's ICC compiler for compiling C programs, and their MKL for BLAS and LAPACK functions. Intel makes these resources free for students (they also have discounted academic pricing for non-students), and I recommend that, if you haven't already, you look into trying them. Since I spend most of my time in R, I compile new versions of R from source using ICC and MKL, but it took me a little time to get it working, so I thought I'd share how I succeeded on my personal computer (running Ubuntu 14.04, and this worked with both R 3.2.0 and the latest development version of R 3.2.1). The main resource I used to do this was this post on Intel's website, but it took a little bit extra to get things working for me.
Before starting, make sure that you have the environmental variables set related to linking ICC and MKL. Assuming your operating system knows where to find everything, these are the commands I use when compiling R:
source /opt/intel/bin/compilervars.sh intel64 export CC="icc -std=c99" export CXX="icpc" export AR="xiar" export LD="xild" export CFLAGS="-openmp -O3 -ipo -xHost -multiple-processes" export CXXFLAGS="-openmp -O3 -ipo -xHost -multiple-processes" export FFLAGS="-fopenmp -O3 -march=native" export FCFLAGS="-fopenmp -O3 -march=native" export MKL="-lmkl_gf_lp64 -lmkl_intel_thread -lmkl_core -liomp5 -lpthread" export FLIBS="-lgfortran" export MAIN_LDFLAGS="-openmp" ./configure --with-blas="$MKL" --with-lapack make sudo -E env "PATH=$PATH" make install
A couple of things that tripped me up in my previous attempts were
-lgfortran flag, which is necessary because I
am using gfortran as the fortran compiler instead of ifort (Intel's
fortran compiler, which is not free for students, and I don't use enough
to justify buying it). Also, the actual linking of the MKL may be done
in a shorthand way now, using
-lmkl_rt, but the above
method has always worked for me. I had originally ommitted
-std=c99 flag, but there are some packages in which
authors define loop variables within the
which will cause an error without the flag. The last line gives
sudo command access to the user's
13 you should make sure that everything is
how you'd like it in the output of
Before running line
advisable to test your newly compiled program by running it
./bin/R before fully installing it and replacing
any old versions.
In case you were wondering if all of this trouble is worth the time, I ran a small test comparing R compiled from source using the default settings with R compiled using the above procedure (I used the latest development version for this). I tested 3 common BLAS/LAPACK functions on 2000 x 2000 matrices. The results show the average time over 10 trials.
|ICC + MKL
|ICC + MKL
Intel's MKL is smart when it comes to hyperthreading since running on all possible threads would be significantly slower if hyperthreading is enabled (hyperthreading is mainly useful if each thread on a core could potentially be performing different types of operations at a given time). On a 6-core processor, MKL will (when it determines it's necessary) only start 6 threads, but these threads may not be run on separate physical cores. To make sure that threads are run on separate cores, use:
Either set this in your
.profileor put it in a startup script for R.
If you want to be able to use the same settings you used to compile R later when
compiling additional packages, you need to put all of the
export commands from
above in a file that R will be able to find. The location I use
~/.R/Makevars. Remember that this file exists because if you later compile a
new version of R, this will be used during the installation of recommended packages.
Even if you're not going to use the MKL, don't use the default shared BLAS and LAPACK libraries with which R comes packaged; they are sooooo slow. An easier option to get up and running, and free, is OpenBLAS. Also, be very careful using any optimized, multithreaded computational libraries on laptops! You could fry your fancy new laptop without too much difficulty if you're running all cores at 100% for long simulation runs. Also, Intel's Haswell cores, and newer cores, can use AVX2 instructions, which can make the processor run even hotter when it is given more voltage.
R has a lot of room for customization, and its potential is not fully utilized by just downloading the pre-compiled binaries. Using an optimized BLAS/LAPACK library should be the first step for any serious R user. I hope that this post can help someone else speed up their computation as much as others have helped me!
Update - 9/11/2015
Intel has now released the 2016 version of their tools, and, at some point previously, they began to include their Fortran compiler, ifort, in their free offering for students. I thought I would share the commands that I used to compile the latest version of R.
This worked on the latest version of R (3.2.2), and the developement version on my system. It required additionally giving the
source /opt/intel/bin/compilervars.sh intel64 export CC="icc -std=c99" export CXX="icpc" export F77="ifort" export FC="ifort" export AR="xiar" export LD="xild" export CFLAGS="-openmp -O3 -ipo -xHost -multiple-processes" export CXXFLAGS="-openmp -O3 -ipo -xHost -multiple-processes" export FFLAGS="-openmp -O3 -ipo -xHost -multiple-processes" export FCFLAGS="-openmp -O3 -ipo -xHost -multiple-processes" export MKL="-lmkl_rt" export MAIN_LDFLAGS="-openmp" ./configure --with-blas="$MKL" --with-lapack --with-tcltk make sudo -E env "LD_LIBRARY_PATH=$LD_LIBRARY_PATH" env "PATH=$PATH" make install
sudocommand access to my
LD_LIBRARY_PATHenvironmental variable compared to previously. This variable includes the path to the Intel libraries.