Showing posts with label quantum espresso. Show all posts
Showing posts with label quantum espresso. Show all posts

25 July 2012

216. Quantum Espresso on ROCKS 5.4.3 / CentOs 5.6 -- Almost got it.

Preamble:

I've seen just about any error message imaginable when compiling Quantum Espresso. You can most likely get around them by following this guide ad verbatim 
(no parallel environment detected, can't build fortran programs, no linker, parallel vs serial compiler etc.)

This compile is nothing like the easy-breezy one on debian, because of one reason: CentOS 5.6 is old. It's so old that the default gcc version can't compile QE. You'd encounter similar problems with an increasing number of software packages, including ABINIT.

      CentOS 5.6: gcc version 4.1.2 20080704 (Red Hat 4.1.2-50)
      Wheezy:       gcc version 4.7.1 (Debian 4.7.1-2)

The crusty compiler collection means we have to hammer out errors like these during compilation:


In file dfile_star.f90:28
    INTEGER,ALLOCATABLE ::  npert (:), irgq (:)
                      1
Error: Attribute at (1) is not allowed in a TYPE definition
 In file dfile_star.f90:37

    REAL(DP),ALLOCATABLE :: gi (:,:), gimq (:), eigen(:)
                       1
Error: Attribute at (1) is not allowed in a TYPE definition
 In file dfile_star.f90:42

    COMPLEX(DP),ALLOCATABLE :: u(:,:), t(:,:,:,:), tmq (:,:,:)
                          1
Error: Attribute at (1) is not allowed in a TYPE definition


Hence the requirement to first compile GCC 4.7, even if it takes a couple of hours. 

To the students out there swearing (and sweating) over their professors' insistence on using ROCKS/CentOS: compiling a compiler is not a wasted skill and just might be useful in building cross compilation environments for the time when you decide that science can take a hike and there's more money in working on setting up embedded systems. At least it'll buy you a little geek cred. 

Also, it seems that you need to use math and mpi libs compiled with the same compiler as you use to compile quantum espresso. So...lots of steps today as well!

Also, I haven't reported the configure bug for openmpi 1.6. Feel free to reproduce and report it.

I'm not happy that Quantum Espresso ignores the compile flags I've given it and defaults to things it should not default. I've had to jump through way too many hoops to be happy about this. OpenMPI not supporting program-suffix annoys me as well.

This is another massive compile...I'd rate it as my most annoying, troublesome and infuriating one yet.


1. Build GCC (gfortran, g++, gcc) 4.7 according to this post:
 http://verahill.blogspot.com.au/2012/07/compiling-gcc-471gfortran-471-on-centos.html
Make sure that
export PATH=$PATH:/share/apps/tools/gcc/gcc47/bin

2. Create target directory structure
sudo mkdir /share/apps/libs
sudo chown $USER /share/apps/lib

3. Recompile GNU OpenMP (libgomp)
It's probably easier you start with a freshly extracted gcc-4.7.1
cd ~/tmp/gcc
mkdir newgcc
cp gcc-4.7.1.tar.gz newgcc/
cd newgcc
tar xvf gcc-4.7.1
cd gcc-4.7.1/libgomp
mkdir build
cd build/
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/share/apps/tools/gcc/mpc/lib:/share/apps/tools/gcc/gmp/lib:/share/apps/tools/gcc/mpfr/lib
.././configure CC=gcc-gcc-4.7 CXX=g++-gcc-4.7 F77=gfortran-gcc-4.7 FC=gfortran-gcc-4.7 --prefix=/share/apps/tools/gcc/gcc47

md5sum /share/apps/tools/gcc/gcc47/lib/libgomp.so.1.0.0
23db68121aacf1e7d896704474e26350  /share/apps/tools/gcc/gcc47/lib/libgomp.so.1.0.0

make
make install

md5sum /share/apps/tools/gcc/gcc47/lib/libgomp.so.1.0.0
791a1ceb0ea4009e22703b82e8ce1ff4  /share/apps/tools/gcc/gcc47/lib/libgomp.so.1.0.0

4.  Compile OpenMPI using your new GCC 
cd ~/tmp/
wget http://www.open-mpi.org/software/ompi/v1.6/downloads/openmpi-1.6.tar.gz
tar xvf openmpi-1.6.tar.gz
cd openmpi-1.6
mkdir build
cd build/
export LD_LIBRARY_PATH=/share/apps/tools/gcc/gcc47/lib64:/share/apps/tools/gcc/gcc47/lib/gcc/x86_64-unknown-linux-gnu/4.7.1:/share/apps/tools/gcc/gmp/lib:/share/apps/tools/gcc/mpc/lib:/share/apps/tools/gcc/mpfr/lib

.././configure --prefix=/share/apps/libs/openmpi47 CC=gcc-gcc-4.7 FC=gfortran-gcc-4.7 F77=gfortran-gcc-4.7 F90=gfortran-gcc-4.7 CPP=cpp-gcc-4.7 CXX=g++-gcc-4.7 --with-sge LDFLAGS="-L/share/apps/tools/gcc/gcc47/lib64 -L/share/apps/tools/gcc/gcc47/lib/gcc/x86_64-unknown-linux-gnu/4.7.1 -L/share/apps/tools/gcc/gmp/lib -L/share/apps/tools/gcc/mpc/lib -L/share/apps/tools/gcc/mpfr/lib -L/share/apps/tools/gcc/gcc47/lib" CPPFLAGS="-I/share/apps/tools/gcc/mpc/include -I/share/apps/tools/gcc/gmp/include -I/share/apps/tools/gcc/mpfr/include -I/share/apps/tools/gcc/gcc47/lib/gcc/x86_64-unknown-linux-gnu/4.7.1/include"

** There's a bug in the openmpi configure script leading to config failure due to to an extra space being introduced in front of -L when testing the F77 compiler. Exporting the LD_LIBRARY_PATH is done to get around that. **

make
make install

cd /share/apps/libs/openmpi47bin/
ls mpi* |xargs -I {} ln -s {} {}-gcc-4.7
echo 'export PATH=$PATH:/share/apps/libs/openmpi47/bin' >> ~/.bashrc
source ~/.bashrc

cd /share/apps/libs/openmpi47share/openmpi/
ln -s mpif77-vt-wrapper-data.txt mpif77-gcc-4.7-vt-wrapper-data.txt
ln -s mpif90-vt-wrapper-data.txt mpif90-gcc-4.7-vt-wrapper-data.txt
ln -s mpicc-vt-wrapper-data.txt mpicc-gcc-4.7-vt-wrapper-data.txt
ln -s mpic++-vt-wrapper-data.txt mpic++-gcc-4.7vt--wrapper-data.txt

ln -s mpicxx-vt-wrapper-data.txt mpicxx-gcc-4.7-vt-wrapper-data.txt
ln -s mpif77-wrapper-data.txt mpif77-gcc-4.7-wrapper-data.txt
ln -s mpif90-wrapper-data.txt mpif90-gcc-4.7-wrapper-data.txt
ln -s mpicc-wrapper-data.txt mpicc-gcc-4.7-wrapper-data.txt
ln -s mpic++-wrapper-data.txt mpic++-gcc-4.7-wrapper-data.txt
ln -s mpicxx-wrapper-data.txt mpicxx-gcc-4.7-wrapper-data.txt


5. Compile FFT using your new GCC
cd ~/tmp
wget ftp://ftp.fftw.org/pub/fftw/fftw-3.3.2.tar.gz
tar xvf fftw-3.3.2.tar.gz

cd fftw-3.3.2/
./configure --enable-float --enable-mpi --enable-threads --with-pic --prefix=/share/apps/libs/fftw-3.3.2-gcc47/single CC=gcc-gcc-4.7 FC=gfortran-gcc-4.7 CPP=cpp-gcc-4.7 MPIF90=mpif90-gcc-4.7 F77=gfortran-gcc-4.7 MPICC=mpicc-gcc-4.7 MPIF77=mpif77-gcc-4.7

make
make install
make check

--------------------------------------------------------------
         FFTW threaded transforms passed basic tests!
--------------------------------------------------------------
..
Executing "mpirun -np 1 ..
..
make[3]: *** [check-local] Error 1
..

That mpirun won't work isn't that surprising since it's still point to /opt/openmpi/bin instead of our mpirun-gcc-4.7

make distclean
./configure --disable-float --enable-mpi --enable-threads --with-pic --prefix=/share/apps/libs/fftw-3.3.2-gcc47/double CC=gcc-gcc-4.7 FC=gfortran-gcc-4.7 CPP=cpp-gcc-4.7 MPIF90=mpif90-gcc-4.7 F77=gfortran-gcc-4.7 MPICC=mpicc-gcc-4.7 MPIF77=mpif77-gcc-4.7

make
make install
make check
--------------------------------------------------------------
         FFTW threaded transforms passed basic tests!
--------------------------------------------------------------
...followed by the same error as above.
It's fine.

6. Optional: Compile openblas using your new GCC
cd /share/apps/tools/gcc/binutils/bin
ln -s ar-gcc-4.7 gcc-ar

cd ~/tmp
wget http://nodeload.github.com/xianyi/OpenBLAS/tarball/v0.1.1
mv v0.1.1 openblas.tar.gz
tar xvf openblas.tar.gz
cd xianyi-OpenBLAS-e6e87a2/
wget http://www.netlib.org/lapack/lapack-3.4.1.tgz

export LD_LIBRARY_PATH=/share/apps/tools/gcc/gmp/lib:/share/apps/tools/gcc/mpc/lib:/share/apps/tools/gcc/mpfr/lib

make all BINARY=64 CC=gcc-gcc-4.7 FC=gfortran-gcc-4.7 MPICC=mpicc-gcc-4.7 USE_THREAD=0 INTERFACE64=1 

make PREFIX=/share/apps/libs/openblas47 install

cp libopenblas_* /share/apps/libs/openblas47/lib
cd /share/apps/libs/openblas47/lib
ln -s libopenblas_nehalem-r0.1.1.so libopenblas.so.1
ln -s libopenblas.so.1 libopenblas.so
ln -s libopenblas.so libopenblas.so.0
ln -s libopenblas_nehalem-r0.1.1.a libopenblas.a

7. Compile Quantum Espresso using your new GCC
sudo mkdir /share/apps/QE
sudo chown $USER /share/apps/QE
mkdir /share/apps/QE/lapack-3.2

mkdir ~/tmp/QE
cd ~/tmp/QE

wget http://qe-forge.org/frs/download.php/211/espresso-5.0.tar.gz
wget http://qe-forge.org/frs/download.php/214/PWgui-5.0.tgz
wget http://qe-forge.org/frs/download.php/204/xspectra-5.0.tar.gz

Edit environmental_variables
PREFIX=/share/apps/QE
TMP_DIR=/tmp/QE
PARA_PREFIX="mpirun-gcc-4.7 -n 8"

Time to configure and compile:
cd ~/tmp/QE
tar xvf espresso-5.0.tar.gz
cd espresso-5.0/
cp lapack-3.2 -R /share/apps/QE/
cp BLAS -R /share/apps/QE/

Create a script called compile.sh:

make clean
export PATH=""
export PATH="/usr/bin:/usr/sbin:/bin:/sbin:/usr/local/ecce/apps/scripts/share/apps/pvm/pvm3/bin/LINUX64:/share/apps/libs/openmpi47/bin:/share/apps/nwchem/nwchem-6.1/bin/LINUX64:/share/apps/sinfo/bin:/share/apps/sinfo/sbin:/share/apps/gromacs/bin:/share/apps/cmake/bin:/share/apps/tools/babel/bin:/share/apps/tools/htop/bin:/share/apps/tools/xmgrace/grace/bin:/share/apps/tools/rasmol/src:/share/apps/tools/strace/bin:/share/apps/tools/gcc/gcc47/bin:/share/apps/tools/gcc/binutils/bin"
which mpif90
read -p "press any key"
export CC=gcc-gcc-4.7
export FC=gfortran-gcc-4.7
export MPIF90=mpif90-gcc-4.7
export MPIF77=mpif77-gcc-4.7
export F77=gfortran-gcc-4.7
export F90=gfortran-gcc-4.7
export CPP=cpp-gcc-4.7
export CXX=g++-gcc-4.7
export CFLAGS=""
export DFLAGS="-DFFT_FFTW3"
export CPPFLAGS="-I/share/apps/tools/gcc/mpc/include -I/share/apps/tools/gcc/gmp/include -I/share/apps/tools/gcc/mpfr/include -I/share/apps/tools/gcc/gcc47/lib/gcc/x86_64-unknown-linux-gnu/4.7.1/include -I/share/apps/libs/openmpi47/include -I//share/apps/tools/gcc/gcc47/lib/gcc/x86_64-unknown-linux-gnu/4.7.1/include"
export LDFLAGS="-L/share/apps/tools/gcc/gcc47/lib -L/share/apps/tools/gcc/gcc47/lib64 -L/share/apps/tools/gcc/gcc47/lib/gcc/x86_64-unknown-linux-gnu/4.7.1 -L/share/apps/tools/gcc/gmp/lib -L/share/apps/tools/gcc/mpc/lib -L/share/apps/tools/gcc/mpfr/lib -L/share/apps/libs/openmpi47/lib -L/lib64 -lc"
#export BLAS_LIBS="-L/share/apps/libs/openblas47/lib -lopenblas" 
export BLAS_LIBS="/share/apps/QE/BLAS/blas.a"
export LAPACK_LIBS="/share/apps/QE/lapack-3.2/lapack.a"
export FFT_LIBS="-L/share/apps/libs/fftw-3.3.2-gcc47/double/lib -lfftw3 -lfftw3_mpi -lfftw3_threads"
export MPI_LIBS="-L/share/apps/libs/openmpi47/lib -l:/share/apps/libs/openmpi47/lib/libmpi.so"
export LD_LIBRARY_PATH=/share/apps/tools/gcc/gmp/lib:/share/apps/tools/gcc/mpfr/lib:/share/apps/tools/gcc/mpc/lib:/share/apps/libs/openmpi47/lib
./configure --prefix=/share/apps/QE/bin|tee conf.log
read -p "Press key to continue"
cd PW
make |tee make.log
cd ../
make all |tee makeall.log

Then run it using

sh compile.sh

Which gives:

--------------------------------------------------------------------
ESPRESSO can take advantage of several optimized numerical libraries
(essl, fftw, mkl...).  This configure script attempts to find them,
but may fail if they have been installed in non-standard locations.
If a required library is not found, the local copy will be compiled.

The following libraries have been found:
  BLAS_LIBS=/share/apps/QE/BLAS?blas.a
  LAPACK_LIBS=/share/apps/QE/lapack-3.2/lapack.a
  FFT_LIBS=-L/share/apps/libs/fftw-3.3.2-gcc47/double/lib -lfftw3 -lfftw3_mpi -lfftw3_threads
  MPI_LIBS=-L/share/apps/libs/openmpi47/lib -l:/share/apps/libs/openmpi47/lib/libmpi.so
Please check if this is what you expect.

If any libraries are missing, you may specify a list of directories
to search and retry, as follows:
  ./configure LIBDIRS="list of directories, separated by spaces"

Parallel environment detected successfully.\
Configured for compilation of parallel executables.

For more info, read the ESPRESSO User's Guide (Doc/users-guide.tex).
--------------------------------------------------------------------
configure: success

and more...Once it's over do

cd ~/tmp/QE/espresso-5.0
cp * -R /share/apps/QE/
echo 'export PATH=$PATH:/share/apps/QE/bin' >> ~/.bashrc
source ~/.bashrc

8. Testing PW -- STUCK
export LD_LIBRARY_PATH=/share/apps/tools/gcc/gmp/lib:/share/apps/tools/gcc/mpfr/lib:/share/apps/tools/gcc/mpc/lib:/share/apps/libs/openmpi47/lib:/share/apps/libs/openblas47/lib:/share/apps/tools/gcc/gcc47/lib64
cd /share/apps/QE/PW/examples/example02
./run_example

/share/apps/QE/PW/examples/example02 : starting

This example shows how to use pw.x to compute the equilibrium geometry
of a simple molecule, CO, and of an Al (001) slab.
In the latter case the relaxation is performed in two ways:
1) using the quasi-Newton BFGS algorithm
2) using a damped dynamics algorithm.

  executables directory: /share/apps/QE/bin
  pseudo directory:      /share/apps/QE/pseudo
  temporary directory:   /tmp/QE
  checking that needed directories and files exist...
Downloading O.pz-rrkjus.UPF to /share/apps/QE/pseudo...
Downloading C.pz-rrkjus.UPF to /share/apps/QE/pseudo... done

  running pw.x as: mpirun-gcc-4.7 -n 8  /share/apps/QE/bin/pw.x  

  cleaning /tmp/QE... done
  running the geometry relaxation for CO... from test_input_xml: Empty input file .. stopping
 from test_input_xml: Empty input file .. stopping
 from test_input_xml: Empty input file .. stopping
 from test_input_xml: Empty input file .. stopping
 from test_input_xml: Empty input file .. stopping
 from test_input_xml: Empty input file .. stopping
STOP 2
STOP 2
--------------------------------------------------------------------------
mpirun-gcc-4.7 noticed that the job aborted, but has no info as to the process
that caused that situation.
--------------------------------------------------------------------------
Error condition encountered during test: exit status = 2
Aborting

It's more insidious than that though -- pw.x crashes with "STOP 2" for any type of input.

NOTE: Like on Debian I had segfaults happening when using my own openblas (which works fine with gromacs and nwchem). Hence why I show you how to compile it, yet never use it.

19 July 2012

209. Quantum Espresso on Debian

Quantum Espresso seems to be a fairly capable software package for ab initio QM and MD calculations. In their own words:
"Quantum ESPRESSO is an integrated suite of Open-Source computer codes for electronic-structure calculations and materials modeling at the nanoscale.It is based on density-functional theory, plane waves, and pseudopotentials."

Reading between the lines it seems to be particularly geared towards solid state simulations, but given that I haven't used it much (I'm just an interested observer), you may take that statement with a grain of salt.

Anyway. Here's how to get it up and running.
Don't ask me how to USE these pieces of software though. For that, read the documentation at the Quantum Espresso website or look in /opt/QE/Doc


The download page can be found here: http://qe-forge.org/frs/?group_id=10. You won't need all the packages, since the espresso-5.0.tar.gz bundles most of them.

1. Housekeeping:
sudo mkdir /opt/QE
sudo chown $USER /opt/QE
mkdir ~/tmp/QE -p
cd ~/tmp/QE

2. Downloading:
wget http://qe-forge.org/frs/download.php/211/espresso-5.0.tar.gz
wget http://qe-forge.org/frs/download.php/214/PWgui-5.0.tgz
wget http://qe-forge.org/frs/download.php/204/xspectra-5.0.tar.gz

3. Extraction:
tar xvf espresso-5.0.tar.gz
tar xvf PWgui-5.0.tgz
tar xvf xspectra-5.0.tar.gz

4. Compilation:
cd espresso-5.0/

Edit environment_variables and set them to e.g.

PREFIX=/opt/QE
TMP_DIR=/scratch
PARA_PREFIX=" mpirun -n 3"

Don't know if any of those params are ever read though.

The following parameters will depend on your system. I tried compiling with openblas without luck. who knows? It might be due to mixing debian fftw3 and my own openblas.

Instead, install libblas-dev, libfftw3-dev, libopenmpi-dev etc.

./configure --prefix=/opt/QE/bin --exec-prefix=/opt/QE/bin/ FC=mpif90 BLAS_LIBS=-lblas LIBS="-lmpi -lopen-rte -lopen-pal -ldl -lmpi_f77 -lpthread" CPPFLAGS="-I/usr/lib/openmpi/include"
cd PW/
make

cd ../
make all -j5 
echo 'export PATH=$PATH:/opt/QE/bin' >>~/.bashrc
echo 'export PSEUDO_DIR=/opt/QE/pseudo' >>~/.bashrc
(replace 5 with  the number of cores you compile with +1)

cp * -R /opt/QE/
source ~/.bashrc

For some reason I had to move everything by hand. Oh well.

[It should be enough to set the env var PSEUDO_DIR to point at /opt/QE/pseudo, but it didn't work for me. Instead I symmlinked the entire /opt/QE to ~/espresso. Desperate? Sure...
ln -s /opt/QE/ /home/me/espresso]


5. Testing PW
Some examples are found in /opt/QE/PW/examples
cd /opt/QE/PW/examples/example01
./run_examples

/opt/QE/PW/examples/example01 : starting
This example shows how to use pw.x to calculate the total energy and
the band structure of four simple systems: Si, Al, Cu, Ni.
  executables directory: /opt/QE/bin
  pseudo directory:      /opt/QE/pseudo
  temporary directory:   /scratch
  checking that needed directories and files exist... done
  running pw.x as: mpirun -n 3 /opt/QE/bin/pw.x
  cleaning /scratch... done
  running the scf calculation for Si... done
  running the band-structure calculation for Si... done
  cleaning /scratch... done
  ..
  cleaning /scratch... done
  running the scf calculation for Ni... done
  running the band-structure calculation for Ni... done
Or you can try

cd /opt/QE/PW/tests/
./check-pw.x.j
A number of tests will now be executed.  Most will pass.
IF YOU DON'T HAVE A vdw_kernel_table file one of the tests will fail.

6. Installing PWgui
Assuming you downloaded and extracted the PWgui-5.0.tgz file in step 2 above.
sudo apt-get install itk3 iwidgets4
cd ~/tmp/QE/
mv PWgui-5.0 /opt/QE/
cd /opt/QE/bin
ln -s /opt/QE/PWgui-5.0/pwgui /opt/QE/bin/pwgui

Start by typing
pwgui


settings
You can try e.g. opening /opt/QE/PW/examples/example01/results/al.scf.cg.in if you ran the test in step 5 above. Then run it.


7. Installing Xspectra
This assumes you downloaded and extracted as shown in steps 2-3 above.
cd ~/tmp/QE/
mv XSpectra/ espresso-5.0/
cd espresso-5.0/XSpectra/
make
cd ..
mv XSpectra/ /opt/QE/
cd /opt/QE/bin
ln -s /opt/QE/XSpectra/src/xspectra.x xspectra.x



Notes: in an ideal world the --prefix during configure should suffice in telling a program where to install. No luck for me here though. Also, I had to start the compile in the PW directory and only by using make without any switches, or there would be complaints about a missing libpw.a