05 May 2012

139. compiling nwchem with custom ATLAS on debian

I'm not yet sure, but there seems to be something weird with this build. Use the openblas build here preferentially:  http://verahill.blogspot.com.au/2012/05/nwchem-with-openblas.html

i.e. do not use this build without testing it against builds without external blas/atlas and against other libraries.

I am not an expert

The atlas build is pretty much like what was described here:
http://verahill.blogspot.com.au/2012/05/compile-atlas-blas-on-debian-testing.html


Build atlas
Check you cpu governor:

cat /sys/devices/system/cpu/cpu1/cpufreq/scaling_governor 
ondemand
Next, set it to performance for all cores (example for three-core AMD)
sudo cpufreq-set -g performance
sudo cp /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor /sys/devices/system/cpu/cpu1/cpufreq/scaling_governor
sudo cp /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor /sys/devices/system/cpu/cpu2/cpufreq/scaling_governor

Prepare the install directory:
sudo mkdir /opt/ATLAS
sudo chown ${USER} /opt/ATLAS

Prepare the compile directory
mkdir ~/tmp
cd ~/tmp
 wget http://downloads.sourceforge.net/project/math-atlas/Developer%20%28unstable%29/3.9.72/atlas3.9.72.tar.bz2
tar xvf atlas3.9.72.tar.bz2
cd ATLAS/
mkdir build/
vim Make.top 
change -V on line 6 to -v

cd build
.././configure --prefix=/opt/ATLAS

There's any level of detail in what you can pass to configure. -A AMD64 would be one option. Look at the bottom for more info.

make
make install
cp lib/lib* /opt/ATLAS/lib

Once it's all done, set the cpu governor back if you so desire.
sudo cpufreq-set -g ondemand
sudo cp /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor /sys/devices/system/cpu/cpu1/cpufreq/scaling_governor
sudo cp /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor /sys/devices/system/cpu/cpu2/cpufreq/scaling_governor


nwchem
For python support you need to edit src/config/makefile.h (see http://verahill.blogspot.com.au/2012/04/adding-python-support-to-nwchem-under.html)

export LARGE_FILES=TRUE
export TCGRSH=/usr/bin/ssh
export NWCHEM_TOP=/opt/nwchem/nwchem-6.0
export NWCHEM_TARGET=LINUX64
export NWCHEM_MODULES="all python"
export PYTHONHOME=/usr
export PYTHONVERSION=2.7
export BLASOPT="-L/opt/ATLAS/lib -latlas -lblas -llapack"
export USE_MPI=y
export USE_MPIF=y
export MPI_LOC=/usr/lib/openmpi/lib
export MPI_INCLUDE=/usr/lib/openmpi/include
export LIBRARY_PATH=$LIBRARY_PATH:/usr/lib/openmpi/lib
export LIBMPI="-lmpi -lopen-rte -lopen-pal -ldl -lmpi_f77 -lpthread"
cd $NWCHEM_TOP/src
make clean
make nwchem_config
make FC=gfortran

Note: don't use HAS_BLAS=y. It gave me no end of grief.

Also, IF you binary executes without a hitch at this point it means that you're libs are not being loaded.  You need to put
export LD_LIBRARY_PATH=/opt/ATLAS/lib:$LD_LIBRARY_PATH
in your ~/.bashrc

Building takes a while, but should go ok. Make sure you have python2.7-dev installed, and if you're using a different python version, make sure to change the options above.




Here's also an example ~/.nwchemrc file for no particular reason at all -- if you've got ecce installed you might want to make sure yours is correct.
nwchem_basis_library /opt/nwchem/nwchem-6.0/src/basis/libraries/
nwchem_nwpw_library /opt/nwchem/nwchem-6.0/src/nwps/libraryps/
ffield amber
amber_1 /opt/nwchem/nwchem-6.0/src/data/amber_s/
amber_2 /opt/nwchem/nwchem-6.0/src/data/amber_x/
spce /opt/nwchem/nwchem-6.0/src/data/solvents/spce.rst
charmm_s /opt/nwchem/nwchem-6.0/src/data/charmm_s/
charmm_x /opt/nwchem/nwchem-6.0/src/data/charmm_x/




You can get a list over compile options for ATLAS using
make xprint_enums ; ./xprint_enums

after you've done .././configure

See http://sourceforge.net/tracker/index.php?func=detail&aid=3021404&group_id=23725&atid=379483


No comments:

Post a Comment