ViewVC logotype

Contents of /branches/doubleplusgood/doc/install/srcommon.tex

Parent Directory Parent Directory | Revision Log Revision Log

Revision 4345 - (show annotations)
Fri Mar 29 07:09:41 2013 UTC (6 years, 9 months ago) by jfenwick
File MIME type: application/x-tex
File size: 12149 byte(s)
Spelling fixes
1 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
2 % Copyright (c) 2003-2013 by University of Queensland
3 % http://www.uq.edu.au
4 %
5 % Primary Business: Queensland, Australia
6 % Licensed under the Open Software License version 3.0
7 % http://www.opensource.org/licenses/osl-3.0.php
8 %
9 % Development until 2012 by Earth Systems Science Computational Center (ESSCC)
10 % Development since 2012 by School of Earth Sciences
11 %
12 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
14 % This file contains material common to all src distributions.
16 % The original version of this content came from the esscc twiki page maintained by ksteube
18 This chapter describes how to build \esfinley from source assuming that the dependencies are already installed (for example using precompiled packages for your OS).
19 Section~\ref{sec:deps} describes the dependencies, while Section~\ref{sec:compilesrc} gives the compile instructions.
21 If you would prefer to build all the dependencies from source in the escript-support packages please see Chapter~\ref{chap:allsrc}.
22 \esfinley is known to compile and run on the following systems:
23 \begin{itemize}
24 \item \linux using gcc
25 \item \linux using icc on SGI ICE 8200. (We do not recommend building with intel-11)
26 \item \macosx using gcc or clang
27 \item \winxp using the Visual C compiler (we do not specifically discuss Windows builds in this guide).
28 \end{itemize}
30 If you have compiled a previous version of \esfinley, the \file{..._options.py} file has the same format
31 as in the previous release, so you can reuse it.
34 \section{External dependencies}
35 \label{sec:deps}
36 The following external packages are required in order to compile and run \esfinley.
37 Where version numbers are specified, more recent versions can probably be substituted.
38 You can either try the standard/precompiled packages available for your operating system or you can download and build them from source.
39 The advantage of using existing packages is that they are more likely to work together properly.
40 You must take greater care if downloading sources separately.
42 \begin{itemize}
43 \item python $\geq 2.6$ (\url{http://python.org}) \\-
44 Python interpreter (you must compile with shared libraries.)
45 \item numpy $\geq 1.1.0$ (\url{http://numpy.scipy.org}) \\-
46 Arrays for Python
47 \item boost $\geq 1.35$ (\url{http://www.boost.org}) \\-
48 Interface between C++ and Python
49 \item scons $\geq 0.989.5$ (\url{http://www.scons.org/}) \\-
50 Python-based alternative to \texttt{make}.
51 \end{itemize}
53 The version numbers given here are not strict requirements, more recent (and in some cases older) versions are very likely to work.
54 The following packages should be sufficient (but not necessarily minimal) for Debian 6.0 (``Squeeze''):
55 \texttt{libboost-python-dev, scons, python-numpy, python-sympy, g++}.
57 \noindent The following packages may be required for some of the optional capabilities of the system:
58 \begin{itemize}
59 \item sympy $\geq$ (\url{http://sympy.org}) \\-
60 Used by \texttt{esys.escript.symbolic}.
61 \item netcdf $\geq 3.6.2$ (\url{http://www.unidata.ucar.edu/software/netcdf}) \\-
62 Used to save data sets in binary form for checkpoint/restart (must be compiled with -fPIC)
63 \item parmetis $\geq 3.1$ (\url{http://glaros.dtc.umn.edu/gkhome/metis/parmetis/overview}) \\-
64 Optimization of the stiffness matrix
65 \item MKL \\(\url{http://www.intel.com/cd/software/products/asmo-na/eng/307757.htm}) \\-
66 Intel's Math Kernel Library for use with their C compiler.
67 \item Lapack - Available in various versions from various places. \\
68 Currently only used to invert dense square matrices larger than 3x3.
69 \item gmsh $\geq 2.2.0$ (\url{http://www.geuz.org/gmsh}) \\-
70 Mesh generation and viewing [esys.pycad uses this]
71 \end{itemize}
73 \noindent Mesh generation: as well as \texttt{gmsh} above you could also use:
74 \begin{itemize}
75 \item triangle $\geq 1.6$ (\url{http://www.cs.cmu.edu/~quake/triangle.html}) \\-
76 Two-dimensional mesh generator and Delaunay triangulator.
77 \end{itemize}
79 Packages for visualization:
80 \begin{itemize}
81 \item mayavi $\geq 1.5$ (\url{http://mayavi.sourceforge.net}) \\-
82 MayaVi is referenced in our User's Guide for viewing VTK files
83 \item visit $\geq 1.11.2$ (\url{https://wci.llnl.gov/codes/visit/}) \\-
84 A powerful visualisation system with movie-making capabilities.
85 \end{itemize}
89 The source code comes with an extensive set of unit tests. If you would like to
90 build those to verify your installation you need:
91 \begin{itemize}
92 \item cppunit $\geq 1.12.1$ (\url{http://cppunit.sourceforge.net})
93 \end{itemize}
95 \section{Compilation}\label{sec:compilesrc}
96 Throughout this section we will assume that the source code is uncompressed in a directory called \file{escript.d}.
97 You can call the directory anything you like, provided that you make the change before you compile.
99 You need to indicate where to find the external dependencies.
100 To do this, create a file in the \file{escript.d/scons} directory called \file{x_options.py} where ``x'' is the name of your computer (output of the \texttt{hostname} command).
101 Please note that if your hostname has non-alphanumeric characters in it (eg - ) you need to replace them with underscores.
102 For example the options file for \texttt{bob-desktop} would be named \file{bob_desktop_options.py}.
104 From now on all paths will be relative to the top level of the source.
105 As a starting point copy the contents of one of the following files into your options file:
106 \begin{itemize}
107 \item \file{scons/TEMPLATE_linux.py} (\linux and \macosx desktop)
108 \item \file{scons/TEMPLATE_windows.py} (\winxp)
109 \end{itemize}
111 This options file controls which features and libraries your build of escript will attempt to use.
112 For example to use OpenMP or MPI you will need to enable it here.
113 If you want to try escript out without customising your build, then change
114 directories to \file{escript.d} and enter
115 \begin{shellCode}
116 scons
117 \end{shellCode}
118 If this works you can skip to Section~\ref{sec:diff}.
119 If not, then you will need to make some modications to the file.
120 Read on.
122 The template files contain all available options with a comment explaining the
123 purpose of each.
124 Check through the file and ensure that the relevant paths and names are correct
125 for your system and that you enable optional components that you wish to use.
126 For example, to use netCDF, find the netcdf-related lines, uncomment them
127 (i.e. remove the \# at the beginning of the lines) and change them according
128 to your installation:
129 \begin{shellCode}
130 netcdf = True
131 netcdf_prefix = '/opt/netcdf4'
132 netcdf_libs = ['netcdf_c++', 'netcdf']
133 \end{shellCode}
135 In this example, netCDF \emph{header} files must be located in
136 \file{/opt/netcdf4/include}\footnote{or \ldots/include32 or \ldots/include64 or \ldots/inc}
137 and the \emph{libraries} in \file{/opt/netcdf4/lib}\footnote{or \ldots/lib32 or \ldots/lib64}.
138 If this scheme does not apply to your installation then you may also specify
139 the include-path and library-path directly like so:
140 \begin{shellCode}
141 netcdf_prefix = ['/usr/local/include/netcdf', '/usr/local/lib']
142 \end{shellCode}
143 The order is important: the first element in the list is the
144 \emph{include}-path, the second element is the \emph{library}-path and both
145 must be specified.
147 If a line in the options file is commented out and you do not require the
148 feature, then it can be ignored.
149 To actually compile (if you have $n$ processors, then you can use \texttt{scons -j$n$} instead):
151 \begin{shellCode}
152 cd escript.d
153 scons
154 \end{shellCode}
156 As part of its output, scons will tell you the name of the options file it used
157 as well as a list of features and whether they are enabled for your build.
158 If you enabled an optional dependency and the library or include files could
159 not be found you will be notified and the build will stop.
161 Note, that you can override all settings from the options-file on the scons
162 command line. For example, if you usually build an optimized version but would
163 like to build a debug version into a separate directory without changing your
164 default settings, you can use:
165 \begin{shellCode}
166 scons debug=1 prefix=debugbuild
167 \end{shellCode}
168 This will install the binaries and libraries built in debug mode into
169 directories underneath \file{./debugbuild}.
171 To run the unit test suite that comes with the source code issue
172 \begin{shellCode}
173 scons py_tests
174 \end{shellCode}
175 (If you have cppunit installed you can run additional tests using \texttt{scons all_tests}.
177 Grab a coffee or two while the tests compile and run.
178 An alternative method is available for running tests on \openmp and \mpi builds.
180 \subsection{Compilation with \openmp}
181 \openmp is generally enabled by setting compiler and linker switches. For the
182 most common compilers these are automatically set by build system and all you
183 have to do is set the \texttt{openmp} option to True in your options file. If
184 this does not work or your compiler is different, then consult your compiler
185 documentation for the precise switches to use and modify the \texttt{omp_flags}
186 and \texttt{omp_ldflags} variables in your options file.
187 For example, for gcc compilers which support \openmp use:
188 \begin{shellCode}
189 openmp = True
190 omp_flags = '-fopenmp'
191 omp_ldflags = '-fopenmp'
192 \end{shellCode}
193 (The two latter settings can also be left out as this is the default OpenMP on gcc.)
195 You can test your \openmp-enabled build, e.g. using 4 threads by issuing
196 \begin{shellCode}
198 scons py_tests
199 \end{shellCode}
201 \subsection{Compilation with \mpi}
202 You need to have \mpi preinstalled on your system.
203 There are a number of implementations so we do not provide any specific advice
204 here.
205 Set the following variables in your options file to according to your
206 installation:
207 \begin{itemize}
208 \item \texttt{mpi} \\
209 which \mpi implementation (flavour) is used. Valid values are
210 \begin{itemize}
211 \item[\texttt{none}] \mpi is disabled
212 \item[\texttt{MPT}] SGI MPI implementation \\
213 \url{http://techpubs.sgi.com/library/manuals/3000/007-3687-010/pdf/007-3687-010.pdf}
214 \item[\texttt{MPICH}] Argonne's MPICH implementation \\
215 \url{http://www.mcs.anl.gov/research/projects/mpi/mpich1/}
216 \item[\texttt{MPICH2}] Argonne's MPICH version 2 implementation \\
217 \url{http://www.mcs.anl.gov/research/projects/mpi/mpich2/}
218 \item[\texttt{OPENMPI}] Open MPI \\
219 \url{http://www.open-mpi.org/}
220 \item[\texttt{INTELMPI}] Intel MPI \\
221 \url{http://software.intel.com/en-us/intel-mpi-library/}
222 \end{itemize}
223 \item \texttt{mpi_prefix} \\
224 where to find \mpi headers and libraries (see netCDF example above)
225 \item \texttt{mpi_libs} \\
226 which libraries to link to.
227 \end{itemize}
229 To test your build using 6 processes enter:
230 \begin{shellCode}
231 export ESCRIPT_NUM_PROCS=6
232 scons py_tests
233 \end{shellCode}
234 and on $2$ processes with $4$ threads each (provided \openmp is enabled)\footnote{Unless your system has $8$ cores expect this to be slow}:
235 \begin{shellCode}
237 export ESCRIPT_NUM_PROCS=2
238 scons py_tests
239 \end{shellCode}
240 Alternatively, you can give a hostfile
241 \begin{shellCode}
243 export ESCRIPT_HOSTFILE=myhostfile
244 scons py_tests
245 \end{shellCode}
246 Note that depending on your \mpi flavour it may be required to start a daemon
247 before running the tests under \mpi.
249 \subsection{Difficulties}\label{sec:diff}
251 \subsubsection{Mismatch of runtime and build libraries}
252 Most external libraries used by \esfinley are linked dynamically.
253 This can lead to problems if after compiling \esfinley these libraries are
254 updated.
255 The same applies to the installed Python executable and libraries.
256 Whenever these dependencies change on your system you should recompile
257 \esfinley to avoid problems at runtime such as load errors or segmentation
258 faults.
260 \subsubsection{\openmp builds segfault running examples}
261 One known cause for this is linking the \file{gomp} library with escript built using gcc 4.3.3.
262 While you need the \texttt{-fopenmp} switch you should not need to link \file{gomp}.

  ViewVC Help
Powered by ViewVC 1.1.26