/[escript]/trunk/doc/install/srcommon.tex
ViewVC logotype

Contents of /trunk/doc/install/srcommon.tex

Parent Directory Parent Directory | Revision Log Revision Log


Revision 3335 - (show annotations)
Wed Nov 3 05:02:57 2010 UTC (8 years, 2 months ago) by jfenwick
File MIME type: application/x-tex
File size: 12435 byte(s)
Fix the launcher so that it can give an environment even if buildvars is not present.
Environment now includes scons for standalone builds.
Fixes and cleanup for the install documentation.
scons TEMPLATES updated to define the optionfile version by default.

...mumble
   finley no longer imports escript into the default namespace
...
1 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
2 %
3 % Copyright (c) 2003-2010 by University of Queensland
4 % Earth Systems Science Computational Center (ESSCC)
5 % http://www.uq.edu.au/esscc
6 %
7 % Primary Business: Queensland, Australia
8 % Licensed under the Open Software License version 3.0
9 % http://www.opensource.org/licenses/osl-3.0.php
10 %
11 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
12
13 % This file contains material common to all src distributions.
14
15 % The original version of this content came from the esscc twiki page maintained by ksteube
16
17 This chapter describes how to build \esfinley from source assuming that the dependencies are already installed (for example using precompiled packages for your OS).
18 Section~\ref{sec:deps} describes the dependencies, while Section~\ref{sec:compilesrc} gives the compile instructions.
19
20 If you would prefer to build all the dependecies from source in the escript-support packages please see Chapter~\ref{chap:allsrc}.
21 \esfinley is known to compile and run on the following systems:
22 \begin{itemize}
23 \item \linux using gcc\footnote{There are some problems with \openmp under gcc prior to version 4.3.2. Also do not link the gomp library with gcc 4.3.3 - it causes problems.}
24 \item \linux using icc on SGI ICE 8200. (At this stage, we do not recommend building with intel-11)\footnote{There is a bug in icpc-11 related to exception handling and openmp. This results in binaries which crash.}
25 \item \macosx using gcc
26 \item \winxp using the Visual C compiler (we do not specifically discuss Windows builds in this guide).
27 \end{itemize}
28
29 If you have compiled a previous version of \esfinley please note that the format of the \file{..._options.py} file has changed so you will not be able to reuse old options file for this build.
30
31
32 \section{External dependencies}
33 \label{sec:deps}
34 The following external packages are required in order to compile and run \esfinley.
35 Where version numbers are specified, more recent versions can probably be subsituted.
36 You can either try the standard/precompiled packages available for your operating system or you can download and build them from source.
37 The advantage of using existing packages is that they are more likely to work together properly.
38 You must take greater care if downloading sources separately.
39
40 \begin{itemize}
41 \item python-2.5.1 (\url{http://python.org}) \\-
42 Python interpreter (you must compile with shared libraries.)
43 \item numpy 1.1.0 (\url{http://numpy.scipy.org}) \\-
44 Arrays for Python
45 \item boost-1.35 (\url{http://www.boost.org}) \\-
46 Interface between C++ and Python
47 \item scons-0.989.5 (\url{http://www.scons.org/}) \\-
48 Python-based alternative to \texttt{make}.
49 \end{itemize}
50
51 The version numbers given here are not strict requirements, more recent (and in some cases older) versions are very likely to work.
52 The following packages should be sufficient (but not necessarily minimal) for Debian 5.0 (``Lenny''):
53 python-dev, libboost-python1.35-dev, scons, python-numpy, g++.
54
55 These packages may be required for some of the optional capabilities of the system:
56
57 \begin{itemize}
58 \item netcdf-3.6.2 (\url{http://www.unidata.ucar.edu/software/netcdf}) \\-
59 Used to save data sets in binary form for checkpoint/restart (must be compiled with -fPIC)
60 % \item vtk-5.0.4 (\url{http://www.vtk.org}) \\-
61 % Used to save VTK files for visualization
62 % \begin{itemize}
63 % \item cmake-2.4.6 (\url{http://www.cmake.org}) \\-
64 % Required to build VTK
65 % \item mesa-7.0.3 (\url{http://www.mesa3d.org})\\-
66 % Free OpenGL replacement used by VTK
67 % \end{itemize}
68 \item netpbm (\url{http://netpbm.sourceforge.com}) \\-
69 Tools for producing movies from images
70 \item parmetis-3.1 (\url{http://glaros.dtc.umn.edu/gkhome/metis/parmetis/overview}) \\-
71 Optimization of the stiffness matrix
72 \item MKL \\(\url{http://www.intel.com/cd/software/products/asmo-na/eng/307757.htm}) \\-
73 Intel's Math Kernel Library for use with their C compiler.
74 \item Lapack - Available in various versions from various places. \\
75 Currently only used to invert dense square matrices larger than 3x3.
76 \end{itemize}
77
78 The following packages might be useful for mesh generation:
79 \begin{itemize}
80 \item gmsh-2.2.0 (\url{http://www.geuz.org/gmsh}) \\-
81 Mesh generation and viewing
82 \begin{itemize}
83 \item fltk-1.1.9 (\url{http://www.fltk.org}) \\-
84 Required to build gmsh
85 \item gsl-1.10 (\url{http://www.gnu.org/software/gsl}) \\-
86 Required to build gmsh
87 \end{itemize}
88 \item triangle-1.6 (\url{http://www.cs.cmu.edu/~quake/triangle.html}) \\-
89 Two-dimensional mesh generator and Delaunay triangulator.
90 \end{itemize}
91
92 Packages for visualization:
93 \begin{itemize}
94 \item mayavi-1.5 (\url{http://mayavi.sourceforge.net}) \\-
95 MayaVi is referenced in our User's Guide for viewing VTK files
96 \item visit-1.11.2 (\url{https://wci.llnl.gov/codes/visit/}) \\-
97 A featureful visualization system with movie-making capabilities.
98 \end{itemize}
99
100 \section{Compilation}\label{sec:compilesrc}
101 Throughout this section we will assume that the source code is uncompressed in a directory called \file{escript.d}.
102 You can call the directory anything you like, provided that you make the change before you compile.
103
104 You need to indicate where to find the external dependencies.
105 To do this, create a file in the \file{escript.d/scons} directory called \file{x_options.py} where ``x'' is the name of your computer (output of the \texttt{hostname} command).
106 Please note that if your hostname has non-alphanumeric characters in it (eg - ) you need to replace them with underscores.
107 For example the options file for \texttt{bob-desktop} would be named \file{bob_desktop_options.py}.
108
109 From now on all paths will be relative to the top level of the source.
110 As a starting point copy the contents one of the following files into your options file:
111 \begin{itemize}
112 \item \file{scons/TEMPLATE_linux.py} (\linux and \macosx desktop)
113 \item \file{scons/TEMPLATE_windows.py} (\winxp)
114 \end{itemize}
115
116 This options file controls which features and libraries your build of escript will attempt to use.
117 For example to use OpenMP or MPI you will need to enable it here.
118
119 If you want to try escript out without customising your build, then change directories to \file{escript.d} and
120 \begin{shellCode}
121 scons
122 \end{shellCode}
123 If this works you can skip to Section~\ref{sec:diff}.
124 If not, then you will need to make some modications to the the file.
125 Read on.
126
127 The template files contain all available options with a comment explaining the
128 purpose of each.
129 Check through the file and ensure that the relevant paths and names are correct
130 for your system and that you enable optional components that you wish to use.
131 For example, to use netCDF, find the netcdf-related lines, uncomment them
132 (i.e. remove the \# at the beginning of the lines) and change them according
133 to your installation:
134 \begin{shellCode}
135 netcdf = True
136 netcdf_prefix = '/opt/netcdf4'
137 netcdf_libs = ['netcdf_c++', 'netcdf']
138 \end{shellCode}
139
140 In this example, netCDF \emph{header} files must be located in
141 \file{/opt/netcdf4/include}\footnote{or \ldots/include32 or \ldots/include64 or \ldots/inc}
142 and the \emph{libraries} in \file{/opt/netcdf4/lib}\footnote{or \ldots/lib32 or \ldots/lib64}.
143 If this scheme does not apply to your installation then you may also specify
144 the include-path and library-path directly like so:
145 \begin{shellCode}
146 netcdf_prefix = ['/usr/local/include/netcdf', '/usr/local/lib']
147 \end{shellCode}
148 The order is important: the first element in the list is the
149 \emph{include}-path, the second element is the \emph{library}-path and both
150 must be specified.
151
152 If a line in the options file is commented out and you do not require the
153 feature, then it can be ignored.
154
155 To actually compile (if you have $n$ processors, then you can use \texttt{scons -j$n$} instead):
156
157 \begin{shellCode}
158 cd escript.d
159 scons
160 \end{shellCode}
161
162 As part of its output, scons will tell you the name of the options file it used
163 as well as a list of features and whether they are enabled for your build.
164 If you enabled an optional dependency and the library or include files could
165 not be found you will be notified and the build will stop.
166
167 Note, that you can override all settings from the options-file on the scons
168 command line. For example, if you usually build an optimized version but would
169 like to build a debug version into a separate directory without changing your
170 default settings, you can use:
171 \begin{shellCode}
172 scons debug=1 prefix=debugbuild
173 \end{shellCode}
174 This will install the binaries and libraries built in debug mode into
175 directories underneath \file{./debugbuild}.
176
177 To run the unit test suite that comes with the source code issue
178 \begin{shellCode}
179 scons all_tests
180 \end{shellCode}
181 Grab a coffee or two while the tests compile and run.
182 An alternative method is available for running tests on \openmp and \mpi builds.
183
184 \subsection{Compilation with \openmp}
185 \openmp is generally enabled by setting compiler and linker switches. For the
186 most common compilers these are automatically set by build system and all you
187 have to do is set the \texttt{openmp} option to True in your options file. If
188 this does not work or your compiler is different, then consult your compiler
189 documentation for the precise switches to use and modify the \texttt{omp_flags}
190 and \texttt{omp_ldflags} variables in your options file.
191 For example, for gcc compilers which support \openmp use:
192 \begin{shellCode}
193 openmp = True
194 omp_flags = '-fopenmp'
195 omp_ldflags = '-fopenmp'
196 \end{shellCode}
197 (The two latter settings can also be left out as this is the default OpenMP on gcc.)
198
199 You can test your \openmp-enabled build, e.g. using 4 threads by issuing
200 \begin{shellCode}
201 export ESCRIPT_NUM_THREADS=4
202 scons all_tests
203 \end{shellCode}
204
205 \subsection{Compilation with \mpi}
206 You need to have \mpi preinstalled on your system.
207 There are a number of implementations so we do not provide any specific advice
208 here.
209 Set the following variables in your options file to according to your
210 installation:
211 \begin{itemize}
212 \item \texttt{mpi} \\
213 which \mpi implementation (flavour) is used. Valid values are
214 \begin{itemize}
215 \item[\texttt{none}] \mpi is disabled
216 \item[\texttt{MPT}] SGI MPI implementation \\
217 \url{http://techpubs.sgi.com/library/manuals/3000/007-3687-010/pdf/007-3687-010.pdf}
218 \item[\texttt{MPICH}] Argonne's MPICH implementation \\
219 \url{http://www.mcs.anl.gov/research/projects/mpi/mpich1/}
220 \item[\texttt{MPICH2}] Argonne's MPICH version 2 implementation \\
221 \url{http://www.mcs.anl.gov/research/projects/mpi/mpich2/}
222 \item[\texttt{OPENMPI}] Open MPI \\
223 \url{http://www.open-mpi.org/}
224 \item[\texttt{INTELMPI}] Intel MPI \\
225 \url{http://software.intel.com/en-us/intel-mpi-library/}
226 \end{itemize}
227 \item \texttt{mpi_prefix} \\
228 where to find \mpi headers and libraries (see netCDF example above)
229 \item \texttt{mpi_libs} \\
230 which libraries to link to.
231 \end{itemize}
232
233 To test your build using 6 processes enter:
234 \begin{shellCode}
235 export ESCRIPT_NUM_PROCS=6
236 scons all_tests
237 \end{shellCode}
238 and on $2$ processes with $4$ threads each (provided \openmp is enabled)\footnote{Unless your system has $8$ cores expect this to be slow}:
239 \begin{shellCode}
240 export ESCRIPT_NUM_THREADS=4
241 export ESCRIPT_NUM_PROCS=2
242 scons all_tests
243 \end{shellCode}
244 Alternatively, you can give a hostfile
245 \begin{shellCode}
246 export ESCRIPT_NUM_THREADS=4
247 export ESCRIPT_HOSTFILE=myhostfile
248 scons all_tests
249 \end{shellCode}
250 Note that depending on your \mpi flavour it may be required to start a daemon
251 before running the tests under \mpi.
252
253 \subsection{Difficulties}\label{sec:diff}
254
255 \subsubsection{Mismatch of runtime and build libraries}
256 Most external libraries used by \esfinley are linked dynamically.
257 This can lead to problems if after compiling \esfinley these libraries are
258 updated.
259 The same applies to the installed Python executable and libraries.
260 Whenever these dependencies change on your system you should recompile
261 \esfinley to avoid problems at runtime such as load errors or segmentation
262 faults.
263
264 \subsubsection{\openmp builds segfault running examples}
265 One known cause for this is linking the \file{gomp} library with escript built using gcc 4.3.3.
266 While you need the \texttt{-fopenmp} switch you should not need to link \file{gomp}.
267
268

  ViewVC Help
Powered by ViewVC 1.1.26