/[escript]/trunk/doc/install/srcommon.tex
ViewVC logotype

Contents of /trunk/doc/install/srcommon.tex

Parent Directory Parent Directory | Revision Log Revision Log


Revision 3989 - (show annotations)
Tue Sep 25 02:21:54 2012 UTC (7 years ago) by jfenwick
File MIME type: application/x-tex
File size: 12868 byte(s)
More copyright fixes.
pyvisi traces removed.
Some install doco
1 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
2 % Copyright (c) 2003-2012 by University of Queensland
3 % http://www.uq.edu.au
4 %
5 % Primary Business: Queensland, Australia
6 % Licensed under the Open Software License version 3.0
7 % http://www.opensource.org/licenses/osl-3.0.php
8 %
9 % Development until 2012 by Earth Systems Science Computational Center (ESSCC)
10 % Development since 2012 by School of Earth Sciences
11 %
12 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
13
14 % This file contains material common to all src distributions.
15
16 % The original version of this content came from the esscc twiki page maintained by ksteube
17
18 This chapter describes how to build \esfinley from source assuming that the dependencies are already installed (for example using precompiled packages for your OS).
19 Section~\ref{sec:deps} describes the dependencies, while Section~\ref{sec:compilesrc} gives the compile instructions.
20
21 If you would prefer to build all the dependecies from source in the escript-support packages please see Chapter~\ref{chap:allsrc}.
22 \esfinley is known to compile and run on the following systems:
23 \begin{itemize}
24 \item \linux using gcc\footnote{There are some problems with \openmp under gcc prior to version 4.3.2. Also do not link the gomp library with gcc 4.3.3 - it causes problems.}
25 \item \linux using icc on SGI ICE 8200. (At this stage, we do not recommend building with intel-11)\footnote{There is a bug in icpc-11 related to exception handling and openmp. This results in binaries which crash.}
26 \item \macosx using gcc
27 \item \winxp using the Visual C compiler (we do not specifically discuss Windows builds in this guide).
28 \end{itemize}
29
30 If you have compiled a previous version of \esfinley please note that the format of the \file{..._options.py} file has changed so you will not be able to reuse old options file for this build.
31
32
33 \section{External dependencies}
34 \label{sec:deps}
35 The following external packages are required in order to compile and run \esfinley.
36 Where version numbers are specified, more recent versions can probably be subsituted.
37 You can either try the standard/precompiled packages available for your operating system or you can download and build them from source.
38 The advantage of using existing packages is that they are more likely to work together properly.
39 You must take greater care if downloading sources separately.
40
41 \begin{itemize}
42 \item python-2.5.1 (\url{http://python.org}) \\-
43 Python interpreter (you must compile with shared libraries.)
44 \item numpy 1.1.0 (\url{http://numpy.scipy.org}) \\-
45 Arrays for Python
46 \item boost-1.35 (\url{http://www.boost.org}) \\-
47 Interface between C++ and Python
48 \item scons-0.989.5 (\url{http://www.scons.org/}) \\-
49 Python-based alternative to \texttt{make}.
50 \end{itemize}
51
52 The version numbers given here are not strict requirements, more recent (and in some cases older) versions are very likely to work.
53 The following packages should be sufficient (but not necessarily minimal) for Debian 5.0 (``Lenny''):
54 python-dev, libboost-python1.35-dev, scons, python-numpy, g++.
55
56 These packages may be required for some of the optional capabilities of the system:
57
58 \begin{itemize}
59 \item netcdf-3.6.2 (\url{http://www.unidata.ucar.edu/software/netcdf}) \\-
60 Used to save data sets in binary form for checkpoint/restart (must be compiled with -fPIC)
61 % \item vtk-5.0.4 (\url{http://www.vtk.org}) \\-
62 % Used to save VTK files for visualization
63 % \begin{itemize}
64 % \item cmake-2.4.6 (\url{http://www.cmake.org}) \\-
65 % Required to build VTK
66 % \item mesa-7.0.3 (\url{http://www.mesa3d.org})\\-
67 % Free OpenGL replacement used by VTK
68 % \end{itemize}
69 \item netpbm (\url{http://netpbm.sourceforge.com}) \\-
70 Tools for producing movies from images
71 \item parmetis-3.1 (\url{http://glaros.dtc.umn.edu/gkhome/metis/parmetis/overview}) \\-
72 Optimization of the stiffness matrix
73 \item MKL \\(\url{http://www.intel.com/cd/software/products/asmo-na/eng/307757.htm}) \\-
74 Intel's Math Kernel Library for use with their C compiler.
75 \item Lapack - Available in various versions from various places. \\
76 Currently only used to invert dense square matrices larger than 3x3.
77 \end{itemize}
78
79 The following packages might be useful for mesh generation:
80 \begin{itemize}
81 \item gmsh-2.2.0 (\url{http://www.geuz.org/gmsh}) \\-
82 Mesh generation and viewing
83 \begin{itemize}
84 \item fltk-1.1.9 (\url{http://www.fltk.org}) \\-
85 Required to build gmsh
86 \item gsl-1.10 (\url{http://www.gnu.org/software/gsl}) \\-
87 Required to build gmsh
88 \end{itemize}
89 \item triangle-1.6 (\url{http://www.cs.cmu.edu/~quake/triangle.html}) \\-
90 Two-dimensional mesh generator and Delaunay triangulator.
91 \end{itemize}
92
93 Packages for visualization:
94 \begin{itemize}
95 \item mayavi-1.5 (\url{http://mayavi.sourceforge.net}) \\-
96 MayaVi is referenced in our User's Guide for viewing VTK files
97 \item visit-1.11.2 (\url{https://wci.llnl.gov/codes/visit/}) \\-
98 A featureful visualization system with movie-making capabilities.
99 \end{itemize}
100
101 The source code comes with an extensive set of unit tests. If you would like to
102 build those to verify your installation you need:
103 \begin{itemize}
104 \item cppunit-1.12.1 (\url{http://cppunit.sourceforge.net})
105 \end{itemize}
106
107 \section{Compilation}\label{sec:compilesrc}
108 Throughout this section we will assume that the source code is uncompressed in a directory called \file{escript.d}.
109 You can call the directory anything you like, provided that you make the change before you compile.
110
111 You need to indicate where to find the external dependencies.
112 To do this, create a file in the \file{escript.d/scons} directory called \file{x_options.py} where ``x'' is the name of your computer (output of the \texttt{hostname} command).
113 Please note that if your hostname has non-alphanumeric characters in it (eg - ) you need to replace them with underscores.
114 For example the options file for \texttt{bob-desktop} would be named \file{bob_desktop_options.py}.
115
116 From now on all paths will be relative to the top level of the source.
117 As a starting point copy the contents of one of the following files into your options file:
118 \begin{itemize}
119 \item \file{scons/TEMPLATE_linux.py} (\linux and \macosx desktop)
120 \item \file{scons/TEMPLATE_windows.py} (\winxp)
121 \end{itemize}
122
123 This options file controls which features and libraries your build of escript will attempt to use.
124 For example to use OpenMP or MPI you will need to enable it here.
125 If you want to try escript out without customising your build, then change
126 directories to \file{escript.d} and enter
127 \begin{shellCode}
128 scons
129 \end{shellCode}
130 If this works you can skip to Section~\ref{sec:diff}.
131 If not, then you will need to make some modications to the the file.
132 Read on.
133
134 The template files contain all available options with a comment explaining the
135 purpose of each.
136 Check through the file and ensure that the relevant paths and names are correct
137 for your system and that you enable optional components that you wish to use.
138 For example, to use netCDF, find the netcdf-related lines, uncomment them
139 (i.e. remove the \# at the beginning of the lines) and change them according
140 to your installation:
141 \begin{shellCode}
142 netcdf = True
143 netcdf_prefix = '/opt/netcdf4'
144 netcdf_libs = ['netcdf_c++', 'netcdf']
145 \end{shellCode}
146
147 In this example, netCDF \emph{header} files must be located in
148 \file{/opt/netcdf4/include}\footnote{or \ldots/include32 or \ldots/include64 or \ldots/inc}
149 and the \emph{libraries} in \file{/opt/netcdf4/lib}\footnote{or \ldots/lib32 or \ldots/lib64}.
150 If this scheme does not apply to your installation then you may also specify
151 the include-path and library-path directly like so:
152 \begin{shellCode}
153 netcdf_prefix = ['/usr/local/include/netcdf', '/usr/local/lib']
154 \end{shellCode}
155 The order is important: the first element in the list is the
156 \emph{include}-path, the second element is the \emph{library}-path and both
157 must be specified.
158
159 If a line in the options file is commented out and you do not require the
160 feature, then it can be ignored.
161 To actually compile (if you have $n$ processors, then you can use \texttt{scons -j$n$} instead):
162
163 \begin{shellCode}
164 cd escript.d
165 scons
166 \end{shellCode}
167
168 As part of its output, scons will tell you the name of the options file it used
169 as well as a list of features and whether they are enabled for your build.
170 If you enabled an optional dependency and the library or include files could
171 not be found you will be notified and the build will stop.
172
173 Note, that you can override all settings from the options-file on the scons
174 command line. For example, if you usually build an optimized version but would
175 like to build a debug version into a separate directory without changing your
176 default settings, you can use:
177 \begin{shellCode}
178 scons debug=1 prefix=debugbuild
179 \end{shellCode}
180 This will install the binaries and libraries built in debug mode into
181 directories underneath \file{./debugbuild}.
182
183 To run the unit test suite that comes with the source code issue
184 \begin{shellCode}
185 scons py_tests
186 \end{shellCode}
187 (If you have cppunit installed you can run additional tests using \texttt{scons all_tests}.
188
189 Grab a coffee or two while the tests compile and run.
190 An alternative method is available for running tests on \openmp and \mpi builds.
191
192 \subsection{Compilation with \openmp}
193 \openmp is generally enabled by setting compiler and linker switches. For the
194 most common compilers these are automatically set by build system and all you
195 have to do is set the \texttt{openmp} option to True in your options file. If
196 this does not work or your compiler is different, then consult your compiler
197 documentation for the precise switches to use and modify the \texttt{omp_flags}
198 and \texttt{omp_ldflags} variables in your options file.
199 For example, for gcc compilers which support \openmp use:
200 \begin{shellCode}
201 openmp = True
202 omp_flags = '-fopenmp'
203 omp_ldflags = '-fopenmp'
204 \end{shellCode}
205 (The two latter settings can also be left out as this is the default OpenMP on gcc.)
206
207 You can test your \openmp-enabled build, e.g. using 4 threads by issuing
208 \begin{shellCode}
209 export ESCRIPT_NUM_THREADS=4
210 scons py_tests
211 \end{shellCode}
212
213 \subsection{Compilation with \mpi}
214 You need to have \mpi preinstalled on your system.
215 There are a number of implementations so we do not provide any specific advice
216 here.
217 Set the following variables in your options file to according to your
218 installation:
219 \begin{itemize}
220 \item \texttt{mpi} \\
221 which \mpi implementation (flavour) is used. Valid values are
222 \begin{itemize}
223 \item[\texttt{none}] \mpi is disabled
224 \item[\texttt{MPT}] SGI MPI implementation \\
225 \url{http://techpubs.sgi.com/library/manuals/3000/007-3687-010/pdf/007-3687-010.pdf}
226 \item[\texttt{MPICH}] Argonne's MPICH implementation \\
227 \url{http://www.mcs.anl.gov/research/projects/mpi/mpich1/}
228 \item[\texttt{MPICH2}] Argonne's MPICH version 2 implementation \\
229 \url{http://www.mcs.anl.gov/research/projects/mpi/mpich2/}
230 \item[\texttt{OPENMPI}] Open MPI \\
231 \url{http://www.open-mpi.org/}
232 \item[\texttt{INTELMPI}] Intel MPI \\
233 \url{http://software.intel.com/en-us/intel-mpi-library/}
234 \end{itemize}
235 \item \texttt{mpi_prefix} \\
236 where to find \mpi headers and libraries (see netCDF example above)
237 \item \texttt{mpi_libs} \\
238 which libraries to link to.
239 \end{itemize}
240
241 To test your build using 6 processes enter:
242 \begin{shellCode}
243 export ESCRIPT_NUM_PROCS=6
244 scons py_tests
245 \end{shellCode}
246 and on $2$ processes with $4$ threads each (provided \openmp is enabled)\footnote{Unless your system has $8$ cores expect this to be slow}:
247 \begin{shellCode}
248 export ESCRIPT_NUM_THREADS=4
249 export ESCRIPT_NUM_PROCS=2
250 scons py_tests
251 \end{shellCode}
252 Alternatively, you can give a hostfile
253 \begin{shellCode}
254 export ESCRIPT_NUM_THREADS=4
255 export ESCRIPT_HOSTFILE=myhostfile
256 scons py_tests
257 \end{shellCode}
258 Note that depending on your \mpi flavour it may be required to start a daemon
259 before running the tests under \mpi.
260
261 \subsection{Difficulties}\label{sec:diff}
262
263 \subsubsection{Mismatch of runtime and build libraries}
264 Most external libraries used by \esfinley are linked dynamically.
265 This can lead to problems if after compiling \esfinley these libraries are
266 updated.
267 The same applies to the installed Python executable and libraries.
268 Whenever these dependencies change on your system you should recompile
269 \esfinley to avoid problems at runtime such as load errors or segmentation
270 faults.
271
272 \subsubsection{\openmp builds segfault running examples}
273 One known cause for this is linking the \file{gomp} library with escript built using gcc 4.3.3.
274 While you need the \texttt{-fopenmp} switch you should not need to link \file{gomp}.
275
276

  ViewVC Help
Powered by ViewVC 1.1.26