/[escript]/trunk/doc/install/srcommon.tex
ViewVC logotype

Annotation of /trunk/doc/install/srcommon.tex

Parent Directory Parent Directory | Revision Log Revision Log


Revision 3335 - (hide annotations)
Wed Nov 3 05:02:57 2010 UTC (8 years, 5 months ago) by jfenwick
File MIME type: application/x-tex
File size: 12435 byte(s)
Fix the launcher so that it can give an environment even if buildvars is not present.
Environment now includes scons for standalone builds.
Fixes and cleanup for the install documentation.
scons TEMPLATES updated to define the optionfile version by default.

...mumble
   finley no longer imports escript into the default namespace
...
1 jfenwick 2289 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
2     %
3 jfenwick 2881 % Copyright (c) 2003-2010 by University of Queensland
4 jfenwick 2289 % Earth Systems Science Computational Center (ESSCC)
5     % http://www.uq.edu.au/esscc
6     %
7     % Primary Business: Queensland, Australia
8     % Licensed under the Open Software License version 3.0
9     % http://www.opensource.org/licenses/osl-3.0.php
10     %
11     %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
12    
13     % This file contains material common to all src distributions.
14    
15     % The original version of this content came from the esscc twiki page maintained by ksteube
16    
17 caltinay 2536 This chapter describes how to build \esfinley from source assuming that the dependencies are already installed (for example using precompiled packages for your OS).
18 jfenwick 3335 Section~\ref{sec:deps} describes the dependencies, while Section~\ref{sec:compilesrc} gives the compile instructions.
19 jfenwick 2529
20 jfenwick 3335 If you would prefer to build all the dependecies from source in the escript-support packages please see Chapter~\ref{chap:allsrc}.
21 jfenwick 2289 \esfinley is known to compile and run on the following systems:
22     \begin{itemize}
23 caltinay 2547 \item \linux using gcc\footnote{There are some problems with \openmp under gcc prior to version 4.3.2. Also do not link the gomp library with gcc 4.3.3 - it causes problems.}
24 jfenwick 2923 \item \linux using icc on SGI ICE 8200. (At this stage, we do not recommend building with intel-11)\footnote{There is a bug in icpc-11 related to exception handling and openmp. This results in binaries which crash.}
25 caltinay 2547 \item \macosx using gcc
26 caltinay 2536 \item \winxp using the Visual C compiler (we do not specifically discuss Windows builds in this guide).
27 jfenwick 2289 \end{itemize}
28    
29 jfenwick 3335 If you have compiled a previous version of \esfinley please note that the format of the \file{..._options.py} file has changed so you will not be able to reuse old options file for this build.
30    
31    
32 jfenwick 2289 \section{External dependencies}
33 jfenwick 2529 \label{sec:deps}
34 jfenwick 2289 The following external packages are required in order to compile and run \esfinley.
35     Where version numbers are specified, more recent versions can probably be subsituted.
36     You can either try the standard/precompiled packages available for your operating system or you can download and build them from source.
37 caltinay 2536 The advantage of using existing packages is that they are more likely to work together properly.
38 jfenwick 2289 You must take greater care if downloading sources separately.
39    
40     \begin{itemize}
41 caltinay 2547 \item python-2.5.1 (\url{http://python.org}) \\-
42     Python interpreter (you must compile with shared libraries.)
43     \item numpy 1.1.0 (\url{http://numpy.scipy.org}) \\-
44 jfenwick 3335 Arrays for Python
45 caltinay 2547 \item boost-1.35 (\url{http://www.boost.org}) \\-
46     Interface between C++ and Python
47     \item scons-0.989.5 (\url{http://www.scons.org/}) \\-
48 jfenwick 3335 Python-based alternative to \texttt{make}.
49 jfenwick 2289 \end{itemize}
50    
51 caltinay 2547 The version numbers given here are not strict requirements, more recent (and in some cases older) versions are very likely to work.
52 jfenwick 2289 The following packages should be sufficient (but not necessarily minimal) for Debian 5.0 (``Lenny''):
53 jfenwick 2583 python-dev, libboost-python1.35-dev, scons, python-numpy, g++.
54 jfenwick 2289
55 caltinay 2536 These packages may be required for some of the optional capabilities of the system:
56 jfenwick 2289
57     \begin{itemize}
58     \item netcdf-3.6.2 (\url{http://www.unidata.ucar.edu/software/netcdf}) \\-
59 caltinay 2547 Used to save data sets in binary form for checkpoint/restart (must be compiled with -fPIC)
60 jfenwick 3335 % \item vtk-5.0.4 (\url{http://www.vtk.org}) \\-
61     % Used to save VTK files for visualization
62     % \begin{itemize}
63     % \item cmake-2.4.6 (\url{http://www.cmake.org}) \\-
64     % Required to build VTK
65     % \item mesa-7.0.3 (\url{http://www.mesa3d.org})\\-
66     % Free OpenGL replacement used by VTK
67     % \end{itemize}
68 caltinay 2536 \item netpbm (\url{http://netpbm.sourceforge.com}) \\-
69     Tools for producing movies from images
70     \item parmetis-3.1 (\url{http://glaros.dtc.umn.edu/gkhome/metis/parmetis/overview}) \\-
71 caltinay 2547 Optimization of the stiffness matrix
72 caltinay 2536 \item MKL \\(\url{http://www.intel.com/cd/software/products/asmo-na/eng/307757.htm}) \\-
73     Intel's Math Kernel Library for use with their C compiler.
74 jfenwick 2780 \item Lapack - Available in various versions from various places. \\
75     Currently only used to invert dense square matrices larger than 3x3.
76 jfenwick 2289 \end{itemize}
77    
78     The following packages might be useful for mesh generation:
79     \begin{itemize}
80     \item gmsh-2.2.0 (\url{http://www.geuz.org/gmsh}) \\-
81 caltinay 2547 Mesh generation and viewing
82 jfenwick 2289 \begin{itemize}
83 caltinay 2536 \item fltk-1.1.9 (\url{http://www.fltk.org}) \\-
84 caltinay 2547 Required to build gmsh
85 caltinay 2536 \item gsl-1.10 (\url{http://www.gnu.org/software/gsl}) \\-
86 caltinay 2547 Required to build gmsh
87 caltinay 2536 \end{itemize}
88 caltinay 2547 \item triangle-1.6 (\url{http://www.cs.cmu.edu/~quake/triangle.html}) \\-
89     Two-dimensional mesh generator and Delaunay triangulator.
90 jfenwick 2289 \end{itemize}
91    
92     Packages for visualization:
93     \begin{itemize}
94     \item mayavi-1.5 (\url{http://mayavi.sourceforge.net}) \\-
95 caltinay 2547 MayaVi is referenced in our User's Guide for viewing VTK files
96     \item visit-1.11.2 (\url{https://wci.llnl.gov/codes/visit/}) \\-
97     A featureful visualization system with movie-making capabilities.
98 jfenwick 2289 \end{itemize}
99    
100 jfenwick 2324 \section{Compilation}\label{sec:compilesrc}
101 jfenwick 3322 Throughout this section we will assume that the source code is uncompressed in a directory called \file{escript.d}.
102 jfenwick 2289 You can call the directory anything you like, provided that you make the change before you compile.
103    
104     You need to indicate where to find the external dependencies.
105 jfenwick 3322 To do this, create a file in the \file{escript.d/scons} directory called \file{x_options.py} where ``x'' is the name of your computer (output of the \texttt{hostname} command).
106 jfenwick 2602 Please note that if your hostname has non-alphanumeric characters in it (eg - ) you need to replace them with underscores.
107 jfenwick 3322 For example the options file for \texttt{bob-desktop} would be named \file{bob_desktop_options.py}.
108 jfenwick 2602
109 jfenwick 2568 From now on all paths will be relative to the top level of the source.
110 jfenwick 2780 As a starting point copy the contents one of the following files into your options file:
111 jfenwick 2289 \begin{itemize}
112 jfenwick 3322 \item \file{scons/TEMPLATE_linux.py} (\linux and \macosx desktop)
113     \item \file{scons/TEMPLATE_windows.py} (\winxp)
114 jfenwick 2289 \end{itemize}
115    
116 jfenwick 3335 This options file controls which features and libraries your build of escript will attempt to use.
117     For example to use OpenMP or MPI you will need to enable it here.
118    
119     If you want to try escript out without customising your build, then change directories to \file{escript.d} and
120     \begin{shellCode}
121     scons
122     \end{shellCode}
123     If this works you can skip to Section~\ref{sec:diff}.
124     If not, then you will need to make some modications to the the file.
125     Read on.
126    
127 caltinay 3265 The template files contain all available options with a comment explaining the
128     purpose of each.
129     Check through the file and ensure that the relevant paths and names are correct
130     for your system and that you enable optional components that you wish to use.
131     For example, to use netCDF, find the netcdf-related lines, uncomment them
132     (i.e. remove the \# at the beginning of the lines) and change them according
133     to your installation:
134 jfenwick 2780 \begin{shellCode}
135 caltinay 3265 netcdf = True
136     netcdf_prefix = '/opt/netcdf4'
137     netcdf_libs = ['netcdf_c++', 'netcdf']
138 jfenwick 2780 \end{shellCode}
139    
140 caltinay 3265 In this example, netCDF \emph{header} files must be located in
141 jfenwick 3322 \file{/opt/netcdf4/include}\footnote{or \ldots/include32 or \ldots/include64 or \ldots/inc}
142     and the \emph{libraries} in \file{/opt/netcdf4/lib}\footnote{or \ldots/lib32 or \ldots/lib64}.
143 caltinay 3265 If this scheme does not apply to your installation then you may also specify
144     the include-path and library-path directly like so:
145     \begin{shellCode}
146     netcdf_prefix = ['/usr/local/include/netcdf', '/usr/local/lib']
147     \end{shellCode}
148     The order is important: the first element in the list is the
149     \emph{include}-path, the second element is the \emph{library}-path and both
150     must be specified.
151    
152     If a line in the options file is commented out and you do not require the
153     feature, then it can be ignored.
154 jfenwick 2780
155 jfenwick 2289 To actually compile (if you have $n$ processors, then you can use \texttt{scons -j$n$} instead):
156    
157     \begin{shellCode}
158 jfenwick 2529 cd escript.d
159 jfenwick 2289 scons
160     \end{shellCode}
161    
162 caltinay 3265 As part of its output, scons will tell you the name of the options file it used
163     as well as a list of features and whether they are enabled for your build.
164     If you enabled an optional dependency and the library or include files could
165     not be found you will be notified and the build will stop.
166 jfenwick 2289
167 caltinay 3265 Note, that you can override all settings from the options-file on the scons
168     command line. For example, if you usually build an optimized version but would
169     like to build a debug version into a separate directory without changing your
170     default settings, you can use:
171 jfenwick 2289 \begin{shellCode}
172 caltinay 3265 scons debug=1 prefix=debugbuild
173 jfenwick 2289 \end{shellCode}
174 caltinay 3265 This will install the binaries and libraries built in debug mode into
175 jfenwick 3322 directories underneath \file{./debugbuild}.
176 jfenwick 2289
177 caltinay 3265 To run the unit test suite that comes with the source code issue
178 jfenwick 2289 \begin{shellCode}
179     scons all_tests
180     \end{shellCode}
181 caltinay 2536 Grab a coffee or two while the tests compile and run.
182 caltinay 3265 An alternative method is available for running tests on \openmp and \mpi builds.
183 jfenwick 2289
184     \subsection{Compilation with \openmp}
185 caltinay 3265 \openmp is generally enabled by setting compiler and linker switches. For the
186     most common compilers these are automatically set by build system and all you
187     have to do is set the \texttt{openmp} option to True in your options file. If
188     this does not work or your compiler is different, then consult your compiler
189     documentation for the precise switches to use and modify the \texttt{omp_flags}
190     and \texttt{omp_ldflags} variables in your options file.
191 caltinay 2536 For example, for gcc compilers which support \openmp use:
192 jfenwick 2289 \begin{shellCode}
193 caltinay 3265 openmp = True
194     omp_flags = '-fopenmp'
195     omp_ldflags = '-fopenmp'
196 jfenwick 2289 \end{shellCode}
197 jfenwick 3335 (The two latter settings can also be left out as this is the default OpenMP on gcc.)
198 jfenwick 2289
199 caltinay 3265 You can test your \openmp-enabled build, e.g. using 4 threads by issuing
200 jfenwick 2289 \begin{shellCode}
201 gross 2363 export ESCRIPT_NUM_THREADS=4
202     scons all_tests
203     \end{shellCode}
204    
205 caltinay 2536 \subsection{Compilation with \mpi}
206 caltinay 3265 You need to have \mpi preinstalled on your system.
207     There are a number of implementations so we do not provide any specific advice
208     here.
209     Set the following variables in your options file to according to your
210     installation:
211 jfenwick 2289 \begin{itemize}
212 caltinay 3265 \item \texttt{mpi} \\
213     which \mpi implementation (flavour) is used. Valid values are
214 caltinay 2536 \begin{itemize}
215 caltinay 3265 \item[\texttt{none}] \mpi is disabled
216 caltinay 2536 \item[\texttt{MPT}] SGI MPI implementation \\
217     \url{http://techpubs.sgi.com/library/manuals/3000/007-3687-010/pdf/007-3687-010.pdf}
218 caltinay 3265 \item[\texttt{MPICH}] Argonne's MPICH implementation \\
219     \url{http://www.mcs.anl.gov/research/projects/mpi/mpich1/}
220 caltinay 2536 \item[\texttt{MPICH2}] Argonne's MPICH version 2 implementation \\
221     \url{http://www.mcs.anl.gov/research/projects/mpi/mpich2/}
222     \item[\texttt{OPENMPI}] Open MPI \\
223     \url{http://www.open-mpi.org/}
224 caltinay 3265 \item[\texttt{INTELMPI}] Intel MPI \\
225 caltinay 2536 \url{http://software.intel.com/en-us/intel-mpi-library/}
226     \end{itemize}
227 caltinay 3265 \item \texttt{mpi_prefix} \\
228     where to find \mpi headers and libraries (see netCDF example above)
229 caltinay 2536 \item \texttt{mpi_libs} \\
230 caltinay 3265 which libraries to link to.
231 jfenwick 2289 \end{itemize}
232    
233 caltinay 3265 To test your build using 6 processes enter:
234 jfenwick 2290 \begin{shellCode}
235 caltinay 3265 export ESCRIPT_NUM_PROCS=6
236     scons all_tests
237 jfenwick 2290 \end{shellCode}
238 jfenwick 3335 and on $2$ processes with $4$ threads each (provided \openmp is enabled)\footnote{Unless your system has $8$ cores expect this to be slow}:
239 gross 2363 \begin{shellCode}
240     export ESCRIPT_NUM_THREADS=4
241 jfenwick 3335 export ESCRIPT_NUM_PROCS=2
242 caltinay 3265 scons all_tests
243 gross 2363 \end{shellCode}
244     Alternatively, you can give a hostfile
245     \begin{shellCode}
246     export ESCRIPT_NUM_THREADS=4
247     export ESCRIPT_HOSTFILE=myhostfile
248 caltinay 3265 scons all_tests
249 gross 2363 \end{shellCode}
250 caltinay 3265 Note that depending on your \mpi flavour it may be required to start a daemon
251     before running the tests under \mpi.
252 gross 2363
253 jfenwick 3335 \subsection{Difficulties}\label{sec:diff}
254 jfenwick 2289
255 caltinay 3265 \subsubsection{Mismatch of runtime and build libraries}
256     Most external libraries used by \esfinley are linked dynamically.
257     This can lead to problems if after compiling \esfinley these libraries are
258     updated.
259     The same applies to the installed Python executable and libraries.
260     Whenever these dependencies change on your system you should recompile
261     \esfinley to avoid problems at runtime such as load errors or segmentation
262     faults.
263 jfenwick 2289
264 caltinay 2536 \subsubsection{\openmp builds segfault running examples}
265 jfenwick 3322 One known cause for this is linking the \file{gomp} library with escript built using gcc 4.3.3.
266     While you need the \texttt{-fopenmp} switch you should not need to link \file{gomp}.
267 caltinay 2536
268 jfenwick 3322

  ViewVC Help
Powered by ViewVC 1.1.26