/[escript]/release/5.1/doc/install/source.tex
ViewVC logotype

Contents of /release/5.1/doc/install/source.tex

Parent Directory Parent Directory | Revision Log Revision Log


Revision 6598 - (show annotations)
Mon Jun 5 07:05:25 2017 UTC (22 months, 2 weeks ago) by jfenwick
File MIME type: application/x-tex
File size: 18561 byte(s)
Minor typo and Mac versioning
1 %!TEX root = install.tex
2 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
3 % Copyright (c) 2012-2017 by The University of Queensland
4 % http://www.uq.edu.au
5 %
6 % Primary Business: Queensland, Australia
7 % Licensed under the Apache License, version 2.0
8 % http://www.apache.org/licenses/LICENSE-2.0
9 %
10 % Development until 2012 by Earth Systems Science Computational Center (ESSCC)
11 % Development 2012-2013 by School of Earth Sciences
12 % Development from 2014 by Centre for Geoscience Computing (GeoComp)
13 %
14 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
15
16 % Notes about compilers
17
18 \chapter{Installing from Source}\label{chap:source}
19
20 This chapter assumes you are using a unix/posix like system (including MacOSX).
21
22 \section{Parallel Technologies}\label{sec:par}
23 It is likely that the computer you run \escript on, will have more than one processor core.
24 \escript can make use of multiple cores [in order to solve problems more quickly] if it is told to do so,
25 but this functionality must be enabled at compile time.
26 Section~\ref{sec:needpar} gives some rough guidelines to help you determine what you need.
27
28 There are two technologies which \escript can employ here.
29 \begin{itemize}
30 \item OpenMP -- the more efficient of the two [thread level parallelism].
31 \item MPI -- Uses multiple processes (less efficient), needs less help from the compiler.
32 \end{itemize}
33
34 Escript is primarily tested on recent versions of the GNU and Intel suites (``g++'' / ``icpc'').
35 However, it also passes our tests when compiled using ``clang++''.
36 Escript now requires compiler support for some features of the C++11 standard.
37 See Appendix~\ref{app:cxxfeatures} for a list.
38
39
40 Our current test compilers include:
41 \begin{itemize}
42 \item g++ 6
43 \item clang++ 7
44 \item intel icpc v17
45 \end{itemize}
46
47 Note that:
48 \begin{itemize}
49 \item OpenMP will not function correctly for g++ $\leq$ 4.2.1 (and is not currently supported by clang).
50 \item icpc v11 has a subtle bug involving OpenMP and C++ exception handling, so this combination should not be used.
51 \end{itemize}
52
53 \subsection{What parallel technology do I need?}\label{sec:needpar}
54 If you are using any version of Linux released in the past few years, then your system compiler will support
55 \openmp with no extra work (and give better performance); so you should use it.
56 You will not need MPI unless your computer is some form of cluster.
57
58 If you are using BSD or MacOSX and you are just experimenting with \escript, then performance is
59 probably not a major issue for you at the moment so you don't need to use either \openmp or MPI.
60 This also applies if you write and polish your scripts on your computer and then send them to a cluster to execute.
61 If in the future you find escript useful and your scripts take significant time to run, then you may want to recompile
62 \escript with more options.
63
64
65
66 Note that even if your version of \escript has support for \openmp or MPI, you will still need to tell the system to
67 use it when you run your scripts.
68 If you are using the \texttt{run-escript} launcher, then this is controlled by
69 the \texttt{-t}, \texttt{-p}, and \texttt{-n} options.
70 If not, then consult the documentation for your MPI libraries (or the compiler documentation in the case of OpenMP
71 \footnote{It may be enough to set the \texttt{OMP\_NUM\_THREADS} environment variable.}).
72
73 If you are using MacOSX, then see the next section, if not, then skip to Section~\ref{sec:build}.
74
75 \section{MacOS}
76 This release of \escript has only been tested on OSX 10.11 and 10.12.
77 For this section we assume you are using either \texttt{homebrew} or \texttt{MacPorts} as a package
78 manager\footnote{Note that package managers will make changes to your computer based on programs configured by other people from
79 various places around the internet. It is important to satisfy yourself as to the security of those systems.}.
80 You can of course install prerequisite software in other ways.
81 For example, we have had \emph{some} success changing the default
82 compilers used by those systems. However this is more complicated and we do not provide a guide here.
83
84 This release has been tested with MacPorts on OSX 10.11 and 10.12 and Homebrew on OSX10.12.
85
86 \noindent Both of those systems require the XCode command line tools to be installed\footnote{As of OSX10.9, the
87 command \texttt{xcode-select --install} will allow you to download and install the commandline tools.}.
88
89 \section{Building}\label{sec:build}
90
91 To simplify things for people, we have prepared \texttt{_options.py} files for a number of
92 systems\footnote{These are correct a time of writing but later versions of those systems may require tweaks.
93 Also, these systems represent a cross section of possible platforms rather than meaning those systems get particular support.}.
94 The \texttt{_options.py} files are located in the \texttt{scons/templates} directory. We suggest that the file most relevant to your OS
95 be copied from the templates directory to the scons directory and renamed to the form XXXX_options.py where XXXX
96 should be replaced with your computer's (host-)name.
97 If your particular system is not in the list below, or if you want a more customised
98 build,
99 see Section~\ref{sec:othersrc} for instructions.
100 \begin{itemize}
101 \item Debian - \ref{sec:debsrc}
102 \item Ubuntu - \ref{sec:ubsrc}
103 \item OpenSuse - \ref{sec:susesrc}
104 \item Centos - \ref{sec:centossrc}
105 \item Fedora - \ref{sec:fedorasrc}
106 \item MacOS (macports) - \ref{sec:macportsrc}
107 \item MacOS (homebrew) - \ref{sec:homebrewsrc}
108 \item FreeBSD - \ref{sec:freebsdsrc}
109 \end{itemize}
110
111 Once these are done proceed to Section~\ref{sec:cleanup} for cleanup steps.
112
113 All of these instructions assume that you have obtained the \escript source (and uncompressed it if necessary).
114 \subsection{Debian}\label{sec:debsrc}
115
116 \begin{shellCode}
117 sudo aptitude install python-dev python-numpy libboost-python-dev libnetcdf-dev
118 sudo aptitude install libnetcdf-cxx-legacy-dev
119 sudo aptitude install scons lsb-release libboost-random-dev
120 sudo aptitude install python-sympy python-matplotlib python-scipy
121 sudo aptitude install python-pyproj python-gdal
122 \end{shellCode}
123
124 \noindent If you are running \textit{Wheezy}, you can use:
125 \begin{shellCode}
126 sudo aptitude install gmsh
127 \end{shellCode}
128 to add extra meshing functionality.
129
130 \begin{optionalstep}
131 If for some reason, you wish to rebuild the documentation, you would also need the following:
132 \begin{shellCode}
133 sudo aptitude install python-sphinx doxygen python-docutils texlive
134 sudo aptitude install zip texlive-latex-extra latex-xcolor
135 \end{shellCode}
136 \end{optionalstep}
137
138 \noindent In the source directory execute the following (substitute jessie for XXXX):
139 \begin{shellCode}
140 scons -j1 options_file=scons/templates/XXXX_options.py
141 \end{shellCode}
142
143 \noindent If you wish to test your build, you can use the following:
144 \begin{shellCode}
145 scons -j1 py_tests options_file=scons/templates/XXXX_options.py
146 \end{shellCode}
147
148 \subsection{Ubuntu}\label{sec:ubsrc}
149
150 If you have not installed \texttt{aptitude}, then substitute \texttt{apt-get} in the following.
151 \begin{shellCode}
152 sudo aptitude install python-dev python-numpy libboost-python-dev
153 sudo aptitude install libnetcdf-cxx-legacy-dev
154 sudo aptitude install libnetcdf-dev libboost-random-dev
155 sudo aptitude install scons lsb-release
156 sudo aptitude install python-sympy python-matplotlib python-scipy
157 sudo aptitude install python-pyproj python-gdal gmsh
158 \end{shellCode}
159
160
161 \begin{optionalstep}
162 If for some reason, you wish to rebuild the documentation, you would also need the following:
163 \begin{shellCode}
164 sudo aptitude install python-sphinx doxygen python-docutils texlive
165 sudo aptitude install zip texlive-latex-extra latex-xcolor
166 \end{shellCode}
167 \end{optionalstep}
168
169 \noindent In the source directory execute the following (substitute precise, quantal or raring as appropriate for XXXX):
170 \begin{shellCode}
171 scons -j1 options_file=scons/templates/XXXX_options.py
172 \end{shellCode}
173
174 \noindent If you wish to test your build, you can use the following:
175 \begin{shellCode}
176 scons -j1 py_tests options_file=scons/templates/XXXX_options.py
177 \end{shellCode}
178
179
180
181 \subsection{OpenSuse}\label{sec:susesrc}
182 These instructions were prepared using release $13.2$.
183
184 \noindent Install packages from the main distribution:
185 \begin{shellCode}
186 sudo zypper in libboost_python1_54_0 libboost_random1_54_0
187 sudo zypper in python-devel python-numpy libnetcdf_c++-devel
188 sudo zypper in python-scipy python-sympy python-matplotlib
189 sudo zypper in gcc gcc-c++ scons boost-devel netcdf-devel
190 \end{shellCode}
191 These will allow you to use most features except some parts of the \downunder inversion library.
192 If you wish to use those, you will need some additional packages [python-pyproj, python-gdal].
193 This can be done now or after Escript installation.
194
195 \begin{shellCode}
196 sudo zypper addrepo \
197 http://ftp.suse.de/pub/opensuse/repositories/Application:/Geo/openSUSE_13.2/ osgf
198 sudo zypper install python-pyproj python-gdal
199 \end{shellCode}
200
201 Now to build escript itself.
202 In the escript source directory:
203 \begin{shellCode}
204 scons -j1 options_file=scons/templates/opensuse13.1_options.py
205 \end{shellCode}
206
207 \noindent If you wish to test your build, you can use the following:
208 \begin{shellCode}
209 scons -j1 py_tests options_file=scons/templates/opensuse13.1_options.py
210 \end{shellCode}
211
212 \noindent Now go to Section~\ref{sec:cleanup} for cleanup.
213
214 \subsection{CentOS}\label{sec:centossrc}
215 These instructions were prepared using CentOS release $7.0$.
216 The core of escript works, however some functionality is not available because the default packages for some dependencies in CentOS are too old.
217
218 \noindent Add the \texttt{EPEL} repository.
219 \begin{shellCode}
220 yum install epel-release.noarch
221 \end{shellCode}
222
223 \noindent Install packages:
224 \begin{shellCode}
225 yum install netcdf-devel netcdf-cxx-devel gdal-python
226 yum install python-devel numpy scipy scons boost-devel
227 yum install python-matplotlib gcc gcc-c++
228 yum install boost-python
229 \end{shellCode}
230
231 The above packages will allow you to use most features except
232 the \downunder inversion library.
233 If you wish to use those it, you will need to install some additional packages.
234
235 \noindent For some coordinate transformations, \downunder can also make use of the python interface to a tool called \texttt{proj}.
236 There does not seem to be an obvious CentOS repository for this though.
237 If it turns out to be necessary for your particular application, the source can be downloaded.
238
239 \noindent Now to build escript itself.
240 In the escript source directory:
241 \begin{shellCode}
242 scons -j1 options_file=scons/templates/centos7_0_options.py
243 \end{shellCode}
244
245 \noindent Now go to Section~\ref{sec:cleanup} for cleanup.
246
247 \subsection{Fedora}\label{sec:fedorasrc}
248 These instructions were prepared using release $21.5$.
249
250 \noindent Install packages
251 \begin{shellCode}
252 yum install netcdf-cxx-devel gcc-c++ scipy
253 yum install sympy scons pyproj gdal python-matplotlib
254 yum install boost-devel
255 \end{shellCode}
256
257 \noindent Now to build escript itself.
258 In the escript source directory:
259 \begin{shellCode}
260 scons -j1 options_file=scons/templates/fedora21_5_options.py
261 \end{shellCode}
262
263 \noindent If you wish to test your build, you can use the following:
264 \begin{shellCode}
265 scons -j1 py_tests options_file=scons/templates/fedora21_5_options.py
266 \end{shellCode}
267
268 \noindent Now go to Section~\ref{sec:cleanup} for cleanup.
269
270 \subsection{MacOS 10.10 and later (macports)}\label{sec:macportsrc}
271
272 The following will install the capabilities needed for the \texttt{macports_10.10_options.py} file (later versions can use the same options file).
273
274 \begin{shellCode}
275 sudo port install scons
276 sudo port select --set python python27
277 sudo port install boost
278 sudo port install py27-numpy
279 sudo port install py27-sympy
280 sudo port select --set py-sympy py27-sympy
281 sudo port install py27-scipy
282 sudo port install py27-pyproj
283 sudo port install py27-gdal
284 sudo port install netcdf-cxx
285 sudo port install silo
286 \end{shellCode}
287
288 \begin{shellCode}
289 scons -j1 options_file=scons/templates/macports_10.10options.py
290 \end{shellCode}
291
292
293 \subsection{MacOS 10.10 and later (homebrew)}\label{sec:homebrewsrc}
294
295 The following will install the capabilities needed for the \texttt{homebrew_10.10_options.py} file.
296 OSX 10.9 can use the same file.
297
298 \begin{shellCode}
299 brew install scons
300 brew install boost-python
301 brew install homebrew/science/netcdf --with-cxx-compat
302 \end{shellCode}
303
304 There do not appear to be formulae for \texttt{sympy} or \texttt{pyproj} so if you wish to use those features, then
305 you will need to install them separately.
306
307
308 \begin{shellCode}
309 scons -j1 options_file=scons/templates/homebrew_10.10_options.py
310 \end{shellCode}
311
312
313 \subsection{FreeBSD}\label{sec:freebsdsrc}
314
315 At time of writing, \texttt{numpy} does not install correctly on FreeBSD.
316 Since \texttt{numpy} is a critical dependency for \escript, we have been unable to test on FreeBSD.
317
318 \begin{comment}
319 \subsubsection{Release 10.0}
320
321 Install the following packages:
322 \begin{itemize}
323 \item python
324 \item scons
325 \item boost-python-libs
326 \item bash
327 \item netcdf
328 \item silo
329 \item py27-scipy
330 \item py27-gdal
331 \item py27-matplotlib
332 \item py27-pyproj
333 \item py27-sympy
334 \end{itemize}
335
336 \noindent Next choose (or create) your options file.
337 For the setup as above the escript source comes with a prepared file in
338 \texttt{scons/templates/freebsd10.0_options.py}.
339 Finally to build escript issue the following in the escript source directory
340 (replace the options file as required):
341 \begin{shellCode}
342 scons -j1 options_file=scons/templates/freebsd10.0_options.py
343 \end{shellCode}
344
345 \emph{Note:} Some packages installed above are built with gcc 4.7. Somewhere
346 in the toolchain a system-installed gcc library is pulled in which is
347 incompatible with the one from version 4.7 and would prevent escript from
348 executing successfully. As explained in the FreeBSD
349 documentation\footnote{see \url{http://www.freebsd.org/doc/en/articles/custom-gcc/article.html}}
350 this can be fixed by adding a line to \texttt{/etc/libmap.conf}:
351 \begin{shellCode}
352 libgcc_s.so.1 gcc47/libgcc_s.so.1
353 \end{shellCode}
354
355 \end{comment}
356 \subsection{Other Systems / Custom Builds}\label{sec:othersrc}
357
358 \escript has support for a number of optional packages.
359 Some, like \texttt{netcdf} need to be enabled at compile time, while others, such as \texttt{sympy} and the projection packages
360 used in \downunder are checked at run time.
361 For the second type, you can install them at any time (ensuring that python can find them) and they should work.
362 For the first type, you need to modify the options file and recompile with scons.
363 The rest of this section deals with this.
364
365 To avoid having to specify the options file each time you run scons, copy an existing \texttt{_options.py} file from the
366 \texttt{scons/} or \texttt{scons/templates/} directories. Put the file in the \texttt{scons} directory and name
367 it \textit{yourmachinename}\texttt{_options.py}.\footnote{If the name
368 has - or other non-alpha characters, they must be replaced with underscores in the filename}.
369 For example: on a machine named toybox, the file would be \texttt{scons/toybox_options.py}.
370
371 Individual lines can be enabled/disabled, by removing or adding \# (the python comment character) to the beginning of the line.
372 For example, to enable OpenMP, change the line
373 \begin{verbatim}
374 #openmp = True
375 \end{verbatim}
376 to
377 \begin{verbatim}
378 openmp = True
379 \end{verbatim}
380
381 If you are using libraries which are not installed in the standard places (or have different names) you will need to
382 change the relevant lines.
383 A common need for this would be using a more recent version of the boost::python library.
384 You can also change the compiler or the options passed to it by modifying the relevant lines.
385
386 \subsubsection*{MPI}
387 If you wish to enable or disable MPI, or if you wish to use a different implementation of MPI, you can use the \texttt{mpi}
388 configuration variable.
389 You will also need to ensure that the \texttt{mpi_prefix} and \texttt{mpi_libs} variables are uncommented and set correctly.
390 To disable MPI use, \verb|mpi = 'none'|.
391
392 \subsubsection{Python3}
393 \escript works with \texttt{python3} but until recently, many distributions have not distributed python3 versions of their packages.
394 You can try it out though by modifying or adding the following variables in your options file:
395
396 \begin{verbatim}
397 pythoncmd='python3'
398 \end{verbatim}
399
400 \begin{verbatim}
401 usepython3=True
402 \end{verbatim}
403
404 \begin{verbatim}
405 pythonlibname='whatever_your_python3_library_is_called'
406 \end{verbatim}
407
408
409
410
411 \subsubsection{Testing}
412 As indicated earlier, you can test your build using \texttt{scons py_tests}.
413 Note however, that some features like \texttt{netCDF} are optional for using \escript, the tests will report a failure if
414 they are missing.
415
416 \section{Cleaning up}
417 \label{sec:cleanup}
418
419 Once the build (and optional testing) is complete, you can remove everything except:
420 \begin{itemize}
421 \item bin
422 \item esys
423 \item lib
424 \item doc
425 \item CREDITS
426 \item LICENSE
427 \item README
428 \end{itemize}
429 The last three aren't strictly required for operation.
430 The \texttt{doc} directory is not required either but does contain examples of escript scripts.
431
432 You can run escript using \texttt{\textit{path_to_escript_files}/bin/run-escript}.
433 Where \texttt{\textit{path_to_escript_files}} is replaced with the real path.
434
435 \begin{optionalstep}
436 You can add the escript \texttt{bin} directory to your \texttt{PATH} variable.
437 The launcher will then take care of the rest of the environment.
438 \end{optionalstep}
439
440 \section{Optional Extras}
441
442 Some other packages which might be useful include:
443 \begin{itemize}
444 \item Lapack and UMFPACK --- direct solvers (install the relevant libraries and enable them in the options file).
445 \item support for the Silo file format (install the relevant libraries and enable them in the options file).
446 \item VisIt --- visualisation package. Can be used independently but our \texttt{weipa} library can make a Visit
447 plug-in to allow direct visualisation of escript simulations.
448 \item gmsh --- meshing software used by our \texttt{pycad} library.
449 \item Mayavi2 --- another visualisation tool.
450 \end{itemize}
451
452
453 \subsection{Trilinos}
454 \escript now has some support for Trilinos\footnote{\url{https://trilinos.org/}}
455 solvers and preconditioners.
456 The most significant limitation is that the current Trilinos release does not
457 support block matrices so \escript can only use Trilinos solvers for single
458 PDEs (i.e. no PDE systems).
459
460 If your distribution does not provide Trilinos packages you can build a working
461 version from source. (See Appendix~\ref{app:trilinos})
462

  ViewVC Help
Powered by ViewVC 1.1.26