/[escript]/trunk/doc/install/source.tex
ViewVC logotype

Contents of /trunk/doc/install/source.tex

Parent Directory Parent Directory | Revision Log Revision Log


Revision 5662 - (show annotations)
Thu Jun 18 04:40:34 2015 UTC (4 years ago) by caltinay
File MIME type: application/x-tex
File size: 18094 byte(s)
Fixes #259 - replaced all references to mayavi by Mayavi2 in the docs.
The arguments were already correct.

1 %!TEX root = install.tex
2 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
3 % Copyright (c) 2012-2015 by The University of Queensland
4 % http://www.uq.edu.au
5 %
6 % Primary Business: Queensland, Australia
7 % Licensed under the Open Software License version 3.0
8 % http://www.opensource.org/licenses/osl-3.0.php
9 %
10 % Development until 2012 by Earth Systems Science Computational Center (ESSCC)
11 % Development 2012-2013 by School of Earth Sciences
12 % Development from 2014 by Centre for Geoscience Computing (GeoComp)
13 %
14 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
15
16 % Notes about compilers
17
18 \chapter{Installing from Source}\label{chap:source}
19
20 This chapter assumes you are using a unix/posix like system (including MacOSX).
21
22 \section{Parallel Technologies}\label{sec:par}
23 It is likely that the computer you run \escript on, will have more than one processor core.
24 \escript can make use of multiple cores [in order to solve problems more quickly] if it is told to do so,
25 but this functionality must be enabled at compile time.
26 Section~\ref{sec:needpar} gives some rough guidelines to help you determine what you need.
27
28 There are two technologies which \escript can employ here.
29 \begin{itemize}
30 \item OpenMP -- the more efficient of the two [thread level parallelism].
31 \item MPI -- Uses multiple processes (less efficient), needs less help from the compiler.
32 \end{itemize}
33
34 Escript is primarily tested on recent versions of the GNU and Intel suites (``g++'' / ``icpc'').
35 However, it also passes our tests when compiled using ``clang++''.
36
37 Our current test compilers include:
38 \begin{itemize}
39 \item g++ 4.7.2, 4.9.1
40 \item clang++ (OSX 10.9 default, OSX 10.10 default)
41 \item intel icpc v14
42 \end{itemize}
43
44 Note that:
45 \begin{itemize}
46 \item OpenMP will not function correctly for g++ $\leq$ 4.2.1 (and is not currently supported by clang).
47 \item icpc v11 has a subtle bug involving OpenMP and c++ exception handling, so this combination should not be used.
48 \end{itemize}
49
50 \subsection{What parallel technology do I need?}\label{sec:needpar}
51 If you are using any version of Linux released in the past few years, then your system compiler will support
52 \openmp with no extra work (and give better performance); so you should use it.
53 You will not need MPI unless your computer is some form of cluster.
54
55 If you are using BSD or MacOSX and you are just experimenting with \escript, then performance is
56 probably not a major issue for you at the moment so you don't need to use either \openmp or MPI.
57 This also applies if you write and polish your scripts on your computer and then send them to a cluster to execute.
58 If in the future you find escript useful and your scripts take significant time to run, then you may want to recompile
59 \escript with more options.
60
61
62
63 Note that even if your version of \escript has support for \openmp or MPI, you will still need to tell the system to
64 use it when you run your scripts.
65 If you are using the \texttt{run-escript} launcher, then this is controlled by the \texttt{-t} and \texttt{-p} options.
66 If not, then consult the documentation for your MPI libraries (or the compiler documentation in the case of OpenMP
67 \footnote{It may be enough to set the \texttt{OMP\_NUM\_THREADS} environment variable.}).
68
69 If you are using MacOSX, then see the next section, if not, then skip to Section~\ref{sec:build}.
70
71 \section{MacOS}
72 This release of \escript has only been tested on OSX 10.9 and 10.10.
73 For this section we assume you are using either \texttt{homebrew} or \texttt{MacPorts} as a package
74 manager\footnote{Note that package managers will make changes to your computer based on programs configured by other people from
75 various places around the internet. It is important to satisfy yourself as to the security of those systems.}.
76 You can of course install prerequisite software in other other ways.
77 For example, we have had \emph{some} success changing the default
78 compilers used by those systems. However this is more complicated and we do not provide a guide here.
79 Successful combinations of OSX and package managers are given in the table below.
80
81 \begin{center}
82 \begin{tabular}{|c|c|c|}\hline
83 & \texttt{homebrew} & \texttt{MacPorts} \\\hline
84 OSX 10.9 & Yes & No\\\hline
85 OSX 10.10& Yes & Yes\\\hline
86 \end{tabular}
87 \end{center}
88
89 \noindent Both of those systems require the XCode command line tools to be installed\footnote{As of OSX10.9, the
90 command \texttt{xcode-select --install} will allow you to download and install the commandline tools.}.
91
92 \section{Building}\label{sec:build}
93
94 To simplify things for people, we have prepared \texttt{_options.py} files for a number of
95 systems\footnote{These are correct a time of writing but later versions of those systems may require tweaks.
96 Also, these systems represent a cross section of possible platforms rather than meaning those systems get particular support.}.
97 The \texttt{_options.py} files are located in the \texttt{scons/templates} directory. We suggest that the file most relevant to your os
98 be copied from the templates directory to the scons directory and renamed to the form XXXX_options.py where XXXX
99 should be replaced with your computer's name.
100 If your particular system is not in the list below, or if you want a more customised
101 build,
102 see Section~\ref{sec:othersrc} for instructions.
103 \begin{itemize}
104 \item Debian - \ref{sec:debsrc}
105 \item Ubuntu - \ref{sec:ubsrc}
106 \item OpenSuse - \ref{sec:susesrc}
107 \item Centos - \ref{sec:centossrc}
108 \item Fedora - \ref{sec:fedorasrc}
109 \item MacOS (macports) - \ref{sec:macportsrc}
110 \item MacOS (homebrew) - \ref{sec:homebrewsrc}
111 \item FreeBSD - \ref{sec:freebsdsrc}
112 \end{itemize}
113
114 Once these are done proceed to Section~\ref{sec:cleanup} for cleanup steps.
115
116 All of these instructions assume that you have obtained the \escript source (and uncompressed it if necessary).
117 \subsection{Debian}\label{sec:debsrc}
118
119 \begin{shellCode}
120 sudo aptitude install python-dev python-numpy libboost-python-dev libnetcdf-dev
121 sudo aptitude install scons lsb-release libboost-random-dev
122 sudo aptitude install python-sympy python-matplotlib python-scipy
123 sudo aptitude install python-pyproj python-gdal
124 \end{shellCode}
125
126 \noindent If you are running \textit{Jessie}, (or if \textit{wheezy-backports} is in your \texttt{apt} sources) you can use:
127 \begin{shellCode}
128 sudo aptitude install gmsh
129 \end{shellCode}
130 to add extra meshing functionality.
131
132 \begin{optionalstep}
133 If for some reason, you wish to rebuild the documentation, you would also need the following:
134 \begin{shellCode}
135 sudo aptitude install python-sphinx doxygen python-docutils texlive
136 sudo aptitude install zip texlive-latex-extra latex-xcolor
137 \end{shellCode}
138 \end{optionalstep}
139
140 \noindent In the source directory execute the following (substitute wheezy for XXXX):
141 \begin{shellCode}
142 scons -j1 options_file=scons/templates/XXXX_options.py
143 \end{shellCode}
144
145 \noindent If you wish to test your build, you can use the following:
146 \begin{shellCode}
147 scons -j1 py_tests options_file=scons/templates/XXXX_options.py
148 \end{shellCode}
149
150 \subsection{Ubuntu}\label{sec:ubsrc}
151
152 If you have not installed \texttt{aptitude}, then substitute \texttt{apt-get} in the following.
153 \begin{shellCode}
154 sudo aptitude install python-dev python-numpy libboost-python-dev
155 sudo aptitude install libnetcdf-dev libboost-random-dev
156 sudo aptitude install scons lsb-release
157 sudo aptitude install python-sympy python-matplotlib python-scipy
158 sudo aptitude install python-pyproj python-gdal gmsh
159 \end{shellCode}
160
161
162 \begin{optionalstep}
163 If for some reason, you wish to rebuild the documentation, you would also need the following:
164 \begin{shellCode}
165 sudo aptitude install python-sphinx doxygen python-docutils texlive
166 sudo aptitude install zip texlive-latex-extra latex-xcolor
167 \end{shellCode}
168 \end{optionalstep}
169
170 \noindent In the source directory execute the following (substitute precise, quantal or raring as appropriate for XXXX):
171 \begin{shellCode}
172 scons -j1 options_file=scons/templates/XXXX_options.py
173 \end{shellCode}
174
175 \noindent If you wish to test your build, you can use the following:
176 \begin{shellCode}
177 scons -j1 py_tests options_file=scons/templates/XXXX_options.py
178 \end{shellCode}
179
180
181
182 \subsection{OpenSuse}\label{sec:susesrc}
183 These instructions were prepared using release $13.2$.
184
185 \noindent Install packages from the main distribution:
186 \begin{shellCode}
187 sudo zypper install libboost_python1_54_0 libboost_random1_54_0
188 sudo zypper python-devel python-numpy libnetcdf_c++-devel
189 sudo zypper install python-scipy python-sympy python-matplotlib
190 sudo zypper install gcc gcc-c++ scons boost-devel netcdf-devel
191 \end{shellCode}
192 These will allow you to use most features except some parts of the \downunder inversion library.
193 If you wish to use those, you will need some additional packages [python-pyproj, python-gdal].
194 This can be done now or after Escript installation.
195
196 \begin{shellCode}
197 sudo zypper addrepo \
198 http://ftp.suse.de/pub/opensuse/repositories/Application:/Geo/openSUSE_13.2/ osgf
199 sudo zypper install python-pyproj python-gdal
200 \end{shellCode}
201
202 Now to build escript itself.
203 In the escript source directory:
204 \begin{shellCode}
205 scons -j1 options_file=scons/templates/opensuse13.1_options.py
206 \end{shellCode}
207
208 \noindent If you wish to test your build, you can use the following:
209 \begin{shellCode}
210 scons -j1 py_tests options_file=scons/templates/opensuse13.1_options.py
211 \end{shellCode}
212
213 \noindent Now go to Section~\ref{sec:cleanup} for cleanup.
214
215 \subsection{Centos}\label{sec:centossrc}
216 These instructions were prepared using centos release $7.0$.
217 The core of escript works, however some functionality is not availible because the default packages for some dependencies in Centos are too old.
218
219 \noindent Add the \texttt{EPEL} repository.
220 \begin{shellCode}
221 yum install epel-release.noarch
222 \end{shellCode}
223
224 \noindent Install packages:
225 \begin{shellCode}
226 yum install netcdf-devel netcdf-cxx-devel gdal-python
227 yum install python-devel numpy scipy scons boost-devel
228 yum install python-matplotlib gcc gcc-c++
229 yum install boost-python
230 \end{shellCode}
231
232 The above packages will allow you to use most features except
233 the \downunder inversion library.
234 If you wish to use those it, you will need to install some additional packages.
235
236 \noindent For some coordinate transformations, \downunder can also make use of the python interface to a tool called \texttt{proj}.
237 There does not seem to be an obvious centos repository for this though.
238 If it turns out to be necessary for your particular application, the source can be downloaded.
239
240 \noindent Now to build escript itself.
241 In the escript source directory:
242 \begin{shellCode}
243 scons -j1 options_file=scons/templates/centos7_0_options.py
244 \end{shellCode}
245
246 \noindent Now go to Section~\ref{sec:cleanup} for cleanup.
247
248 \subsection{Fedora}\label{sec:fedorasrc}
249 These instructions were prepared using release $21.5$.
250
251 \noindent Install packages
252 \begin{shellCode}
253 yum install netcdf-cxx-devel gcc-c++ scipy
254 yum install sympy scons pyproj gdal python-matplotlib
255 yum install boost-devel
256 \end{shellCode}
257
258 \noindent Now to build escript itself.
259 In the escript source directory:
260 \begin{shellCode}
261 scons -j1 options_file=scons/templates/fedora21_5_options.py
262 \end{shellCode}
263
264 \noindent If you wish to test your build, you can use the following:
265 \begin{shellCode}
266 scons -j1 py_tests options_file=scons/templates/fedora21_5_options.py
267 \end{shellCode}
268
269 \noindent Now go to Section~\ref{sec:cleanup} for cleanup.
270
271 \subsection{MacOS 10.10 (macports)}\label{sec:macportsrc}
272
273 The following will install the capabilities needed for the \texttt{macports_10.10_options.py} file.
274
275 \begin{shellCode}
276 sudo port install scons
277 sudo port select --set python python27
278 sudo port install boost
279 sudo port install py27-numpy
280 sudo port install py27-sympy
281 sudo port select --set py-sympy py27-sympy
282 sudo port install py27-scipy
283 sudo port install py27-pyproj
284 sudo port install py27-gdal
285 sudo port install netcdf-cxx
286 sudo port instal silo
287 \end{shellCode}
288
289 \begin{shellCode}
290 scons -j1 options_file=scons/templates/macports_10.10options.py
291 \end{shellCode}
292
293
294 \subsection{MacOS 10.9, 10.10 (homebrew)}\label{sec:homebrewsrc}
295
296 The following will install the capabilities needed for the \texttt{homebrew_10.10_options.py} file.
297 OSX 10.9 can use the same file.
298
299 \begin{shellCode}
300 brew install scons
301 brew install boost-python
302 brew install homebrew/science/netcdf --with-cxx-compat
303 \end{shellCode}
304
305 There do not appear to be formulae for \texttt{sympy} or \texttt{pyproj} so if you wish to use those features, then
306 you will need to install them separately.
307
308
309 \begin{shellCode}
310 scons -j1 options_file=scons/templates/homebrew_10.10_options.py
311 \end{shellCode}
312
313
314 \subsection{FreeBSD}\label{sec:freebsdsrc}
315
316 At time of writing, \texttt{numpy} does not install correctly on FreeBSD.
317 Since \texttt{numpy} is a critical dependency for \escript, we have been unable to test on FreeBSD.
318
319 \begin{comment}
320 \subsubsection{Release 10.0}
321
322 Install the following packages:
323 \begin{itemize}
324 \item python
325 \item scons
326 \item boost-python-libs
327 \item bash
328 \item netcdf
329 \item silo
330 \item py27-scipy
331 \item py27-gdal
332 \item py27-matplotlib
333 \item py27-pyproj
334 \item py27-sympy
335 \end{itemize}
336
337 \noindent Next choose (or create) your options file.
338 For the setup as above the escript source comes with a prepared file in
339 \texttt{scons/templates/freebsd10.0_options.py}.
340 Finally to build escript issue the following in the escript source directory
341 (replace the options file as required):
342 \begin{shellCode}
343 scons -j1 options_file=scons/templates/freebsd10.0_options.py
344 \end{shellCode}
345
346 \emph{Note:} Some packages installed above are built with gcc 4.7. Somewhere
347 in the toolchain a system-installed gcc library is pulled in which is
348 incompatible with the one from version 4.7 and would prevent escript from
349 executing successfully. As explained in the FreeBSD
350 documentation\footnote{see \url{http://www.freebsd.org/doc/en/articles/custom-gcc/article.html}}
351 this can be fixed by adding a line to \texttt{/etc/libmap.conf}:
352 \begin{shellCode}
353 libgcc_s.so.1 gcc47/libgcc_s.so.1
354 \end{shellCode}
355
356 \end{comment}
357 \subsection{Other Systems / Custom Builds}\label{sec:othersrc}
358
359 \escript has support for a number of optional packages.
360 Some, like \texttt{netcdf} need to be enabled at compile time, while others, such as \texttt{sympy} and the projection packages
361 used in \downunder are checked at run time.
362 For the second type, you can install them at any time (ensuring that python can find them) and they should work.
363 For the first type, you need to modify the options file and recompile with scons.
364 The rest of this section deals with this.
365
366 To avoid having to specify the options file each time you run scons, copy an existing \texttt{_options.py} file from the
367 \texttt{scons/} or \texttt{scons/templates/} directories. Put the file in the \texttt{scons} directory and name
368 it \textit{yourmachinename}\texttt{_options.py}.\footnote{If the name
369 has - or other non-alpha characters, they must be replaced with underscores in the filename}.
370 For example: on a machine named toybox, the file would be \texttt{scons/toybox_options.py}.
371
372 Individual lines can be enabled/disabled, by removing or adding \# (the python comment character) to the beginning of the line.
373 For example, to enable OpenMP, change the line
374 \begin{verbatim}
375 #openmp = True
376 \end{verbatim}
377 to
378 \begin{verbatim}
379 openmp = True
380 \end{verbatim}
381
382 If you are using libraries which are not installed in the standard places (or have different names) you will need to
383 change the relevant lines.
384 A common need for this would be using a more recent version of the boost::python library.
385 You can also change the compiler or the options passed to it by modifying the relevant lines.
386
387 \subsubsection*{MPI}
388 If you wish to enable or disable MPI, or if you wish to use a different implementation of MPI, you can use the \texttt{mpi}
389 configuration variable.
390 You will also need to ensure that the \texttt{mpi_prefix} and \texttt{mpi_libs} variables are uncommented and set correctly.
391 To disable MPI use, \verb|mpi = 'none'|.
392
393 \subsubsection{Python3}
394 \escript works with \texttt{python3} but until recently, many distributions have not distributed python3 versions of their packages.
395 You can try it out though by modifying or adding the following variables in your options file:
396
397 \begin{verbatim}
398 pythoncmd='python3'
399 \end{verbatim}
400
401 \begin{verbatim}
402 usepython3=True
403 \end{verbatim}
404
405 \begin{verbatim}
406 pythonlibname='whateveryourpython3libraryiscalled'
407 \end{verbatim}
408
409
410
411
412 \subsubsection{Testing}
413 As indicated earlier, you can test your build using \texttt{scons py_tests}.
414 Note however, that some features like \texttt{netCDF} are optional for using \escript, the tests will report a failure if
415 they are missing.
416
417 \section{Cleaning up}
418 \label{sec:cleanup}
419
420 Once the build (and optional testing) is complete, you can remove everything except:
421 \begin{itemize}
422 \item bin
423 \item esys
424 \item lib
425 \item doc
426 \item CREDITS.TXT
427 \item README_LICENSE
428 \end{itemize}
429 The last two aren't strictly required for operation.
430 The \texttt{doc} directory is not required either but does contain examples of escript scripts.
431
432 You can run escript using \texttt{\textit{path_to_escript_files}/bin/run-escript}.
433 Where \texttt{\textit{path_to_escript_files}} is replaced with the real path.
434
435 \begin{optionalstep}
436 You can add the escript \texttt{bin} directory to your \texttt{PATH} variable.
437 The launcher will then take care of the rest of the environment.
438 \end{optionalstep}
439
440 \section{Optional Extras}
441
442 Some other packages which might be useful include:
443 \begin{itemize}
444 \item support for silo format (install the relevant libraries and enable them in the options file).
445 \item Visit --- visualisation package. Can be used independently but our \texttt{weipa} library can make a Visit
446 plug-in to allow direct visualisation of escript files.
447 \item gmsh --- meshing software used by our \texttt{pycad} library.
448 \item Mayavi2 --- another visualisation tool.
449 \end{itemize}
450
451
452 %Need a better title but this is stuff like visit and silo (for non-debian distros)
453 %Perhaps - optional extras
454
455
456
457
458

  ViewVC Help
Powered by ViewVC 1.1.26