ViewVC logotype

Contents of /trunk/doc/install/source.tex

Parent Directory Parent Directory | Revision Log Revision Log

Revision 4439 - (show annotations)
Wed Jun 5 02:09:35 2013 UTC (6 years, 1 month ago) by jfenwick
File MIME type: application/x-tex
File size: 15071 byte(s)
some work on install doco
1 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
2 % Copyright (c) 2012-2013 by University of Queensland
3 % http://www.uq.edu.au
4 %
5 % Primary Business: Queensland, Australia
6 % Licensed under the Open Software License version 3.0
7 % http://www.opensource.org/licenses/osl-3.0.php
8 %
9 % Development until 2012 by Earth Systems Science Computational Center (ESSCC)
10 % Development since 2012 by School of Earth Sciences
11 %
12 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
14 % Notes about compilers
16 \chapter{Installing from Source}\label{chap:source}
18 \section{Intro}
20 Some initial questions:
21 \begin{enumerate}
22 \item Are you using MacOS?
23 \item Which parallel technologies do you wish to use?
24 \item Which packaging system are you using?
25 \end{enumerate}
29 \subsection{MacOS and compilers}
30 \emph{This information is as accurate as far as we can tell at time of writing but things may change.}
32 In order to install \escript from source, you need a compiler.
33 This is a bit harder on MacOS than on other systems which either provide a compiler as part of the base install (BSD) or allow
34 people to download one as part of the install process (Linux).
35 In earlier releases of its operating system (``Snow Leopard'' and earlier), Apple included an optional ``developer tools'' package (\texttt{XCode}) on the install media.
36 For more recent releases, \texttt{XCode} is available for ``free'' from Apple's application store.
37 Why ``free''?
38 The only way to access the application store is with an AppleID which requires either:
39 \begin{itemize}
40 \item purchasing an iTunes gift card.
41 \item giving Apple access to your credit card.
42 \item signing up as an Apple developer\footnote{If you do this you can download a ``command line tools'' package
43 which installs the relevant compilers without needing to install all of \texttt{XCode}.} and giving up personal information.
44 \end{itemize}
46 If you install \texttt{XCode}, you will need to download the ``command line tools'' optional package [see \texttt{XCode} documentation for details].
48 There are a number of projects on the net which aim to deliver compilers for MacOS.
49 For example:
50 \begin{itemize}
51 \item \url{http://hpc.sourceforge.net}
52 \item \url{http://kennethreitz.org/experiments/xcode-gcc-and-homebrew}
53 \end{itemize}
54 Use at your own risk.
58 \subsection{Parallel Technologies}
59 It is extremely likely that the computer you run \escript on, will have more than one processor core.
60 \escript can make use of multiple cores [in order to solve problems more quickly] if it is told to do so,
61 but this functionality must be enabled at compile time.
63 There are two technologies which \escript can employ here.
64 \begin{itemize}
65 \item OpenMP -- more efficient of the two [thread level parallelism].
66 \item MPI -- Uses multiple processes (less efficient), needs less help from the compiler.
67 \end{itemize}
69 Escript is primarily tested on recent versions of the GNU or Intel suites (``gcc, g++'' / ``icc, icpc'').
70 However, it also passes our tests when compiled using ``clang, clang++''.
71 The table below shows what methods are available with which compilers.
73 \begin{center}
74 \begin{tabular}{|l|c|c|c|}\hline
75 & Serial & OpenMP & MPI \\\hline
76 $\leq$ gcc-4.2.1 & \checkmark & \raisebox{-0.1cm}{\footnotemark}& \checkmark \\\hline
77 gcc (recent $\geq 4.3.2$) & \checkmark& \checkmark& \checkmark \\\hline
78 icc(10) & \checkmark& \checkmark& \checkmark \\\hline
79 icc(11) & \checkmark& \raisebox{-0.1cm}{\footnotemark} &\checkmark \\\hline
80 icc(12) & \checkmark& \checkmark&\checkmark \\\hline
81 clang & \checkmark& & \checkmark\\\hline
82 \end{tabular}
83 \end{center}
84 \addtocounter{footnote}{-1}
85 \footnotetext{The \openmp support in gcc-4.2.1 is buggy/non-functional.}
86 \addtocounter{footnote}{1}
87 \footnotetext{There is a subtle bug in icc-11 when \openmp and c++ exception handling
88 are combined.}
90 \noindent Where both \openmp and \mpi are marked, \escript can be compiled with either or both.
91 A \checkmark mark means that combination passes our tests.
95 \subsection{Packaging System}
96 The packaging system (also known as the package manager) is the tool you use to search for and install new open source software.
97 For Linux, there will be one set up by default: the apt tools on Debian and Ubuntu, yast on Suse, yum on the RedHat family.
98 On BSD systems this will be a combination of \texttt{pkg_add} and the \texttt{ports} tree.
100 For MacOS, this is a bit more tricky.
101 There are a number of possible systems including \texttt{macports} and \texttt{homebrew}\footnote{There is also \texttt{fink}, but we have not experimented with that.}, but they do not come pre-installed so you will need to install one.
103 Packaging systems will make changes to your computer based on programs configured by other people from
104 various places around the internet.
105 It is important to satisfy yourself as to the security of those systems.
107 If you are using Linux or have decided that you don't want to use OpenMP skip to Section~\ref{}
109 \subsubsection{Changing Compilers on MacOS and BSD}
110 \emph{Not for the faint-hearted.}
112 In order to use OpenMP, your compiler must support it.
113 The compilers provided in Apple's package and BSD do not support OpenMP.
114 \texttt{Clang} has no support for it at all, while \texttt{GCC4.2.1} has some, but it does not work very well\footnote{Rejects valid code, crashes etc}.
115 You can use the packaging system to install a more up to date version of \texttt{gcc}.
116 However, you will need to install a number of other dependencies in order for escript to operate.
117 If these dependencies end up fighting over which versions of libraries to use, there will be problems
118 \footnote{For example: Trying to use two versions of \texttt{Python} in the same program or two versions of the \texttt{c++} standard library.}
119 To avoid this, if you are intend to change the compiler or python versions, you should install the new compiler first,
120 then the new version of python.
121 After this, you can install the other dependencies.
122 \emph{Please note that none of the Mac based packages nor BSD recommend changing the default $C$ compiler so
123 do so at your own risk. If you want OpenMP though, there does not seem to be a choice.}
125 \subsubsection*{Changing compiler on FreeBSD(9.1)}
126 The following sequence passes our unit tests under OpenMP.
127 \begin{itemize}
128 \item Install \texttt{gcc46} from ports.
129 \item Modify \texttt{/etc/make.conf} to set the default compiler to be \texttt{gcc46}
130 \item Install the remaining dependencies.
131 \item Configure \escript to build with \openmp.
132 \end{itemize}
134 We chose version $4.6$ rather than a later one because one of the optional dependencies (scipy) will try to install it anyway.
136 \subsection*{Changing compiler on MacPorts}
137 The following sequence passes our unit tests under OpenMP.
138 \begin{itemize}
139 \item \texttt{port install gcc47}
140 \item Set the default compiler for the command line with: \texttt{port select --set gcc mp-gcc47}
141 \item Set the default compiler for macports by adding the following line to \linebreak[2] \file{/opt/local/etc/macports/macports.conf}:
142 \begin{shellCode}
143 #Added by user to coerce macports into using a newer compiler
144 default_compiler macports-gcc-4.7
145 \end{shellCode}
146 \item Now install the remainder of the dependencies using \texttt{port -s install X}.
147 This will build them from source rather than downloading precompiled versions.
148 (This will unfortunately mean larger downloads.).
149 In some cases, some dependencies will not build properly from source.
150 In those cases(where XYZ is the dependency that fails to build), \texttt{port clean XYZ} then \texttt{port install XYZ}.
151 Then \emph{try to install the rest from source}.
153 Installing from source via macports is necessary to convince macports to link against the libraries provided by
154 your new compiler rather than the default one.
155 \end{itemize}
160 \subsection*{Changing compiler on Homebrew}
161 There is no configuration file that needs to be changed in order to use a different compiler.
162 You do need to do things in the correct order and make sure that you have made any required changes to
163 your environment variables before moving on to the next step.
164 In particular, the compiler must be installed before anything else and then python.
167 % make it clear that escript can be customised to use whatever you have
169 %Should include optional customisation
171 %Talk about options_files and the ability to specify them
173 %talk about -j1 and replacing it with more ops
174 %Talk about installation prefix
176 %also note that this doesn't build the doco but we do have downloads for that or you can install extra packages
178 \section{Building}
180 To simplify things for people, we have prepared \texttt{_options.py} files for a number of
181 systems\footnote{These are correct a time of writing but later versions of those systems may require tweaks.
182 Also, these systems represent a cross section of possible platforms rather than meaning those systems get particular support.}.
183 If your particular system is not in the above list, or if you want a more customised build,
184 see Section~\ref{sec:othersrc} for instructions.
185 \begin{itemize}
186 \item Debian - \ref{sec:debsrc}
187 \item Ubuntu - \ref{sec:ubsrc}
188 \item OpenSuse - \ref{sec:susesrc}
189 \item Centos - \ref{sec:centossrc}
190 \item Fedora - \ref{sec:fedorasrc}
191 \item MacOS (macports) - \ref{sec:macportsrc}
192 \item MacOS (homebrew) - \ref{sec:homebrewsrc}
193 \item FreeBSD - \ref{sec:freebsdsrc}
194 \end{itemize}
196 Once these are done proceed to Section~\ref{sec:cleanup} for cleanup steps.
198 All of these instructions assume that you have obtained the source (uncompressed it if necessary).
199 \subsection{Debian}\label{sec:debsrc}
201 \begin{shellCode}
202 sudo aptitude install python-dev python-numpy libboost-python-dev libnetcdf-dev
203 sudo aptitude scons lsb-release
204 sudo aptitude install python-sympy python-matplotlib python-scipy
205 sudo aptitude install python-pyproj python-gdal
206 \end{shellCode}
209 \begin{optionalstep}
210 If for some reason, you wish to rebuild the documentation, you would also need the following:
211 \begin{shellCode}
212 sudo aptitude install python-sphinx doxygen python-docutils texlive
213 sudo aptitude install ghostscript texlive-latex-extra latex-xcolor
214 \end{shellCode}
215 \end{optionalstep}
217 In the source directory execute the following (substitute squeeze or wheezy as appropriate for XXXX):
218 \begin{shellCode}
219 scons -j1 options_file=scons/os/XXXX_options.py
220 \end{shellCode}
222 If you wish to test your build, you can use the following:
223 \begin{shellCode}
224 scons -j1 py_tests options_file=scons/os/XXXX_options.py
225 \end{shellCode}
227 \subsection{Ubuntu}\label{sec:ubsrc}
229 \subsection{OpenSuse}\label{sec:susesrc}
230 These instructions were prepared using release $12.3$.
232 Install packages from the main distribution:
233 \begin{shellCode}
234 sudo yast2 --install libboost_python1_49_0 python-devel python-numpy
235 sudo yast2 --install python-scipy python-sympy python-matplotlib libnetcdf_c++-devel
236 sudo yast2 --install gcc-c++ scons boost-devel netcdf-devel
237 \end{shellCode}
238 These will allow you to use most features except some parts of the \downunder inversion library.
239 If you wish to use those, you will need some additional packages [python-pyproj, python-gdal].
240 This can be done after Escript installation.
242 \begin{optionalstep}
243 Add \url{http://ftp.suse.de/pub/opensuse/repositories/Application:/Geo/openSUSE_12.3/}
244 to your repositories in \texttt{YaST}.
245 \begin{shellCode}
246 sudo yast2 --install python-pyproj, python-gdal
247 \end{shellCode}
248 \end{optionalstep}
250 Now to build escript itself.
251 In the escript source directory:
252 \begin{shellCode}
253 scons -j1 options_file=scons/os/opensuse12.3_options.py
254 \end{shellCode}
256 If you wish to test your build, you can use the following:
257 \begin{shellCode}
258 scons -j1 py_tests options_file=scons/os/opensuse12.3_options.py
259 \end{shellCode}
261 Now go to Section~\ref{sec:cleanup} for cleanup.
263 \subsection{Centos}\label{sec:centossrc}
264 These instructions were prepared using release $6.4$.
266 Install packages from the main distribution:
267 \begin{shellCode}
268 yum install python-devel numpy scipy scons boost-devel
269 yum install python-matplotlib gcc-g++
270 yum install boost-python
271 \end{shellCode}
273 The above packages will allow you to use most features except saving and loading files in \texttt{netCDF}
274 format and the \downunder inversion library.
275 If you wish to use those features, you will need to install some additional packages.
276 NetCDF needs to be installed when you compile if you wish to use it.
277 \begin{optionalstep}
278 Add the \texttt{EPEL} repository.
279 \begin{shellCode}
280 rpm -U http://download.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
281 \end{shellCode}
283 \begin{shellCode}
284 yum install netcdf-devel sympy gdal-python
285 \end{shellCode}
287 For some coordinate transformations, \downunder can also make use of the python interface to a tool called \texttt{proj}.
288 There does not seem to be an obvious centos repository for this though.
289 If it turns out to be necessary for your particular application, the source can be downloaded.
290 \end{optionalstep}
292 Now to build escript itself.
293 In the escript source directory:
294 \begin{shellCode}
295 scons -j1 options_file=scons/os/centos6.4_options.py
296 \end{shellCode}
298 If you wish to test your build, you can use the following:
299 \begin{shellCode}
300 scons -j1 py_tests options_file=scons/os/centos6.4_options.py
301 \end{shellCode}
303 Now go to Section~\ref{sec:cleanup} for cleanup.
305 \subsection{Fedora}\label{sec:fedorasrc}
306 These instructions were prepared using release $18$.
308 Install packages
309 \begin{shellCode}
310 yum install netcdf-cxx-devel gcc-c++ scipy
311 yum install sympy scons pyproj gdal python-matplotlib
312 yum install boost-devel
313 \end{shellCode}
315 Now to build escript itself.
316 In the escript source directory:
317 \begin{shellCode}
318 scons -j1 options_file=scons/os/fedora18_options.py
319 \end{shellCode}
321 If you wish to test your build, you can use the following:
322 \begin{shellCode}
323 scons -j1 py_tests options_file=scons/os/fedora18_options.py
324 \end{shellCode}
326 Now go to Section~\ref{sec:cleanup} for cleanup.
328 \subsection{MacOS (macports)}\label{sec:macportsrc}
330 \subsection{MacOS (homebrew)}\label{sec:homebrewsrc}
332 \subsection{FreeBSD}\label{sec:freebsdsrc}
336 \subsection{Other Systems / Custom Builds}\label{sec:othersrc}
338 \section{Cleaning up}
339 \label{sec:cleanup}
341 Once the build (and optional testing) is complete, you can remove everything except:
342 \begin{itemize}
343 \item bin
344 \item esys
345 \item lib
346 \item doc
347 \item CREDITS.TXT
349 \end{itemize}
350 The last two aren't strictly required for operation.
351 The \texttt{doc} directory is not required either but does contain examples of escript scripts.
353 You can run escript using \texttt{\textit{path_to_escript_files}/bin/run-escript}.
354 Where \texttt{\textit{path_to_escript_files}} is replaced with the real path.
356 \begin{optionalstep}
357 You can add the escript \texttt{bin} directory to your \texttt{PATH} variable.
358 The launcher will then take care of the rest of the environment.
359 \end{optionalstep}
361 \section{Optional Extras}
362 %Need a better title but this is stuff like visit and silo (for non-debian distros)
363 %Perhaps - optional extras

  ViewVC Help
Powered by ViewVC 1.1.26