# Diff of /trunk/doc/user/execute.tex

revision 2780 by jfenwick, Thu Nov 26 03:54:24 2009 UTC revision 2923 by jfenwick, Thu Feb 4 04:05:36 2010 UTC
# Line 2  Line 2
2  \label{EXECUTION}  \label{EXECUTION}
3
4  \section{Overview}  \section{Overview}
5  A typical way of starting your {\it escript} script \file{myscript.py} is with the \program{escript} command\footnote{The \program{escript} launcher is not supported under \WINDOWS yet.}:  A typical way of starting your {\it escript} script \file{myscript.py} is with the \program{run-escript} command\index{run-escript}\footnote{The \program{run-escript} launcher is not supported under \WINDOWS yet.}
6    This command was renamed from \program{escript} (used in previous releases) to avoid clashing with an unrelated program installed by default on
7    some systems.
8    Most 3.1~releases\footnote{ie. not \WINDOWS or Ubuntu~9.10} of \escript allow either \program{run-escript} or \program{escript} to be used but the latter name will be removed in future releases:
9  \begin{verbatim}  \begin{verbatim}
10  escript myscript.py  escript myscript.py
11  \end{verbatim}  \end{verbatim}
12  as already shown in section~\ref{FirstSteps}\footnote{For this discussion, it is assumed that \program{escript} is included in your \env{PATH} environment. See installation guide for details.}  as already shown in section~\ref{FirstSteps}\footnote{For this discussion, it is assumed that \program{run-escript} is included in your \env{PATH} environment. See installation guide for details.}
13  . In some cases  . In some cases
14  it can be useful to work interactively e.g. when debugging a script, with the command  it can be useful to work interactively e.g. when debugging a script, with the command
15  \begin{verbatim}  \begin{verbatim}
16  escript -i myscript.py  run-escript -i myscript.py
17  \end{verbatim}  \end{verbatim}
18  This will execute \var{myscript.py} and when it completes (or an error occurs), a \PYTHON prompt will be provided.  This will execute \var{myscript.py} and when it completes (or an error occurs), a \PYTHON prompt will be provided.
19  To leave the prompt press \kbd{Control-d}.  To leave the prompt press \kbd{Control-d}.
20
21  To start  To start
22  \program{escript} using four threads (eg. if you use a multi-core processor) you can use  \program{run-escript} using four threads (eg. if you use a multi-core processor) you can use
23  \begin{verbatim}  \begin{verbatim}
24  escript -t 4 myscript.py  run-escript -t 4 myscript.py
25  \end{verbatim}  \end{verbatim}
26  This will require {\it escript} to be compiled for \OPENMP\cite{OPENMP}.  This will require {\it escript} to be compiled for \OPENMP\cite{OPENMP}.
27
28  To start \program{escript} using \MPI\cite{MPI} with $8$ processes you use  To start \program{run-escript} using \MPI\cite{MPI} with $8$ processes you use
29  \begin{verbatim}  \begin{verbatim}
30  escript -p 8 myscript.py  run-escript -p 8 myscript.py
31  \end{verbatim}  \end{verbatim}
32  If the processors which are used are multi--core processors or multi--processor shared memory architectures you can use threading in addition to \MPI. For instance to run $8$ \MPI processes with using $4$ threads each, you use the command  If the processors which are used are multi--core processors or multi--processor shared memory architectures you can use threading in addition to \MPI. For instance to run $8$ \MPI processes with using $4$ threads each, you use the command
33  \begin{verbatim}  \begin{verbatim}
34  escript -p 8 -t 4 myscript.py  run-escript -p 8 -t 4 myscript.py
35  \end{verbatim}  \end{verbatim}
36  In the case of a super computer or a cluster, you may wish to distribute the workload over a number of nodes\footnote{For simplicity, we will use the term node to refer to either a node in a super computer or an individual machine in a cluster}.  In the case of a super computer or a cluster, you may wish to distribute the workload over a number of nodes\footnote{For simplicity, we will use the term node to refer to either a node in a super computer or an individual machine in a cluster}.
37  For example, to use $8$ nodes, with $4$ \MPI processes per node, write  For example, to use $8$ nodes, with $4$ \MPI processes per node, write
38  \begin{verbatim}  \begin{verbatim}
39  escript -n 8 -p 4 myscript.py  run-escript -n 8 -p 4 myscript.py
40  \end{verbatim}  \end{verbatim}
41  Since threading has some performance advantages over processes, you may specify a number of threads as well.  Since threading has some performance advantages over processes, you may specify a number of threads as well.
42  \begin{verbatim}  \begin{verbatim}
43  escript -n 8 -p 4 -t 2 myscript.py  run-escript -n 8 -p 4 -t 2 myscript.py
44  \end{verbatim}  \end{verbatim}
45  This runs the script on $8$ nodes, with $4$ processes per node and $2$ threads per process.  This runs the script on $8$ nodes, with $4$ processes per node and $2$ threads per process.
46
47  \section{Options}  \section{Options}
48  The general form of the \program{escript} launcher is as follows:  The general form of the \program{run-escript} launcher is as follows:
49
50  %%%%  %%%%
51  % If you are thinking about changing this please remember to update the man page as well  % If you are thinking about changing this please remember to update the man page as well
52  %%%%  %%%%
53
54  \program{escript}  \program{run-escript}
55  \optional{\programopt{-n \var{nn}}}  \optional{\programopt{-n \var{nn}}}
56  \optional{\programopt{-p \var{np}}}  \optional{\programopt{-p \var{np}}}
57  \optional{\programopt{-t \var{nt}}}  \optional{\programopt{-t \var{nt}}}
# Line 66  The general form of the \program{escript Line 69  The general form of the \program{escript
69  \optional{\var{ARGS}}  \optional{\var{ARGS}}
70
71  where \var{file} is the name of a script, \var{ARGS} are arguments for the script.  where \var{file} is the name of a script, \var{ARGS} are arguments for the script.
72  The \program{escript} program will import your current environment variables.  The \program{run-escript} program will import your current environment variables.
73  If no \var{file} is given, then you will be given a \PYTHON prompt (see \programopt{-i} for restrictions).  If no \var{file} is given, then you will be given a \PYTHON prompt (see \programopt{-i} for restrictions).
74
75  The options are used as follows:  The options are used as follows:
# Line 94  hosts defined in \var{hostfile} must be Line 97  hosts defined in \var{hostfile} must be
97   \item[\programopt{-i}] executes the script \var{file} and switches to interactive mode after the execution is finished or an exception has occurred. This option is useful for debugging a script. The option cannot be used if more than one process ($\var{nn} \cdot \var{np}>1$) is used.   \item[\programopt{-i}] executes the script \var{file} and switches to interactive mode after the execution is finished or an exception has occurred. This option is useful for debugging a script. The option cannot be used if more than one process ($\var{nn} \cdot \var{np}>1$) is used.
98  \item[\programopt{-b}] do not invoke python. This is used to run non-python programs.  \item[\programopt{-b}] do not invoke python. This is used to run non-python programs.
99
100   \item[\programopt{-e}] shows additional environment variables and commands used during \program{escript} execution. This option is useful if users wish to execute scripts without using the \program{escript} command.   \item[\programopt{-e}] shows additional environment variables and commands used to set up the \escript environment.
101    This option is useful if users wish to execute scripts without using the \program{run-escript} command.
102
103   \item[\programopt{-o}] switches on the redirection of output of processors with \MPI rank greater than zero to the files \file{stdout_\var{r}.out} and \file{stderr_\var{r}.out} where \var{r} is the rank of the processor. The option overwrites the value of the \env{ESCRIPT_STDFILES} environment variable   \item[\programopt{-o}] switches on the redirection of output of processors with \MPI rank greater than zero to the files \file{stdout_\var{r}.out} and \file{stderr_\var{r}.out} where \var{r} is the rank of the processor. The option overwrites the value of the \env{ESCRIPT_STDFILES} environment variable
104
# Line 107  hosts defined in \var{hostfile} must be Line 111  hosts defined in \var{hostfile} must be
111  \begin{itemize}  \begin{itemize}
112   \item Make sure that \program{mpiexec} is in your \env{PATH}.   \item Make sure that \program{mpiexec} is in your \env{PATH}.
113   \item For MPICH and INTELMPI and for the case a hostfile is present   \item For MPICH and INTELMPI and for the case a hostfile is present
114  \program{escript} will start the \program{mpd} demon before execution.  \program{run-escript} will start the \program{mpd} demon before execution.
115  \end{itemize}  \end{itemize}
116
117  \section{Input and Output}  \section{Input and Output}
118  When \MPI is used on more than one process ($\var{nn} \cdot \var{np} >1$) no input from the standard input is accepted. Standard output on any process other than the master process (\var{rank}=0) will not be available.  When \MPI is used on more than one process ($\var{nn} \cdot \var{np} >1$) no input from the standard input is accepted. Standard output on any process other than the master process (\var{rank}=0) will not be available.
119  Error output from any processor will be redirected to the node where \program{escript} has been invoked.  Error output from any processor will be redirected to the node where \program{run-escript} has been invoked.
120  If the \programopt{-o} or \env{ESCRIPT_STDFILES} is set\footnote{That is, it has a non-empty value.}, then the standard and error output from any process other than the master process will be written to files of the names \file{stdout_\var{r}.out} and \file{stderr_\var{r}.out} (where  If the \programopt{-o} or \env{ESCRIPT_STDFILES} is set\footnote{That is, it has a non-empty value.}, then the standard and error output from any process other than the master process will be written to files of the names \file{stdout_\var{r}.out} and \file{stderr_\var{r}.out} (where
121  \var{r} is the rank of the process).  \var{r} is the rank of the process).
122
# Line 120  If files are created or read by individu Line 124  If files are created or read by individu
124  This will avoid problems if processes are using a shared file system.  This will avoid problems if processes are using a shared file system.
125  Files which collect data which are global for all \MPI processors will be created by the process with \MPI rank 0 only.  Files which collect data which are global for all \MPI processors will be created by the process with \MPI rank 0 only.
126  Users should keep in mind that if the file system is not shared, then a file containing global information  Users should keep in mind that if the file system is not shared, then a file containing global information
127  which is read by all processors needs to be copied to the local file system before \program{escript} is invoked.  which is read by all processors needs to be copied to the local file system before \program{run-escript} is invoked.
128
129
130  \section{Hints for MPI Programming}  \section{Hints for MPI Programming}
131  In general a script based on the \escript module does not require modifications when running under \MPI. However, one needs to be careful if other modules are used.  In general a script based on the \escript module does not require modifications to run under \MPI.
132    However, one needs to be careful if other modules are used.
133
134  When \MPI is used on more than one process ($\var{nn} \cdot \var{np} >1$) the user needs to keep in mind that several copies of his script are executed at the same time  When \MPI is used on more than one process ($\var{nn} \cdot \var{np} >1$) the user needs to keep in mind that several copies of his script are executed at the same time
135  \footnote{In case of OpenMP only one copy is running but \escript temporarily spawns threads.} while data exchange is performed through the \escript module.  \footnote{In case of OpenMP only one copy is running but \escript temporarily spawns threads.} while data exchange is performed through the \escript module.
# Line 133  This has three main implications: Line 138  This has three main implications:
138  \begin{enumerate}  \begin{enumerate}
139   \item most arguments (\var{Data} excluded) should the same values on all processors. eg \var{int}, \var{float}, \var{str}   \item most arguments (\var{Data} excluded) should the same values on all processors. eg \var{int}, \var{float}, \var{str}
140  and \numpy parameters.  and \numpy parameters.
141  \item the same operations will be called on processors.  \item the same operations will be called on all processors.
142  \item different processors may store different amounts of information.  \item different processors may store different amounts of information.
143  \end{enumerate}  \end{enumerate}
144
# Line 182  dioes not handle exceptions. Line 187  dioes not handle exceptions.
187  However, \MPI will terminate the other processes but  However, \MPI will terminate the other processes but
188  may not inform the user of the reason in an obvious way. The user needs to inspect the  may not inform the user of the reason in an obvious way. The user needs to inspect the
189  error output files to identify the exception.  error output files to identify the exception.
190
191    \section{Lazy Evaluation}
192    \label{sec:lazy}
193    Escript now supports lazy evaluation~\cite{lazyauspdc}.
194    Lazy evaluation is when expressions are not evaluated until it is actually needed.
195    When applied to suitable problems, it can reduce both the memory and cpu time required to
196    perform a simulation.
197    This implementation is designed to be as transparent as possible; so significant
198    alterations to scripts are not required.
199
200    \subsection*{How to use it}
201    To have lazy evaluation applied automatically, put the following command in your script
202    after the imports.
203
204    \begin{python}
205    from esys.escript import setEscriptParamInt
206    setEscriptParamInt('AUTOLAZY',1)
207    \end{python}
208
209    To get greater benefit, some fine tuning may be required.
210    If your simulation involves iterating for a number of timesteps,
211    you will probably have some state variables which are updated in
212    each iteration based on their value in the previous iteration.
213    For example:
214
215    \begin{python}
216    x=f(x_previous)
217    y=g(x)
218    z=h(y,x, ...)
219    \end{python}
220
221    Could be modified to:
222
223    \begin{python}
224    x=f(x_previous)
225    resolve(x)
226    y=g(x)
227    z=h(y,x, ...)
228    \end{python}
229
230    The resolve command forces x to be evaluated immediately.
231
232    \subsection*{When to use it}
233    We believe that problems involving large domains and complicated expressions
234    will benefit most from lazy evaluation.
235    In cases where lazy does provide a benefit, larger domains should provide
236    larger benefit.
237    If you are uncertain, try running a test on a smaller domain.
238
239

Legend:
 Removed from v.2780 changed lines Added in v.2923