# Contents of /trunk/doc/user/execute.tex

Revision 6927 - (show annotations)
Sun Dec 8 22:26:40 2019 UTC (11 months, 3 weeks ago) by acodd
File MIME type: application/x-tex
File size: 13023 byte(s)
user guide updates


 1 2 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 3 % Copyright (c) 2003-2018 by The University of Queensland 4 5 % 6 % Primary Business: Queensland, Australia 7 % Licensed under the Apache License, version 2.0 8 9 % 10 % Development until 2012 by Earth Systems Science Computational Center (ESSCC) 11 % Development 2012-2013 by School of Earth Sciences 12 % Development from 2014 by Centre for Geoscience Computing (GeoComp) 13 % 14 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 15 16 \chapter{Execution of an {\it escript} Script} 17 \label{EXECUTION} 18 19 \section{Overview} 20 A typical way of starting your {\it escript} script \file{myscript.py} is with 21 the \program{run-escript} command\index{run-escript}\footnote{The 22 \program{run-escript} launcher is not supported under \WINDOWS.}. 23 This command was renamed from \program{escript} (used in previous releases) to 24 avoid clashing with an unrelated program installed by default on some systems. 25 To run your script, issue\footnote{For this discussion, it is assumed that 26 \program{run-escript} is included in your \env{PATH} environment. See the 27 installation guide for details.} 28 \begin{verbatim} 29 run-escript myscript.py 30 \end{verbatim} 31 as already shown in \Sec{FirstSteps}. 32 In some cases it can be useful to work interactively, e.g. when debugging a 33 script, with the command 34 \begin{verbatim} 35 run-escript -i myscript.py 36 \end{verbatim} 37 This will execute \var{myscript.py} and when it completes (or an error occurs), 38 a \PYTHON prompt will be provided. 39 To leave the prompt press \kbd{Control-d} (\kbd{Control-z} on \WINDOWS). 40 41 To run the script using four threads (e.g. if you have a multi-core processor) 42 you can use 43 \begin{verbatim} 44 run-escript -t 4 myscript.py 45 \end{verbatim} 46 This requires {\it escript} to be compiled with \OPENMP\cite{OPENMP} support. 47 To run the script using \MPI\cite{MPI} with 8 processes use 48 \begin{verbatim} 49 run-escript -p 8 myscript.py 50 \end{verbatim} 51 If the processors which are used are multi-core processors or you are working 52 on a multi-processor shared memory architecture you can use threading in 53 addition to \MPI. 54 For instance to run 8 \MPI processes with 4 threads each, use the command 55 \begin{verbatim} 56 run-escript -p 8 -t 4 myscript.py 57 \end{verbatim} 58 In the case of a supercomputer or a cluster, you may wish to distribute the 59 workload over a number of nodes\footnote{For simplicity, we will use the term 60 \emph{node} to refer to either a node in a supercomputer or an individual 61 machine in a cluster}. 62 For example, to use 8 nodes with 4 \MPI processes per node, write 63 \begin{verbatim} 64 run-escript -n 8 -p 4 myscript.py 65 \end{verbatim} 66 Since threading has some performance advantages over processes, you may 67 specify a number of threads as well: 68 \begin{verbatim} 69 run-escript -n 8 -p 2 -t 4 myscript.py 70 \end{verbatim} 71 This runs the script on 8 nodes, with 2 processes per node and 4 threads per 72 process. 73 74 \section{Options} 75 The general form of the \program{run-escript} launcher is as follows: 76 77 %%%% 78 % If you are thinking about changing this please remember to update the man page as well 79 %%%% 80 81 \program{run-escript} 82 \optional{\programopt{-n \var{nn}}} 83 \optional{\programopt{-p \var{np}}} 84 \optional{\programopt{-t \var{nt}}} 85 \optional{\programopt{-f \var{hostfile}}} 86 \optional{\programopt{-x}} 87 \optional{\programopt{-V}} 88 \optional{\programopt{-e}} 89 \optional{\programopt{-h}} 90 \optional{\programopt{-v}} 91 \optional{\programopt{-o}} 92 \optional{\programopt{-c}} 93 \optional{\programopt{-i}} 94 \optional{\programopt{-b}} 95 \optional{\programopt{-m \var{tool}}} 96 \optional{\var{file}} 97 \optional{\var{ARGS}} 98 99 where \var{file} is the name of a script and \var{ARGS} are the arguments to 100 be passed to the script. 101 The \program{run-escript} program will import your current environment variables. 102 If no \var{file} is given, then you will be presented with a regular \PYTHON 103 prompt (see \programopt{-i} for restrictions). 104 105 The options have the following meaning: 106 \begin{itemize} 107 \item[\programopt{-n} \var{nn}] the number of compute nodes \var{nn} to be used. 108 The total number of processes being used is $\var{nn} \cdot \var{np}$. 109 This option overrides the value of the \env{ESCRIPT_NUM_NODES} 110 environment variable. 111 If a \var{hostfile} is given (see below), the number of nodes needs to 112 match the number of hosts given in that file. 113 If $\var{nn}>1$ but {\it escript} is not compiled for \MPI, a warning is 114 printed but execution is continued with $\var{nn}=1$. 115 If \programopt{-n} is not set the number of hosts in the host file is 116 used. The default value is 1. 117 118 \item[\programopt{-p} \var{np}] the number of \MPI processes (per node). 119 The total number of processes to be used is $\var{nn} \cdot \var{np}$. 120 This option overwrites the value of the \env{ESCRIPT_NUM_PROCS} 121 environment variable. 122 If $\var{np}>1$ but {\it escript} is not compiled for \MPI, a warning is 123 printed but execution is continued with $\var{np}=1$. 124 The default value is 1. 125 126 \item[\programopt{-t} \var{nt}] the number of threads used per process. 127 The option overwrites the value of the \OPENMP environment variable \env{ESCRIPT_NUM_THREADS}. 128 If $\var{nt}>1$ but {\it escript} is not compiled for \OPENMP, a warning 129 is printed but execution is continued with $\var{nt}=1$. 130 The default value is 1. 131 132 \item[\programopt{-f} \var{hostfile}] the name of a file with a list of host names. 133 Some systems require to specify the addresses or names of the compute 134 nodes where \MPI processes should be spawned. 135 These addresses or names of the compute nodes are listed in the file with 136 the name \var{hostfile}. 137 If \programopt{-n} is set, the number of different hosts defined in \var{hostfile} 138 must be equal to the number of requested compute nodes \var{nn}. 139 The option overwrites the value of the \env{ESCRIPT_HOSTFILE} environment 140 variable. By default no host file is used. 141 142 \item[\programopt{-c}] prints information about the settings used to compile {\it escript} and stops execution. 143 144 \item[\programopt{-V}] prints the version of {\it escript} and stops execution. 145 146 \item[\programopt{-h}] prints a help message and stops execution. 147 148 \item[\programopt{-i}] executes the script \var{file} and switches to 149 interactive mode after the execution is finished or an exception has occurred. 150 This option is useful for debugging a script. 151 The option cannot be used if more than one process ($\var{nn} \cdot \var{np}>1$) is used. 152 153 \item[\programopt{-b}] do not invoke python. This is used to run non-python 154 programs within an environment set for {\it escript}. 155 156 \item[\programopt{-e}] shows additional environment variables and commands 157 used to set up the {\it escript} environment. 158 This option is useful if users wish to execute scripts without using 159 the \program{run-escript} command. 160 161 \item[\programopt{-o}] enables the redirection of messages printed by 162 processors with \MPI rank greater than zero to the files 163 \file{stdout_\var{r}.out} and \file{stderr_\var{r}.out} where \var{r} is 164 the rank of the processor. 165 The option overwrites the value of the \env{ESCRIPT_STDFILES} environment 166 variable. 167 168 \item[\programopt{-x}] runs everything within a new \emph{xterm} instance. 169 170 \item[\programopt{-v}] prints some diagnostic information. 171 172 \item[\programopt{-m} \var{tool}] runs under \emph{valgrind}. The argument 173 \var{tool} must be one of \var{m} (for memcheck), \var{c} (for callgrind), 174 or \var{h} (for cachegrind). Valgrind output is written to a file under 175 \file{valgrind_logs} as reported when {\it escript} terminates. 176 \end{itemize} 177 178 \subsection{Notes} 179 The \program{run-escript} script is generated at build time taking into 180 account the \var{prelaunch}, \var{launcher}, and \var{postlaunch} settings 181 passed to \program{scons}. This makes it possible to easily customize the 182 script for different environments, such as batch systems (PBS, SLURM) and 183 different implementations of MPI (Intel, SGI, OpenMPI, etc.). 184 185 \section{Input and Output} 186 When \MPI is used on more than one process ($\var{nn} \cdot \var{np} >1$) no 187 input from the standard input is accepted. 188 Standard output on any process other than the master process (\var{rank}= 0) 189 will be silently discarded by default. 190 Error output from any processor will be redirected to the node where \program{run-escript} has been invoked. 191 If the \programopt{-o} option or \env{ESCRIPT_STDFILES} is set\footnote{That is, it has a non-empty value.}, 192 then the standard and error output from any process other than the master 193 process will be written to files of the names \file{stdout_R.out} 194 and \file{stderr_R.out} (where \var{R} is the rank of the process). 195 196 If files are created or read by individual \MPI processes with information 197 local to the process (e.g. in the \function{dump} function) and more than one 198 process is used ($\var{nn} \cdot \var{np} >1$), the \MPI process rank is 199 appended to the file names. 200 This is to avoid problems if processes are using a shared file system. 201 Files which collect data that are global for all \MPI processors are created 202 by the process with \MPI rank 0 only. 203 Users should keep in mind that if the file system is not shared among the 204 processes, then a file containing global information which is read by all 205 processors needs to be copied to the local file system(s) before \program{run-escript} is invoked. 206 207 \section{Hints for MPI Programming} 208 In general a script based on the \escript module does not require 209 modifications to run under \MPI. 210 However, one needs to be careful if other modules are used. 211 212 When \MPI is used on more than one process ($\var{nn} \cdot \var{np} >1$) the 213 user needs to keep in mind that several copies of his script are executed at 214 the same time\footnote{In the case of \OPENMP only one copy is running 215 but {\it escript} temporarily spawns threads.} while data exchange is 216 performed through the \escript module. 217 218 This has three main implications: 219 \begin{enumerate} 220 \item most arguments (\var{Data} excluded) should have the same values on all 221 processors, e.g. \var{int}, \var{float}, \var{str} and \numpy parameters. 222 \item the same operations will be called on all processors. 223 \item different processors may store different amounts of information. 224 \end{enumerate} 225 226 With a few exceptions\footnote{\var{getTupleForDataPoint}}, values of 227 types \var{int}, \var{float}, \var{str} and \numpy returned by \escript will 228 have the same value on all processors. 229 If values produced by other modules are used as arguments, the user has to 230 make sure that the argument values are identical on all processors. 231 For instance, the usage of a random number generator to create argument values 232 bears the risk that the value may depend on the processor. 233 234 Some operations in \escript require communication with all processors 235 executing the job. It is not always obvious which operations these are. 236 For example, \var{Lsup} returns the largest value on all processors. 237 \var{getValue} on \var{Locator} may refer to a value stored on another processor. 238 For this reason it is better if scripts do not have conditional operations 239 (which manipulate data) based on which processor the script is on. 240 Crashing or hanging scripts can be an indication that this has happened. 241 242 It is not always possible to divide data evenly amongst processors. 243 In fact some processors might not have any data at all. 244 Try to avoid writing scripts which iterate over data points, instead try to 245 describe the operation you wish to perform as a whole. 246 247 Special attention is required when using files on more than one processor as 248 several processors access the file at the same time. Opening a file for 249 reading is safe, however the user has to make sure that the variables which 250 are set from reading data from files are identical on all processors. 251 252 When writing data to a file it is important that only one processor is writing 253 to the file at any time. As all values in \escript are global it is sufficient 254 to write values on the processor with \MPI rank $0$ only. 255 The \class{FileWriter} class provides a convenient way to write global data 256 to a simple file. The following script writes to the file \file{test.txt} on 257 the processor with rank 0 only: 258 \begin{python} 259 from esys.escript import FileWriter 260 f = FileWriter('test.txt') 261 f.write('test message') 262 f.close() 263 \end{python} 264 We strongly recommend using this class rather than \PYTHON's built-in \function{open} 265 function as it will guarantee a script which will run in single processor mode 266 as well as under \MPI. 267 268 If the situation occurs that one of the processors throws an exception, for 269 instance when opening a file for writing fails, the other processors are not 270 automatically made aware of this since \MPI does not handle exceptions. 271 However, \MPI will still terminate the other processes but may not inform the 272 user of the reason in an obvious way. 273 The user needs to inspect the error output files to identify the exception.