Parent Directory
| 
Revision Log
| Links to HEAD: | (view) (annotate) |
| Sticky Revision: |
And get the *(&(*&(* name right
Restore the trunk that existed before the windows changes were committed to the (now moved to branches) old trunk.
Make a temp copy of the trunk before checking in the windows changes
The MPI branch is hereby closed. All future work should be in trunk. Previously in revision 1295 I merged the latest changes to trunk into trunk-mpi-branch. In this revision I copied all files from trunk-mpi-branch over the corresponding trunk files. I did not use 'svn merge', it was a copy.
Started the removal of IS_WINDOWS_PLATFORM, and now insis that the windows options file be kept up to date, and be complete. boost_libs -> boost_lib.
Finished renaming of boost_lib to boost_libs and boost_lib_path to boost_libs_path.
Changes needed for windows. This still does not work on and has not run on linux and the altix. There are a few changes in names in the scons script, in particular _libs is used rather than _lib and _libs is always a list. By default on windows MSVC is used. The location of python is at C:\Program Files\python<version>, boost at C:\Program Files\boost and netcdf at C:\Program Files\netcdf.
A few changes in the build mechanism and the file structure so scons can build release tar files:
* paso/src/Solver has been moved to paso/src
* all test_.py are now run_.py files and are assumed to be passing python tests. they can run by
scons py_tests and are part of the release test set
* escript/py_src/test_ are moved to escript/test/python and are installed in to the build directory
(rather then the PYTHONPATH).
* all py files in test/python which don't start with run_ or test_ are now 'local_py_tests'. they are installed i
by not run automatically.
* CppUnitTest is now treated as a escript module (against previous decisions).
* scons realse builds nor tar/zip files with relvant source code (src and tests in seperate files)
the python tests don't pass yet due to path problems.
+ NEW BUILD SYSTEM
This commit contains the new build system with cross-platform support.
Most things work are before though you can have more control.
ENVIRONMENT settings have changed:
+ You no longer require LD_LIBRARY_PATH or PYTHONPATH to point to the
esysroot for building and testing performed via scons
+ ACcESS altix users: It is recommended you change your modules to load
the latest intel compiler and other libraries required by boost to match
the setup in svn (you can override). The correct modules are as follows
module load intel_cc.9.0.026
export
MODULEPATH=${MODULEPATH}:/data/raid2/toolspp4/modulefiles/gcc-3.3.6
module load boost/1.33.0/python-2.4.1
module load python/2.4.1
module load numarray/1.3.3
+ Modified test SConscript dependencies to remove redundant call to explicit dependency + Modified scon_extensions.py - run unit tests (py and C++) now use scons::Execute rather than python os.system. This ensures the development environment is used rather than the users environment to run the tests + SConstruct file now sets up LD_LIBRARY_PATH and PYTHONPATH to point to the current builds pyinstall and libinstall paths as required. IT IS NO LONGER NECESSARY TO SET LD_LIBRARY_PATH and PYTHONPATH to point at your build outputs. Much safer in the presence of multiple checked out builds. Under these circumstances you are better of using scons to run the tests rather than doing so directly. Easiest way to do this is to build the test output target for the test you want: e.g. scons build/posix/bruce/test/python/ImportTest.passed or if you prefer cd build/posix/bruce/test/python scons -u ImportTest.passed
+ Changed include paths to no longer require the cpp suffix (missed in earlier commit) + modified tests so they no longer install to #/lib
+ finally figured out why the tests failed when the python extension libs were
no longer sym links. The original setup for python extension modules was to
have sym links in the esys/* directories named, for example escriptcpp.so.
These would link to the actual libraries libescriptcpp.so. The lib*.so would
link to each other. When you replaced the symlink with a copy of the lib*.so
but renamed without the lib you would then get problems. In particular,
py_tests would suddenly start failing.
The problem appears (I've not been able to find documentary evidence to this
case) to be that when, for example, you import bruce, brucecpp.so imports
lib/libescriptcpp.so (which intialises escript python bits), bruce then
imports escriptcpp.so (which also initialised escript python bits).
Whether that is exactly correct or not is of interest but the important bit
is you appear to get two versions. After thinking about this for a bit and
reviewing a bunch of other examples of working python modules I noticed a
pattern. NONE of the other examples ever included more than the python
wrapper code in the python extension library. Instead they just link to
the pure C++ library. This would avoid the duplicate load.
So, I've refactored the code. If you consider escript the pattern in now:
lib/libescript.so - which has all of escript EXCEPT the python escriptcpp.cpp
esys/escript/escriptcpp.so - which has ONLY the escriptcpp.cpp and links to
lib/libescript.so
Run the tests and low and behold they all pass again.
Q: Why doesn't this problem occur with sym links?
A: My guess is that python and the dynamic linker take a look at the actual
absolute python of libraries to determine if its a "different" library.
I did fine some discussions that seem to suggest this.
Q: Why can't you just set LD_LIBRARY_PATH==PYTHONPATH and stick all the
libs in that directory?
A: I (in fact we, Peter Hornby was there) tried. With the renaming of the
python module so it doesn't have a lib prefix you get problems with
getting the shared libraries to be looked up in LD_LIBRARY_PATH. Do
the opposite and use lib on the python modules and you have problems
with windows which doesn't prepend the lib. Various combination
in between were also tried but you end up in a catch 22 situation so far
as I could tell
Please if you know more about the ins and outs of python and shared
libraries let me know if this isn't true. I'd really like to know if my
guesses are correct. In any event, the fix is more consistent with the
patterns I've seen
Phew! this was a long log message, glad it is on a branch!
+ OUCH ARGHH + Okay the wizzy linkHack doesn't work. Not sure why but py_tests fail when the shared library is on the link line directly (without a -l) + Unfortunately we can't figure out how to use -l without it prepending a lib prefix. Hence all the shareable objects in python now need to be called libblah.so (and lib will need to be prepended on windows!!!!). This is not the python way and its very annoying. Why do we have this problem? Because the C++ shared libraries link with each other.
+ Fix up python esys area:
+ libraries are named without lib prefix on posix platforms (python standard)
+ libraries are now installed into the pyinstall area (python standard)
+ symlinks removed, no longer required
+ LD_LIBRARY_PATH is now optional for PYTHON programs (still required for c++)
+ lib PREFIX removal NOTES:
+ removing the lib prefix is non-standard for the linker. As such we've created a
custom function sharedLinkHack to specify the .so on the link command line
(as opposed to using -l<archive> which will prepend the lib). This has a
a small slight of hand for scons which was being to "helpful". Scons
verifies that when creating a shared libray all objects specified are
created shareable (e.g. ensure objects are compiler with -fPIC). Since
we are linking a .so out of the blue we had to wrap it up in a File note
and flag it as shared.
Easily done, once you know how. Thankfully this is all wrapped
up in a simple function (sharedLinkHack) that looks like ordinary scons so
you will never know!
+py_tests now runs. Be patient, it is all of them!!
+Opps, helps if you include the two new files for Finley tests
This form allows you to request diffs between any two revisions of this file. For each of the two "sides" of the diff, enter a numeric revision.
| ViewVC Help | |
| Powered by ViewVC 1.1.26 |