/[escript]/branches
ViewVC logotype

Log of /branches

View Directory Listing Directory Listing

Sticky Revision:
Revision 678 - Directory Listing
Modified Sun Mar 26 12:20:00 2006 UTC (17 years ago) by robwdcock 
+Allow for environment variable PATH, LD_LIBRARY_PATH, and PYTHONPATH to 
not exist in the users environment.
Revision 677 - Directory Listing
Modified Sun Mar 26 11:25:19 2006 UTC (17 years ago) by robwdcock 
+ Opps forgot to commit the lines that add the PYTHONPATH to the scons ENV used for calling external programs
Revision 676 - Directory Listing
Modified Sun Mar 26 10:13:34 2006 UTC (17 years ago) by robwdcock 
+ Modified test SConscript dependencies to remove redundant call to explicit dependency
+ Modified scon_extensions.py - run unit tests (py and C++) now use scons::Execute rather than python os.system. This ensures the development environment
is used rather than the users environment to run the tests
+ SConstruct file now sets up LD_LIBRARY_PATH and PYTHONPATH to point to the current builds pyinstall and libinstall paths as required.
IT IS NO LONGER NECESSARY TO SET LD_LIBRARY_PATH and PYTHONPATH to point at your build outputs. Much safer in the presence of multiple checked out builds.
Under these circumstances you are better of using scons to run the tests rather than doing so directly. Easiest way to do this is to build the test output target for the test you want:
e.g. scons build/posix/bruce/test/python/ImportTest.passed
or if you prefer
cd build/posix/bruce/test/python
scons -u ImportTest.passed
Revision 675 - Directory Listing
Modified Sun Mar 26 07:28:06 2006 UTC (17 years ago) by robwdcock 
+ Add an all_tests targets: run_tests and py_tests
Revision 674 - Directory Listing
Modified Sun Mar 26 07:21:48 2006 UTC (17 years ago) by robwdcock 
+Some more svn:ignore properties.
Revision 673 - Directory Listing
Modified Sun Mar 26 07:16:42 2006 UTC (17 years ago) by robwdcock 
+ Added some svn:ignore properties so .pyc files aren't reported during svn status and friends
+ Removed esys directory. Build system will create the pyinstall/esys target and touch the __init__.py file in it as part of the build process. This means you can now direct scons to perform an install of the python directly into your desired location rather than just in the root of the source tree.
Revision 672 - Directory Listing
Modified Sat Mar 25 16:21:22 2006 UTC (17 years ago) by robwdcock 
+ Scons cflags are added to cxxflags. Only need to list extra flags for
C++
Revision 671 - Directory Listing
Modified Sat Mar 25 16:04:24 2006 UTC (17 years ago) by robwdcock 
+ scons adds CCFLAGS to CXXFLAGS so the c++ flags only need to specify the c++ specific ones in the options file.
Revision 670 - Directory Listing
Modified Sat Mar 25 14:56:39 2006 UTC (17 years ago) by robwdcock 
+ Changed include paths to no longer require the cpp suffix (missed in earlier
  commit)
+ modified tests so they no longer install to #/lib
Revision 669 - Directory Listing
Modified Sat Mar 25 13:35:55 2006 UTC (17 years ago) by robwdcock 
+ Added lib back in. Once cppunit is removed this will no longer be
necessary as scons will no longer through up an error when looking for 
cppunit lib path when it hasn't been built yet.
Revision 668 - Directory Listing
Modified Sat Mar 25 13:27:27 2006 UTC (17 years ago) by robwdcock 
+ finally figured out why the tests failed when the python extension libs were
  no longer sym links. The original setup for python extension modules was to
  have sym links in the esys/* directories named, for example escriptcpp.so.
  These would link to the actual libraries libescriptcpp.so. The lib*.so would
  link to each other. When you replaced the symlink with a copy of the lib*.so
  but renamed without the lib you would then get problems. In particular,
  py_tests would suddenly start failing.

  The problem appears (I've not been able to find documentary evidence to this
  case) to be that when, for example, you import bruce, brucecpp.so imports
  lib/libescriptcpp.so (which intialises escript python bits), bruce then
  imports escriptcpp.so (which also initialised escript python bits).

  Whether that is exactly correct or not is of interest but the important bit
  is you appear to get two versions. After thinking about this for a bit and
  reviewing a bunch of other examples of working python modules I noticed a
  pattern. NONE of the other examples ever included more than the python
  wrapper code in the python extension library. Instead they just link to
  the pure C++ library. This would avoid the duplicate load.

  So, I've refactored the code. If you consider escript the pattern in now:
  lib/libescript.so - which has all of escript EXCEPT the python escriptcpp.cpp
  esys/escript/escriptcpp.so - which has ONLY the escriptcpp.cpp and links to
                               lib/libescript.so
  
  Run the tests and low and behold they all pass again.

  Q: Why doesn't this problem occur with sym links?
  A: My guess is that python and the dynamic linker take a look at the actual
     absolute python of libraries to determine if its a "different" library.
     I did fine some discussions that seem to suggest this.

  Q: Why can't you just set LD_LIBRARY_PATH==PYTHONPATH and stick all the
     libs in that directory?
  A: I (in fact we, Peter Hornby was there) tried. With the renaming of the
     python module so it doesn't have a lib prefix you get problems with
     getting the shared libraries to be looked up in LD_LIBRARY_PATH. Do
     the opposite and use lib on the python modules and you have problems
     with windows which doesn't prepend the lib. Various combination
     in between were also tried but you end up in a catch 22 situation so far
     as I could tell

  Please if you know more about the ins and outs of python and shared
  libraries let me know if this isn't true. I'd really like to know if my
  guesses are correct. In any event, the fix is more consistent with the
  patterns I've seen

  Phew! this was a long log message, glad it is on a branch!
Revision 667 - Directory Listing
Modified Fri Mar 24 13:25:00 2006 UTC (17 years ago) by robwdcock 
+ OUCH ARGHH
+ Okay the wizzy linkHack doesn't work. Not sure why but py_tests fail
when the shared library is on the link line directly (without a -l)

+ Unfortunately we can't figure out how to use -l without it prepending
a lib prefix. Hence all the shareable objects in python now need to be
called libblah.so (and lib will need to be prepended on windows!!!!).
This is not the python way and its very annoying. Why do we have this
problem? Because the C++ shared libraries link with each other.
Revision 666 - Directory Listing
Modified Fri Mar 24 11:29:40 2006 UTC (17 years ago) by robwdcock 
+ Fix up python esys area:
  + libraries are named without lib prefix on posix platforms (python standard)
  + libraries are now installed into the pyinstall area (python standard)
  + symlinks removed, no longer required
  + LD_LIBRARY_PATH is now optional for PYTHON programs (still required for c++)
+ lib PREFIX removal NOTES:
  + removing the lib prefix is non-standard for the linker. As such we've created a
    custom function sharedLinkHack to specify the .so on the link command line
    (as opposed to using -l<archive> which will prepend the lib). This has a
    a small slight of hand for scons which was being to "helpful". Scons
    verifies that when creating a shared libray all objects specified are
    created shareable (e.g. ensure objects are compiler with -fPIC). Since
    we are linking a .so out of the blue we had to wrap it up in a File note
    and flag it as shared.
    Easily done, once you know how. Thankfully this is all wrapped
    up in a simple function (sharedLinkHack) that looks like ordinary scons so
    you will never know!
Revision 665 - Directory Listing
Modified Fri Mar 24 09:30:44 2006 UTC (17 years ago) by robwdcock 
+Some tidying up and fixme comments so we remember to fix this test later
Revision 664 - Directory Listing
Modified Fri Mar 24 09:28:36 2006 UTC (17 years ago) by robwdcock 
+Remove old unit test
Revision 663 - Directory Listing
Modified Fri Mar 24 09:26:58 2006 UTC (17 years ago) by robwdcock 
+Remove old Sconstruct files
Revision 662 - Directory Listing
Modified Fri Mar 24 09:15:54 2006 UTC (17 years ago) by robwdcock 
+Modellib now include in build
Revision 661 - Directory Listing
Modified Fri Mar 24 09:13:00 2006 UTC (17 years ago) by robwdcock 
+py_tests now runs. Be patient, it is all of them!!
Revision 659 - Directory Listing
Modified Fri Mar 24 08:40:50 2006 UTC (17 years ago) by robwdcock 
+Added dependency to libinstall. Painful because these tests should depend only on escript. Unfortunately some python unit tests are requiring finley to do stuff.
Revision 658 - Directory Listing
Modified Fri Mar 24 08:39:41 2006 UTC (17 years ago) by robwdcock 
+Linker is sensitive to library order. mkl_solver needs to be near the
end or paradiso_ goes missing
Revision 656 - Directory Listing
Modified Fri Mar 24 08:12:40 2006 UTC (17 years ago) by robwdcock 
+Add sys_libs to libraries to link with for escript
Revision 655 - Directory Listing
Modified Fri Mar 24 07:55:21 2006 UTC (17 years ago) by robwdcock 
# fix some lib flags
Revision 654 - Directory Listing
Modified Fri Mar 24 07:35:06 2006 UTC (17 years ago) by robwdcock 
+Opps, missed a file again
Revision 652 - Directory Listing
Modified Fri Mar 24 07:34:18 2006 UTC (17 years ago) by robwdcock 
+Add py_tests targets and get script ones running
Revision 651 - Directory Listing
Modified Fri Mar 24 06:44:23 2006 UTC (17 years ago) by robwdcock 
+Python file now compile and install into pyinstall directory
Revision 650 - Directory Listing
Modified Fri Mar 24 06:13:52 2006 UTC (17 years ago) by phornby 
Builds and passes on linux (debian)
Revision 649 - Directory Listing
Modified Fri Mar 24 06:07:07 2006 UTC (17 years ago) by robwdcock 
+Opps, helps if you include the two new files for Finley tests
Revision 648 - Directory Listing
Modified Fri Mar 24 06:04:57 2006 UTC (17 years ago) by robwdcock 
+Finely unit tests now running in build system

All C++ tests run and compile on Altix, I expect the same result for Linux so you may not see another log message for that.
Revision 647 - Directory Listing
Modified Fri Mar 24 05:23:05 2006 UTC (17 years ago) by robwdcock 
+ Prep for unit test consolidation consistent with other modules
NOTE: HBtest and MMtest appear to be stale tests and are not built using the scons system. They have been removed.
Revision 646 - Directory Listing
Modified Fri Mar 24 04:22:28 2006 UTC (17 years ago) by robwdcock 
+ Now compiles and passes tests on AcCESS machine. Some minor mods to
the configuraiton file required as we've moved to the intel 9.0.026
compiler
Revision 642 - Directory Listing
Modified Thu Mar 23 14:03:49 2006 UTC (17 years ago) by robwdcock 
+ Bruce unit tests consolidated and added to build system
Revision 641 - Directory Listing
Modified Thu Mar 23 14:02:26 2006 UTC (17 years ago) by robwdcock 
+ First compilation under Linux with new xplatform build system. Some minor adjustments because didn't like None objects in a list so using '' instead
Note: Most everything appears to compile but the Data test suite is failing to pass in testMore - Exercise wherePositive method.
This test passes on altix.
Revision 640 - Directory Listing
Modified Thu Mar 23 13:45:12 2006 UTC (17 years ago) by robwdcock 
+Preparing for unit test consolidation
Revision 639 - Directory Listing
Modified Thu Mar 23 13:06:40 2006 UTC (17 years ago) by robwdcock 
+Unit tests for esysUtils and escript added
Revision 638 - Directory Listing
Modified Thu Mar 23 13:06:01 2006 UTC (17 years ago) by robwdcock 
+Unit tests unified into escript_UnitTest
+SConscript support for build_tests and run_tests
Revision 636 - Directory Listing
Modified Thu Mar 23 09:46:31 2006 UTC (17 years ago) by robwdcock 
Clean up test into single test suite and only one SConscript file so it is easier to maintain.
Note:
+ Use the command line options on tests in order to run only one of them if you need to
+ THIS DOESN'T WORK YET: I've not merged the tests, that will follow shortly
+ I do hope this merges with the changes on the trunk!
Revision 635 - Directory Listing
Modified Thu Mar 23 08:56:35 2006 UTC (17 years ago) by robwdcock 
+EsysExceptionTest and build_tests and run_tests targets now operating as per original build system
Revision 634 - Directory Listing
Modified Thu Mar 23 06:03:46 2006 UTC (17 years ago) by robwdcock 
+ Opps windows build wasn't getting options passed into environment!
Note: Windows build environment for intel_c compiler is only partially building. Looks like there will be a few source code adjustments necessary to tweak things. Still, much better than the msvc compiler. It also has openmp support - now I just need a dual core cpu notebook upgrade.
Revision 632 - Directory Listing
Modified Thu Mar 23 05:00:51 2006 UTC (17 years ago) by robwdcock 
+Fix problem in setting flags when no options file is used
Revision 630 - Directory Listing
Modified Thu Mar 23 04:02:59 2006 UTC (17 years ago) by robwdcock 
Some minor tweaks on the flags for the new build system
Revision 624 - Directory Listing
Modified Wed Mar 22 23:00:01 2006 UTC (17 years ago) by robwdcock 
+ Finish off the header file path changes
+ The C/C++ libraries now all compile on the ivec altix (cognac) using the xplatform build system
Revision 623 - Directory Listing
Modified Wed Mar 22 14:11:03 2006 UTC (17 years ago) by robwdcock 
Beginning of a fully cross-platform (win32+intelc, linux+g++, altix+intelc)
scons build system
WORK IN PROGRESS NOT COMPLETE
+Incorporates ideas from trunks scons build system and the RW_WIN32 branch
+New Header layout as include/<modulename>, hence the changes to the headers
+Not all modules or features from trunk build system are fully implemented
-Removes need for intermediary SConstruct files - the top SConstruct is more complex but handle cross-platform better AND the SConscript are a LOT simpler
and only differ slightly. It should be simpler to maintain
-Options are now handled via either a: config file supplied on command line, a config file in the scons/hostname_options.py, or as individual command line arguments
-Need help with options: scons -h will give you a list of options and their current settings
Revision 622 - Directory Listing
Modified Wed Mar 22 14:00:08 2006 UTC (17 years ago) by robwdcock 
Creating a private branch so I can test and verify cross-platform build
Revision 333 - Directory Listing
Modified Wed Dec 7 08:00:54 2005 UTC (17 years, 3 months ago) by phornby 
Getting to build on our altix to see if any of the source files have been broken by the windows changes.
Revision 332 - Directory Listing
Modified Wed Dec 7 07:50:18 2005 UTC (17 years, 3 months ago) by phornby 
Split the include and lib dirs for python & boost.
Added the altix support to SConstruct & tidied the options handling a bit.
Revision 305 - Directory Listing
Modified Sat Dec 3 05:05:25 2005 UTC (17 years, 3 months ago) by robwdcock 
+ Added default targets and test target aliases to control what is built better.
Revision 240 - Directory Listing
Modified Mon Nov 28 06:09:41 2005 UTC (17 years, 4 months ago) by robwdcock 
+ Fixed problem with build paths for python tests being in the wrong location
+ Added build & run for python unit tests for escript and finley.

NOTES
+ scons defaults to building ALL targets so if will build everything, including running all tests. This behaviour can be modified by specifying the DEFAULT targets but I've not done this yet. In the interim specify the target you want.
+ Some dependencies need to be specified as scons does not figure them out automatically, though a helper scanner might be able to. These can be specified explicitly using scons Depend function. Not yet done so you can get an invalid build (especially in parallel). Examples include python depends on shared libraries, possibly a couple of library dependencies for install targets - need to verify.
Revision 234 - Directory Listing
Modified Sun Nov 27 05:34:35 2005 UTC (17 years, 4 months ago) by robwdcock 
Paso -> paso  - why do these changes just not stick!!!!
Revision 233 - Directory Listing
Modified Sun Nov 27 05:15:46 2005 UTC (17 years, 4 months ago) by robwdcock 
fixed case for Paso/ includes to be paso/
Revision 232 - Directory Listing
Modified Fri Nov 25 09:59:48 2005 UTC (17 years, 4 months ago) by phornby 
1) Rob, could you please set up your editor to not put CR's into the files.
2) The import esys.bruce fails on Linux, but the rest seems OK.

Here is how things go.

If a directory (bruce say) contains an __init__.py, then import esys.bruce actually imports
the __init__.py file. One can either phoney up the namespace by then saying

from bruce import *

in __init__.py (ugly!!) or just rename bruce.py to __init__.py.

All the sub-packages need this treatment. I suggest doing it with the .py files in the
source tree rather than re-naming at build time. Any other modules in the sub-directory
can then be imported into the __init__.py in the usual way in order to make them visible
to the user. Alternatively, any module not imported to __init__.py is invisible to the user.
This is much better than mucking with PYTHON_PATH and LD_LIBRARY_PATH, which has a tendency
to make everything visible, and creates module name clashes in site-packages on occasions. 
The .so associated with a sub-directory should reside with the corresponding __init__.py
that imports it.

Careful use of "from AClass import AClass" and "from ASharedLib import *" in __init__.py
is also a good method of making classes contained in submodules
look like they are in the namespace of the __init__.py, thereby hiding
the intervening module name from the user. That way esys.bruce.AClass.AClass can be morphed
into esys.bruce.AClass, eliminating the annoying intervening module namespace.
Revision 231 - Directory Listing
Modified Fri Nov 25 09:15:24 2005 UTC (17 years, 4 months ago) by phornby 
Case sensitive file names.
Revision 230 - Directory Listing
Modified Fri Nov 25 06:51:36 2005 UTC (17 years, 4 months ago) by robwdcock 
+ Added ability to RunPyUnitTests when anything they depend on changes (assuming all dependencies are set up that is). Only implemented for bruce/test/python at this stage
Revision 229 - Directory Listing
Modified Fri Nov 25 06:17:13 2005 UTC (17 years, 4 months ago) by robwdcock 
+ Added Options capability
+ All py files are now compiled and installed into lib directory hierarchy
+ scons now reads custom options from custom.py. Examples are in the scons_custom_templates directory, just copy to top level build dir and rename to custom.py
+ scons_ext.py now has some helper functions for building pyc from python and running unit tests (but only when some dependency changes).
Revision 222 - Directory Listing
Modified Thu Nov 24 12:54:42 2005 UTC (17 years, 4 months ago) by robwdcock 
+Unified with linux build (tested on windows only at this point)
Revision 221 - Directory Listing
Modified Thu Nov 24 11:23:33 2005 UTC (17 years, 4 months ago) by phornby 
Builds on Linux. Changes to path names in top level
SConstruct will break the windows build. I've done my best
to not change anything else that would break windows, but
something might have slipped past.

Please check the finley_UnitTests.cpp, bruce_UnitTests.cpp and
escript_UnitTest.cpp (look at the TODO).
Revision 220 - Directory Listing
Modified Thu Nov 24 09:11:38 2005 UTC (17 years, 4 months ago) by robwdcock 
Python compile builder function added to scons. Bruce sconscript file shows usage
Revision 219 - Directory Listing
Modified Thu Nov 24 08:21:21 2005 UTC (17 years, 4 months ago) by phornby 
Case matters on Linux
Revision 218 - Directory Listing
Modified Thu Nov 24 07:42:12 2005 UTC (17 years, 4 months ago) by robwdcock 
All C++/C Unit tests now build (using scons) and pass on win32
Revision 213 - Directory Listing
Modified Thu Nov 24 04:18:40 2005 UTC (17 years, 4 months ago) by robwdcock 
SystemMatrixAdapterTestCase is broken and will not compile. Looks like it hasn't been updated to use Paso_SystemMatrix (used to be Finley_SystemMatrix?). I don't know the history so I'm moving the test out of the way for the moment.
Revision 212 - Directory Listing
Modified Thu Nov 24 04:16:33 2005 UTC (17 years, 4 months ago) by robwdcock 
Reorganise the unit test source tree and how it operates. All tests in the suite are now compiled into a single executable. If for some reason you want to run just one then use the following syntax on the command line:
finley_UnitTests --suite=Atest,AnotherTest,YetAnotherTest,...
Revision 210 - Directory Listing
Modified Wed Nov 23 09:54:02 2005 UTC (17 years, 4 months ago) by robwdcock 
PARTIAL WIN32 BUILD SYSTEM AND PORT
+ All libraries now build under new build system. No unit test routines yet
+ Reversed some incorrect refactorings in Bruce.cpp
Revision 209 - Directory Listing
Modified Wed Nov 23 06:32:25 2005 UTC (17 years, 4 months ago) by robwdcock 
PARTIAL WIN32 BUILD SYSTEM AND PORT
+ bruce, escript build system and library now ported
+ other no longer necessary directories removed from this branches subversion repository
Revision 200 - Directory Listing
Modified Tue Nov 22 07:55:09 2005 UTC (17 years, 4 months ago) by robwdcock 
PARTIAL WIN32 BUILD SYSTEM AND PORT
+ CppUnitTest build system and library now ported
+ make directory and other no longer necessary directories removed from this branches subversion repository
+ moved a few include files so they are now resident with sources. Build system will Install these to the appropriate location during build.
Revision 194 - Directory Listing
Modified Mon Nov 21 08:39:19 2005 UTC (17 years, 4 months ago) by phornby 
Remove hanging }
Revision 186 - Directory Listing
Modified Sun Nov 20 10:51:16 2005 UTC (17 years, 4 months ago) by robwdcock 
PARTIAL WIN32 PORT
+ Paso, ext_math and mmio libraries build using scons in top level directory
+ Scons build system currently hard coded against RW's third-party library paths.
+ Still lots TODO to complete port and build system
Revision 185 - Directory Listing
Modified Sun Nov 20 05:43:12 2005 UTC (17 years, 4 months ago) by robwdcock 
Robert Woodcock private branch for win32 port. Will eventually be merged with trunk when build systems are unified.
Revision 1 - Directory Listing
Added Wed Nov 2 04:33:37 2005 UTC (17 years, 4 months ago) by jgs 
add branches/tags/trunk directories to esys13 repository
  ViewVC Help
Powered by ViewVC 1.1.26