Opened 6 years ago

Closed 6 years ago

Last modified 6 years ago

#2628 closed user story (fixed)

Ubuntu 14.10 Support

Reported by: GaryM Owned by:
Priority: high Milestone: Iteration I3
Component: infrastructure Keywords:
Cc: Estimated pair-hours remaining: 0
Pair-hours expended on ticket: 8 Initial estimate of effort: 4
Editable by public: yes

Description

Ubuntu 14.10 was released the other day.

I'll try a VM on skip, as my OS is unusually stable I don't fancy taking this upgrade for my whole machine!

Change History (23)

comment:1 Changed 6 years ago by jonc@…

  • Editable by public set

comment:2 Changed 6 years ago by jonc@…

To set up the repository for utopic I did the following on the server:

cd ~/repo
mkdir utopic
cp -a trusty/*deb utopic/
cp trusty/Release.conf utopic/
# Edit utopic/Release.conf to refer to the correct release
cd
./update-chaste-debs

comment:3 Changed 6 years ago by GaryM

I followed InstallGuides/UbuntuPackage developer install instructions. I couldn't get the download with username anonymous as per ChasteGuides/AccessCodeRepository to work, it kept refusing to let me authenticate. So had to check out as GaryM.

Then my VM isn't picking up 'ubuntu.py' because VMWare tools install wasn't set up for Ubuntu 14.10 correctly yet, so I did a manual install exactly as described on a youtube video and found this mess left in /etc/issue:

garmir@ubuntu:~$ more /etc/issue
************************************************
************************************************
 VMware Easy Install

 PLEASE WAIT! VMware Tools is currently being
 installed on your system. Depending on the
 version of Ubuntu you are installing, you may
 log in below and use the system during the
 installation. Otherwise, please wait for the
 graphical environment to launch. Thank you.

************************************************
************************************************

So I got rid of that and put "Ubuntu 14.10" in there, then global built fine, gives this in the build info:

        <Libraries>
		<CompiledIn>
			<PETSc>3.4.2</PETSc>
			<Boost>1.55.0</Boost>
			<HDF5>1.8.12</HDF5>
			<Parmetis>3.1</Parmetis>
		</CompiledIn>
		<Binaries>
			<XSD>4.0.0</XSD>
		</Binaries>
		<Optional>
			<VTK>5.8</VTK>
			<SUNDIALS>2.5.0</SUNDIALS> <!-- includes Cvode of a different version number --> 
		</Optional>
	</Libraries>

heart: build fails and gives:

                 from heart/src/problem/HeartConfig.hpp:47,
                 from heart/src/postprocessing/PostProcessingWriter.cpp:37:
/usr/include/xsd/cxx/xml/elements.hxx:14:4: error: #error Xerces-C++ 2-series is not supported
 #  error Xerces-C++ 2-series is not supported
    ^

followed by various disasters involving XSD and xerces. Presumably this is because XSD has gone from version 3.3.0 in trusty to 4.0.0 in utopic.

cell_based: Even more fun, I got:

g++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-4.9/README.Bugs> for instructions.

comment:4 follow-up: Changed 6 years ago by jonc@…

  • Pair-hours expended on ticket changed from 0 to 1.5

r23154 updates the package metadata to prefer Xerces 3 if available, and r23157 makes the default hostconfig link against it if installed. The updated chaste-dependencies package is now available in the beta repository. This seems to be allowing heart to compile on my VM; it hasn't got as far as cell_based yet.

comment:5 in reply to: ↑ 4 Changed 6 years ago by GaryM

Replying to jonc@…:

I tried with a fresh 14.10 VM and got this:

scons cl=1 global/test/TestChasteBuildInfo.hpp 
scons: Reading SConscript files ...
dpkg-query: no packages found matching libxerces-c-dev
CalledProcessError: Command '['dpkg-query', '-W', '-f', '${version}', 'libxerces-c-dev']' returned non-zero exit status 1:
  File "/home/garmir/Chaste/SConstruct", line 79:
    import hostconfig
  File "python/hostconfig/hostconfig.py", line 102:
    conf = imp.load_module(distro, file, pathname, desc)
  File "python/hostconfig/ubuntu.py", line 106:
    if subprocess.check_output(['dpkg-query', '-W', '-f', '${version}', 'libxerces-c-dev']):
  File "/usr/lib/python2.7/subprocess.py", line 573:
    raise CalledProcessError(retcode, cmd, output=output)

comment:6 Changed 6 years ago by jonc@…

Eek, I hadn't realised it gave a non-zero exit code if the package wasn't installed. Will fix that asap...

comment:7 Changed 6 years ago by jonc@…

Try with r23161.

comment:8 Changed 6 years ago by GaryM

That allows it to build, but you get a message that might concern people:

scons cl=1 heart
scons: Reading SConscript files ...
dpkg-query: no packages found matching libxerces-c-dev
Using hostconfig settings from python/hostconfig/ubuntu.py

can this be piped into the ether or something?

I still get a build failure in XSD:

In file included from /usr/include/xsd/cxx/tree/elements.hxx:41:0,
                 from heart/build/debug/src/io/ChasteParameters_3_3.hpp:64,
                 from heart/src/problem/HeartConfig.hpp:47,
                 from heart/src/tissue/AbstractCardiacTissue.hpp:59,
                 from heart/src/problem/AbstractCardiacProblem.hpp:59,
                 from heart/src/problem/MonodomainProblem.hpp:43,
                 from heart/src/problem/CardiacElectroMechanicsProblem.hpp:45,
                 from ./heart/test/TestAbstractContractionCellFactory.hpp:45,
                 from heart/build/debug/TestAbstractContractionCellFactoryRunner.cpp:21:
/usr/include/xsd/cxx/xml/elements.hxx:14:4: error: #error Xerces-C++ 2-series is not supported
 #  error Xerces-C++ 2-series is not supported
    ^
In file included from heart/build/debug/src/io/ChasteParameters_3_3.hpp:64:0,
                 from heart/src/problem/HeartConfig.hpp:47,
                 from heart/src/tissue/AbstractCardiacTissue.hpp:59,
                 from heart/src/problem/AbstractCardiacProblem.hpp:59,
                 from heart/src/problem/MonodomainProblem.hpp:43,
                 from heart/src/problem/CardiacElectroMechanicsProblem.hpp:45,
                 from ./heart/test/TestAbstractContractionCellFactory.hpp:45,
                 from heart/build/debug/TestAbstractContractionCellFactoryRunner.cpp:21:
/usr/include/xsd/cxx/tree/elements.hxx:51:4: error: #error Xerces-C++ 2-series is not supported
 #  error Xerces-C++ 2-series is not supported
    ^
In file included from /usr/include/xsd/cxx/tree/types.hxx:3941:0,
                 from heart/build/debug/src/io/ChasteParameters_3_3.hpp:65,
                 from heart/src/problem/HeartConfig.hpp:47,
                 from heart/src/tissue/AbstractCardiacTissue.hpp:59,
                 from heart/src/problem/AbstractCardiacProblem.hpp:59,
                 from heart/src/problem/MonodomainProblem.hpp:43,
                 from heart/src/problem/CardiacElectroMechanicsProblem.hpp:45,
                 from ./heart/test/TestAbstractContractionCellFactory.hpp:45,
                 from heart/build/debug/TestAbstractContractionCellFactoryRunner.cpp:21:
/usr/include/xsd/cxx/tree/types.txx: In member function 'std::basic_string<_CharT> xsd::cxx::tree::base64_binary<C, B>::encode() const':
/usr/include/xsd/cxx/tree/types.txx:337:16: error: no matching function for call to 'xercesc_2_8::Base64::encode(const XMLByte*&, XMLSize_t&, XMLSize_t*&, xsd::cxx::xml::std_memory_manager*&)'
             &mm),
                ^
/usr/include/xsd/cxx/tree/types.txx:337:16: note: candidate is:
In file included from /usr/include/xsd/cxx/tree/types.txx:5:0,
                 from /usr/include/xsd/cxx/tree/types.hxx:3941,
                 from heart/build/debug/src/io/ChasteParameters_3_3.hpp:65,
                 from heart/src/problem/HeartConfig.hpp:47,
                 from heart/src/tissue/AbstractCardiacTissue.hpp:59,
                 from heart/src/problem/AbstractCardiacProblem.hpp:59,
                 from heart/src/problem/MonodomainProblem.hpp:43,
                 from heart/src/problem/CardiacElectroMechanicsProblem.hpp:45,
                 from ./heart/test/TestAbstractContractionCellFactory.hpp:45,
                 from heart/build/debug/TestAbstractContractionCellFactoryRunner.cpp:21:
/usr/include/xercesc/util/Base64.hpp:70:21: note: static XMLByte* xercesc_2_8::Base64::encode(const XMLByte*, unsigned int, unsigned int*, xercesc_2_8::MemoryManager*)
     static XMLByte* encode(const XMLByte* const inputData
                     ^
/usr/include/xercesc/util/Base64.hpp:70:21: note:   no known conversion for argument 3 from 'XMLSize_t* {aka long unsigned int*}' to 'unsigned int*'
In file included from /usr/include/xsd/cxx/tree/types.hxx:3941:0,
                 from heart/build/debug/src/io/ChasteParameters_3_3.hpp:65,
                 from heart/src/problem/HeartConfig.hpp:47,
                 from heart/src/tissue/AbstractCardiacTissue.hpp:59,
                 from heart/src/problem/AbstractCardiacProblem.hpp:59,
                 from heart/src/problem/MonodomainProblem.hpp:43,
                 from heart/src/problem/CardiacElectroMechanicsProblem.hpp:45,
                 from ./heart/test/TestAbstractContractionCellFactory.hpp:45,
                 from heart/build/debug/TestAbstractContractionCellFactoryRunner.cpp:21:
/usr/include/xsd/cxx/tree/types.txx: In member function 'void xsd::cxx::tree::base64_binary<C, B>::decode(const XMLCh*)':
/usr/include/xsd/cxx/tree/types.txx:370:73: error: no matching function for call to 'xercesc_2_8::Base64::decodeToXMLByte(const XMLCh*&, XMLSize_t*&, xsd::cxx::xml::std_memory_manager*&, xercesc_2_8::Base64::Conformance&)'
           Base64::decodeToXMLByte (src, &size, &mm, Base64::Conf_RFC2045),
                                                                         ^
/usr/include/xsd/cxx/tree/types.txx:370:73: note: candidate is:
In file included from /usr/include/xsd/cxx/tree/types.txx:5:0,
                 from /usr/include/xsd/cxx/tree/types.hxx:3941,
                 from heart/build/debug/src/io/ChasteParameters_3_3.hpp:65,
                 from heart/src/problem/HeartConfig.hpp:47,
                 from heart/src/tissue/AbstractCardiacTissue.hpp:59,
                 from heart/src/problem/AbstractCardiacProblem.hpp:59,
                 from heart/src/problem/MonodomainProblem.hpp:43,
                 from heart/src/problem/CardiacElectroMechanicsProblem.hpp:45,
                 from ./heart/test/TestAbstractContractionCellFactory.hpp:45,
                 from heart/build/debug/TestAbstractContractionCellFactoryRunner.cpp:21:
/usr/include/xercesc/util/Base64.hpp:157:21: note: static XMLByte* xercesc_2_8::Base64::decodeToXMLByte(const XMLCh*, unsigned int*, xercesc_2_8::MemoryManager*, xercesc_2_8::Base64::Conformance)
     static XMLByte* decodeToXMLByte(
                     ^
/usr/include/xercesc/util/Base64.hpp:157:21: note:   no known conversion for argument 2 from 'XMLSize_t* {aka long unsigned int*}' to 'unsigned int*'
In file included from /usr/include/xsd/cxx/xml/dom/serialization-header.hxx:78:0,
                 from heart/build/debug/src/io/ChasteParameters_3_3.hpp:85,
                 from heart/src/problem/HeartConfig.hpp:47,
                 from heart/src/tissue/AbstractCardiacTissue.hpp:59,
                 from heart/src/problem/AbstractCardiacProblem.hpp:59,
                 from heart/src/problem/MonodomainProblem.hpp:43,
                 from heart/src/problem/CardiacElectroMechanicsProblem.hpp:45,
                 from ./heart/test/TestAbstractContractionCellFactory.hpp:45,
                 from heart/build/debug/TestAbstractContractionCellFactoryRunner.cpp:21:
/usr/include/xsd/cxx/xml/dom/serialization-header.txx: In function 'std::basic_string<_CharT> xsd::cxx::xml::dom::prefix(const C*, xercesc_2_8::DOMElement&, const C*)':
/usr/include/xsd/cxx/xml/dom/serialization-header.txx:35:29: error: 'class xercesc_2_8::DOMElement' has no member named 'lookupPrefix'
           const XMLCh* p (e.lookupPrefix (xns.c_str ()));
                             ^
scons: *** [heart/build/debug/TestAbstractContractionCellFactoryRunner.o] Error 1
scons: building terminated because of errors.

comment:9 Changed 6 years ago by jonc@…

r23163 forces the use of Xerces 3, and a new package is now available in beta.

comment:10 Changed 6 years ago by GaryM

Jon - I included the beta package in the /etc/apt/sources... and now heart appears to be building successfully. I'm a bit confused as to whether it would work without the beta package, as that's not in the instructions for 'normal' code developers who would be working with trunk.

comment:11 Changed 6 years ago by jonc@…

It won't work yet without the beta package, but once we confirm that the beta is working we can put a version in the utopic repo; possibly we can do that now? Obviously release 3.2 won't (ever) work, but 3.3 should be out soon :)

comment:12 Changed 6 years ago by GaryM

Ah, OK. Well yes it seems to work compiling heart now, apart from one problem - the newer versions of xerces seems to have taken blank lines out of generated XML, and this causes TestMonodomainProblem.hpp to fail!

Still have errors when compiling cell-based, possibly related to a gcc 4.9 bug.

comment:13 Changed 6 years ago by GaryM

  • Pair-hours expended on ticket changed from 1.5 to 4

comment:14 Changed 6 years ago by jonc@…

  • Estimated pair-hours remaining changed from 4 to 2
  • Pair-hours expended on ticket changed from 4 to 5

I've just managed to compile cell_based and crypt on a VM, and all tests pass. Which means apart from the TestMonodomainProblem.hpp failure mentioned above (due to XSD) all continuous tests in release code pass. So with release 3.3 (hopefully next week??) I think we can release the 14.10 packaging.

Possibly the compile failures Gary saw were related to running out of memory? Certainly I needed to limit myself to -j2 even on a 6-core VM.

Main todo is adding XSD 4 to the lofty cycle and fixing TestMonodomainProblem.hpp for it.

comment:15 Changed 6 years ago by jonc@…

  • Estimated pair-hours remaining changed from 2 to 0.25
  • Pair-hours expended on ticket changed from 5 to 6

r23318 fixes the failing test, and r23319 added XSD 4 to the lofty cycle. Assuming all goes well with that, I think we're done!

comment:16 Changed 6 years ago by jonc@…

  • Estimated pair-hours remaining changed from 0.25 to 2

We had some lofty build failures due to scons not re-linking/compiling for the Xerces library change (from 2.8 to 3.1). I've removed the stale libraries and .if files which will hopefully sort that. However in my first check to see if it's fixed I discovered that XSD 2.3 doesn't support Xerces 3.1, so we'll need to allow the hostconfig to switch Xerces versions too!

comment:17 Changed 6 years ago by jonc@…

It seems XSD 3.1 also fails to compile with Xerces 3.1; see e.g. this log or this log.

comment:18 Changed 6 years ago by jonc@…

  • Pair-hours expended on ticket changed from 6 to 7

r23323 and r23325 get lofty using Xerces 2.8 for all pre-4 versions of XSD. Unfortunately this means the filter for a particular build type will be different before and after this change, but it looks like it'll fix the build issues. We will need to check whether this will cause issues with making the Ubuntu package force use of Xerces 3 on all distros - will a Lucid, Precise or Trusty users upgrading to Chaste 3.3 see failing tests/builds? (Note that those 3 LTS releases are now the only supported versions apart from Utopic.) We didn't spot issues in brief testing with a trusty VM, but need to investigate further, especially since userpc59 is a lucid machine with system Xerces 3.1!

comment:19 Changed 6 years ago by jonc@…

As a nicety, it would be handy to define a CHASTE_HAS_XERCES flag that can be checked in ExecutableSupport so we can print out the Xerces version used when available.

comment:20 Changed 6 years ago by jonc@…

  • Estimated pair-hours remaining changed from 2 to 1
  • Pair-hours expended on ticket changed from 7 to 7.5

Looks like XSD starts working with Xerces 3.1 as of release 3.2, which makes sense since that's the version packaged in Lucid, which depends on Xerces 3.1. So we should be fine for all Ubuntu releases to force Xerces 3.1. Updated lofty to use more variety for Xerces in r23333.

comment:21 Changed 6 years ago by jonc@…

  • Estimated pair-hours remaining changed from 1 to 0
  • Pair-hours expended on ticket changed from 7.5 to 8
  • Resolution set to fixed
  • Status changed from new to closed

Added Xerces version reporting in r23336. We should double-check that the new packages work on all supported Ubuntus as part of releasing 3.3, but I think Utopic support is working fully in trunk, so this ticket is complete.

comment:22 Changed 6 years ago by GaryM

er...

garmir@skip:~/workspace/Chaste$ scons 
scons: Reading SConscript files ...
Using hostconfig settings from python/hostconfig/machines/gary.py
AttributeError: 'list' object has no attribute 'startswith':
  File "/home/garmir/workspace/Chaste/SConstruct", line 320:
    env.Append(CPPDEFINES = hostconfig.CppDefines() + ['TRILIBRARY', 'TETLIBRARY', 'ANSI_DECLARATORS'])
  File "python/hostconfig/hostconfig.py", line 497:
    return OptionalLibraryDefines()
  File "python/hostconfig/hostconfig.py", line 419:
    if lib.startswith('xerces-c'):

comment:23 Changed 6 years ago by jonc@…

Issue fixed in r23337.

Note: See TracTickets for help on using tickets.