Everything should be rebuilt after GCC upgrade

[jdemeyer]

Due to C++ ABI incompatibilities, problems occur if code built with earlier versions of GCC is linked against code built with newer versions of GCC. This leads for example to patchbot failures mentioned below.

The simplest solution is to make every package depend on the GCC executable. This way, a GCC upgrade will force everything to be rebuilt. This may be overkill, but it's not a big problem in practice since GCC is upgraded rarely (unlikely other Sage packages, it is never automatically rebuilt except if absolutely needed, see for example #24599).

---

See attachment: crash_zrVXNw.log for a crash log from the arando i686 patchbot with Sage 8.2.beta5 using GCC 7.2.0. It involves brial and C++ strings.

The following doctests fail:

sage -t --warn-long 67.1 src/sage/rings/polynomial/multi_polynomial_sequence.py  # Killed due to abort
sage -t --warn-long 67.1 src/sage/rings/polynomial/pbori.pyx  # Killed due to abort
sage -t --warn-long 67.1 src/sage/rings/polynomial/multi_polynomial_libsingular.pyx  # Killed due to abort
sage -t --warn-long 67.1 src/sage/rings/polynomial/polynomial_ring_constructor.py  # Killed due to abort

CC: @kiwifb @embray

Component: packages: standard

Author: Jeroen Demeyer

Branch: 70ff832

Reviewer: François Bissey

Issue created by migration from https://trac.sagemath.org/ticket/24703

[jdemeyer]

Attachment: crash_zrVXNw.log

[jdemeyer]

Author: Jeroen Demeyer

[kiwifb]

comment:7

Yup, pretty much anything that use C++ has to be rebuilt when you move to gcc-5. gcc dev have made some commitments to ABI stability for the time being. But then you also have the change of ABI for gfortran at gcc-7. The only thing that's rock solid for almost ever is C.

[jdemeyer]

Branch: u/jdemeyer/everything_should_be_rebuilt_after_gcc_upgrade

[sagetrac-git]

Commit: c59430d

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

c59430d

Everything should be rebuilt after GCC upgrade

[sagetrac-git]

Changed commit from c59430d to 057b434

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

057b434

Everything should be rebuilt after GCC upgrade

[kiwifb]

Reviewer: François Bissey

[kiwifb]

comment:13

Not sure why the patchbot is failing but the stuff looks good me, and we should include it ASAP.

[embray]

comment:14

Replying to @kiwifb:

Not sure why the patchbot is failing but the stuff looks good me, and we should include it ASAP.

I need to upgrade that patchbot. It's running an old version of the patchbot package that has a bug when testing "unsafe" tickets.

[embray]

comment:15

Okay, I don't really care since I'm not in the habit of building gcc anyways, but I would appreciate if someone sign off on #21524 so I don't have to keep upgrading it to incorporate changes like this.

[vbraun]

Changed branch from u/jdemeyer/everything_should_be_rebuilt_after_gcc_upgrade to 057b434

[vbraun]

comment:26

Apparently the build was interleaved with gcc, which shouldn't happen. This also happens after backing out the ticket so its either a preexisting issue or the makefile isn't regenerated. This is the second build of gcc, it builds in parallel with everything else.

[jdemeyer]

comment:27

Replying to @vbraun:

This also happens after backing out the ticket

I think it's a bug in the previous version of this ticket then.

This is the second build of gcc

There shouldn't be a second build of gcc.

It would be very useful to see the auto-generated build/make/Makefile from the failed build to understand what is happening.

[vbraun]

Attachment: Makefile.gz

[vbraun]

comment:29

I attached the makefile...

[vbraun]

comment:30

I checked out just this ticket and ran make distclean && make and gcc builds twice

buildbot@sagebd07_64s02:~/slave/sage_git/build$ grep 'Found local metadata for gcc' logs/install.log 
[gcc-7.2.0] Found local metadata for gcc-7.2.0
[gcc-7.2.0] Found local metadata for gcc-7.2.0

and the second time is in parallel with lots of other stuff...

[jdemeyer]

comment:31

OK, I think I know what happened: there is indeed a serious bug that GCC is installed twice.

But that bug is not because of this ticket, it is most likely happening since #24599. I think you are just unlucky to hit this bug now and not before.

[jdemeyer]

comment:32

I would rather prefer to merge this ticket anyway and then deal with further fixes to the GCC build process in a future ticket.

[vbraun]

comment:33

Sure, did you open a followup ticket?

[vbraun]

Changed branch from 70ff832 to u/jdemeyer/everything_should_be_rebuilt_after_gcc_upgrade

[vbraun]

Commit: 057b434

[vbraun]

New commits:

057b434

Everything should be rebuilt after GCC upgrade

[vbraun]

New commits:

70ff832

Everything should be rebuilt after GCC upgrade

[vbraun]

Changed branch from u/jdemeyer/everything_should_be_rebuilt_after_gcc_upgrade to u/jdemeyer/057b434babb6ec6eaa7bae4afad4255a7481b1af

[vbraun]

Changed commit from 057b434 to 70ff832

[vbraun]

Changed branch from u/jdemeyer/057b434babb6ec6eaa7bae4afad4255a7481b1af to 70ff832

[jhpalmieri]

comment:37

A question related to #24961: should we be checking that x$SAGE_INSTALL_GCC = xexists or that x$SAGE_INSTALL_GCC = xyes? On an OS X machine, I have Sage's gcc installed (from a few betas ago), but now Sage uses clang instead of gcc, so Sage's gcc will not be rebuilt as I upgraded Sage.

[jhpalmieri]

Changed commit from 70ff832 to none

[kiwifb]

comment:38

Replying to @jhpalmieri:

A question related to #24961: should we be checking that x$SAGE_INSTALL_GCC = xexists or that x$SAGE_INSTALL_GCC = xyes? On an OS X machine, I have Sage's gcc installed (from a few betas ago), but now Sage uses clang instead of gcc, so Sage's gcc will not be rebuilt as I upgraded Sage.

Really, you need to check both, with the caveat that if you installed with clang and want to switch to gcc you will break your install. So, unfortunately, a not so stupid user could break things.

[jhpalmieri]

comment:39

In my case, I switched from gcc (the default) to clang (the new default), and upgrading to the most recent beta broke things, because this test ended up adding bad stuff to build/make/Makefile, as reported in sage-release and #21524, and now being fixed in #24961.

[jdemeyer]

comment:40

Replying to @jhpalmieri:

In my case, I switched from gcc (the default) to clang (the new default)

Are you sure? If the GCC spkg is installed, it should be used by Sage. If this is not the case, I consider that a bug.

[jdemeyer]

comment:41

Replying to @kiwifb:

Replying to @jhpalmieri:

A question related to #24961: should we be checking that x$SAGE_INSTALL_GCC = xexists or that x$SAGE_INSTALL_GCC = xyes? On an OS X machine, I have Sage's gcc installed (from a few betas ago), but now Sage uses clang instead of gcc, so Sage's gcc will not be rebuilt as I upgraded Sage.

Really, you need to check both

I don't see why, but maybe I'm missing something?

[jhpalmieri]

comment:42

Replying to @jdemeyer:

Replying to @jhpalmieri:

In my case, I switched from gcc (the default) to clang (the new default)

Are you sure? If the GCC spkg is installed, it should be used by Sage. If this is not the case, I consider that a bug.

I guess you're right, gcc is used, but configure tells me that gcc will not be installed.

[jdemeyer]

comment:43

Replying to @jhpalmieri:

configure tells me that gcc will not be installed.

It won't be installed if it's already installed. Consider that a feature.

[jhpalmieri]

comment:44

I also consider it a feature if it's not installed because clang is going to be used instead. configure does not give a reason for it not being installed, leaving me to guess as to what's going on.

[jdemeyer]

comment:45

Replying to @jhpalmieri:

I also consider it a feature if it's not installed because clang is going to be used instead.

Of course. But that should work now, right?