Update all CI scripts

[uilianries]

Description of Problem, Request, or Question

Hi!

Conan 1.8.0 came with some structural changes, including Docker images.

Lasote changed all docker image names to conanio/<compiler><version>

We need some script to update ALL our travis recipes! It's important because lasote's images will not be updated.

[Croydon]

This should be combined to update:

• adding Clang 7
• adding apple-clang 10
• renaming .travis/ -> .ci/
  • the directory plus the references in .travis.yml
• (maybe more I'm not thinking about right now)

[uilianries]

I created a script to update our projects. It includes all points mentioned by @Croydon + Conan conventions (Conan 1.8).

https://github.com/bincrafters/bincrafters-conventions

https://pypi.org/project/bincrafters-conventions/ (0.2.0)

The script applies [outdated] message on commit to avoid unnecessary jobs.

I used the project conan-cunit as experiment:

bincrafters/conan-cunit@2aa644c

The command used by me was:
bincrafters-conventions --remote bincrafters/conan-cunit --branch-pattern "testing/*"

I'll wait for more comments before to run over our projects. I think we could use official packages in Conan center only.

Have a nice weekend!

/cc @solvingj @SSE4 @Croydon

[Croydon]

Excellent work!

If we want to include Clang 7 then we still need to wait until this discussion around the version schema is solved upstream (which I would prefer, otherwise we would have mass updates soon again). conan-io/conan#3643

Maybe until then we also made progress with the Python 2 -> 3 upgrades everywhere (not just in the docker containers) conan-io/conan-docker-tools#53
We could deal with eventual failures because of that across all packages at once then

The only downside I'm seeing in this script: That's now the fourth (or so) place where we have fragments of our templates which needs to be kept up to date.

How is it handling the situation if old compilers are removed because they are unsupported by the project? Are the getting re-added? Would it be complicated to read the latest existing compiler versions from the current CI config files and only add newer ones?

[Croydon]

And how complicated is it to update the default_options to a dict? 🤔

[uilianries]

Excellent observation, I'll update default_options later

[uilianries]

Would it be complicated to read the latest existing compiler versions from the current CI config files and only add newer ones?

That's exactly what I did. I read Travis file, list all compilers and their version, and merge with newest compilers list.

[uilianries]

If we want to include Clang 7 then we still need to wait until this discussion around the version schema is solved upstream (which I would prefer, otherwise we would have mass updates soon again). conan-io/conan#3643

That's true. Maybe we could remove clang 7 for now and add only when finished by discussion.

[SSE4]

so far, summary of changes to be made:

• add Apple Clang 10.0
• add Clang 7 (wait for - use major-only for Clang >= 8.0 conan-io/conan#3643)
• rename .travis to .ci
• update docker image names from lasote/conan<compiler> to conanio/<compiler>
• update python version to 3.7.0 (remove useless variables PYTHON_VERSION & PYTHON_ARCH)
• update ubuntu to xenial
• update default_options to use dict()
• update protected methods and properties (e.g. build_subfolder, sources_subfolder, configure_cmake)
• move MSVC/MinGW builds to travis? (???)

[uilianries]

To update Python3 on Appveyor we should declare both version and arch AFAIK:

https://github.com/pyinstaller/pyinstaller/blob/develop/appveyor.yml
http://tjelvarolsson.com/blog/how-to-continuously-test-your-python-code-on-windows-using-appveyor/

[Croydon]

The pyinstaller example is using these two variables for custom scripts (e.g. https://github.com/pyinstaller/pyinstaller/blob/develop/appveyor.yml#L122) I don't think we need them in any way.

We are also not updating or installing Python or something like that. AppVeyor has via default installed many Python versions including 3.7.0. We just need to set the PATH in a way, that the specific version we want is getting picked up.

[uilianries]

Is PYTHON only enough?

PYTHON: "C:\\Python37"

[Croydon]

I'm pretty sure, yes

bincrafters/templates#37

[uilianries]

I update Bincrafters Conventions ... but I'm waiting for CI

the script looks dirty and a bit complex but works! 🤕

• Apple Clang 10.0
• add Clang 7 (wait for - use major-only for Clang >= 8.0 conan-io/conan#3643)
• rename .travis to .ci
• update docker image names from lasote/conan to conanio/
• update python version to 3.7.0 (remove useless variables PYTHON_VERSION & PYTHON_ARCH)
• update ubuntu to xenial
• update default_options to use dict()
• update protected methods and properties (e.g. build_subfolder, sources_subfolder, configure_cmake)
• support conan ci pages

[Croydon]

Maybe @SSE4 has already implemented this in the meantime, but it should also remove the sudo: required line altogether in Travis. Using the docker service implicits sudo: true and sudo disable environments are soon getting removed altogether from Travis.

[Croydon]

bincrafters-conventions is removing all jobs with something like CONAN_ARCHS=x86 CONAN_DOCKER_32_IMAGES=1

[SSE4]

okay, I think we need to fix case with CONAN_ARCHS=x86 CONAN_DOCKER_32_IMAGES=1, I'll take a look

[SSE4]

I believe it's safe to close now. with bincrafters/bincrafters-conventions#6 we can update all recipes without losing jobs.