WIP: R 3 2 2 and cairo #312

mvdbeek · 2015-10-09T16:19:32Z

As discussed in issue #304, here is a new version of R, package_cairo and package_r_cairo, as well as some changes in the exposed include and lib paths to get this working.

…version that was not compiled with cairo.

Conflicts: packages/package_cairo_1_14_2/.shed.yml packages/package_cairo_1_14_2/tool_dependencies.xml

bgruening · 2015-10-09T16:24:35Z

packages/package_cairo_1_14_2/tool_dependencies.xml

@@ -31,7 +31,7 @@
                    </repository>
                </action>
                <action type="shell_command">
-                    export LDFLAGS="-Wl,-rpath,$PIXMAN_LIB_PATH -Wl,-rpath,$LIBPNG_ROOT/lib -Wl,-rpath,$FREETYPE_LIB_PATH" &amp;&amp; \ 
+                    export LDFLAGS="-Wl,-rpath,$PIXMAN_LIB_PATH -Wl,-rpath,$LIBPNG_ROOT/lib -Wl,-rpath,$FREETYPE_LIB_PATH" &amp;&amp; \


\ at the end should be removed.

yhoogstrate · 2015-10-09T16:28:39Z

http://downloads.sourceforge.net/project/libpng/libpng16/older-releases/1.6.7/libpng-1.6.7.tar.gz
is can be downloaded via https

edit: WHOOPS not! wget forwarded me !

yhoogstrate · 2015-10-09T16:29:48Z

packages/package_libpng_1_6_7/tool_dependencies.xml

@@ -5,7 +5,7 @@
          <actions>
              <action type="download_by_url">http://downloads.sourceforge.net/project/libpng/libpng16/older-releases/1.6.7/libpng-1.6.7.tar.gz</action>
                <action type="shell_command">./configure --prefix=$INSTALL_DIR</action>
-                <action type="shell_command">make &amp;&amp; make install</action>
+                <action type="make_install" />


I think you can reduce it even further:
<action type="autoconf">--prefix=$INSTALL_DIR</action>

mvdbeek · 2015-10-09T16:30:28Z

@yhoogstrate OK, I guess I should also add md5 sums?

bgruening · 2015-10-09T16:31:43Z

Let's wait with merging until we have the binaries.

mvdbeek · 2015-10-09T16:45:57Z

@yhoogstrate I don't think that forwarding would be a problem for urllib2, which is used for downloading. Also we got the shasum. I'll go ahead with https?

…ine continuation for cairo. Adjust binary R download link.

davebx · 2015-10-09T16:56:49Z

@bgruening @mvdbeek https://depot.galaxyproject.org/package/linux/x86_64/R/R-3.2.2-Linux-x86_64.tar.gz

yhoogstrate · 2015-10-09T16:57:53Z

The URL gets redirected to HTTP because HTTPS seems not to work. So I would not go for HTTPS url. Forwarding to HTTP is really not something you want, because that may introduce MITM attacks.

SHA256SUMS are really good because they confirm whether the transfer completed correctly, source code hasn't been modified over time and no MITM attack has taken place.

bgruening · 2015-10-09T16:58:06Z

super fast @davebx!

bgruening · 2015-10-10T15:59:14Z

packages/package_cairo_1_14_2/tool_dependencies.xml

@@ -31,7 +31,7 @@
                    </repository>
                </action>
                <action type="shell_command">
-                    export LDFLAGS="-Wl,-rpath,$PIXMAN_LIB_PATH -Wl,-rpath,$LIBPNG_ROOT/lib -Wl,-rpath,$FREETYPE_LIB_PATH" &amp;&amp; \ 
+                    export LDFLAGS="-Wl,-rpath,$PIXMAN_LIB_PATH -Wl,-rpath,$LIBPNG_ROOT/lib -Wl,-rpath,$FREETYPE_LIB_PATH"


We do need the && and the end isn't it? We could also use CDATA here.

CDATA definitely makes sense, the && is not needed, but can be added back. It depends if you prefer one or 2 issued commands, If you prefer one command I'll also need to add back the line continuation character \.

As you like :) But this doesn't seem to work currently.

My recommendation would be && and no \

OK, no problem. Still stuck with #312 (comment) though.

mvdbeek · 2015-10-11T14:50:16Z

@bgruening would you mind taking a look at this pixman tool_dependencies.xml?
https://gist.github.com/mvdbeek/9fdf008c010435087f4e
It is now not receiving the environmental variables defined by libpng ...
I have been scratching my head for a while on this one now

bgruening · 2015-10-11T17:05:58Z

It this somewhere in the TS. No obvious mistake that I can see.

bgruening · 2015-10-12T15:45:03Z

@mvdbeek can you updload it in some test account in the TS. I will have a look at this in the evening. Just need a clean docker instance.

mvdbeek · 2015-10-12T16:05:36Z

@bgruening
This one is from the gist, and doesn't receive the LIBPNG_LIB_PATH:
https://testtoolshed.g2.bx.psu.edu/view/mvdbeek/package_pixman_0_32_6_actions_group_test/e7dbececd3d0
This one has the uname stuff to get the OS, and correctly recevices LIBPNG_LIB_PATH:
https://testtoolshed.g2.bx.psu.edu/view/mvdbeek/package_pixman_0_32_6_osx_bash/461e0b3b76b4
Don't be fooled if both install correctly. When you check the INSTALL.log, you should see

PATH=/home/marius/src/galaxy/tool-dependencies/libpng/1.6.7/iuc/package_libpng_1_6_7/ac6ef08bf0e6/bin:$PATH; export PATH
LIBPNG_ROOT=/home/marius/src/galaxy/tool-dependencies/libpng/1.6.7/iuc/package_libpng_1_6_7/ac6ef08bf0e6; export LIBPNG_ROOT
LIBPNG_LIB_PATH=/home/marius/src/galaxy/tool-dependencies/libpng/1.6.7/iuc/package_libpng_1_6_7/ac6ef08bf0e6/lib; export LIBPNG_LIB_PATH
LIBPNG_INCLUDE_PATH=/home/marius/src/galaxy/tool-dependencies/libpng/1.6.7/iuc/package_libpng_1_6_7/ac6ef08bf0e6/include; export LIBPNG_INCLUDE_PATH
LD_LIBRARY_PATH=/home/marius/src/galaxy/tool-dependencies/libpng/1.6.7/iuc/package_libpng_1_6_7/ac6ef08bf0e6/lib:$LD_LIBRARY_PATH; export LD_LIBRARY_PATH
PKG_CONFIG_PATH=/home/marius/src/galaxy/tool-dependencies/libpng/1.6.7/iuc/package_libpng_1_6_7/ac6ef08bf0e6/lib/pkgconfig:$PKG_CONFIG_PATH; export PKG_CONFIG_PATH
LIBPNG_ROOT_DIR=/home/marius/src/galaxy/tool-dependencies/libpng/1.6.7/iuc/package_libpng_1_6_7/ac6ef08bf0e6; export LIBPNG_ROOT_DIR
...

This doesn't happen with the first repo.

yhoogstrate · 2015-10-12T16:25:45Z

@mvdbeek In the examples I've seen so far <action type="set_environment"> is not within a <actions ...> element. Is it possible this has something to do with it?

mvdbeek · 2015-10-12T16:30:20Z

I don't think so (see https://github.com/galaxyproject/tools-iuc/blob/master/packages/package_r_3_2_1/tool_dependencies.xml).
Anyway, the problematic action is <action type="set_environment_for_install">, which populates the environmental variables during the install process.
It looks like <action type="set_environment_for_install"> doesn't work within actions_group tags to me

bgruening · 2015-10-12T16:50:08Z

@mvdbeek Oh I see! How did you updated the repository. The toolshed and the revisions are missing. This should not happen and is a bug if you uploaded it with planemo.

mvdbeek · 2015-10-12T16:55:20Z

I did upload with planemo ... you think it's a bug in the toolshed, right?
That does explain the things. i never bothered to actually check in the toolshed :D

bgruening · 2015-10-12T18:41:34Z

@mvdbeek actually I thought this was fixed in galaxyproject/galaxy#637
This makes me wonder how old the TTS is :(

martenson · 2015-10-12T19:22:09Z

I updated the TTS
you can see how old is the deployed TTS version here: https://github.com/galaxyproject/usegalaxy-playbook/commits/master/stage/group_vars/toolshedservers.yml

bgruening · 2015-10-12T19:25:32Z

Thanks @martenson. Hopefully this fixes @mvdbeek issue.

mvdbeek · 2015-10-13T06:33:05Z

It does, thx @martenson @bgruening

mvdbeek · 2015-10-16T14:08:13Z

I'm closing this PR in favor of more incremental changes:

PR Fix cairo cflags #337
PR Fix pixman compilation on OS X. #336
and finally r_3_2_2, but that needs more testing still.

mvdbeek added 3 commits October 9, 2015 17:46

Fix passing of enviromental variables to properly build cairo on a R …

1730cfe

…version that was not compiled with cairo.

Merge remote-tracking branch 'upstream/master' into r_3_2_2_and_cairo

a5434e0

Conflicts: packages/package_cairo_1_14_2/.shed.yml packages/package_cairo_1_14_2/tool_dependencies.xml

Remove emmpty newline and change set_to to prepend_to.

64756ee

bgruening reviewed Oct 9, 2015
View reviewed changes

yhoogstrate reviewed Oct 9, 2015
View reviewed changes

Use autoconf, download through https and add shasum for libpng. Fix l…

b89a7fb

…ine continuation for cairo. Adjust binary R download link.

mvdbeek added 4 commits October 9, 2015 21:47

Fix pixman on OSX.

8e96bb0

Remove non-existing reference to PNG_INCLUDES env var.

49c11e2

Include freetype in dependencies.

374bfa2

Uncomment binary R package.

da674f4

bgruening reviewed Oct 10, 2015
View reviewed changes

mvdbeek changed the title ~~R 3 2 2 and cairo~~ WIP: R 3 2 2 and cairo Oct 11, 2015

mvdbeek mentioned this pull request Oct 16, 2015

Fix syntax of error parameter of tryCatch in setup_r_environment action. galaxyproject/galaxy#929

Merged

mvdbeek closed this Oct 16, 2015

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

WIP: R 3 2 2 and cairo #312

WIP: R 3 2 2 and cairo #312

mvdbeek commented Oct 9, 2015

bgruening Oct 9, 2015

yhoogstrate commented Oct 9, 2015

yhoogstrate Oct 9, 2015

bgruening Oct 9, 2015

mvdbeek commented Oct 9, 2015

bgruening commented Oct 9, 2015

mvdbeek commented Oct 9, 2015

davebx commented Oct 9, 2015

yhoogstrate commented Oct 9, 2015

bgruening commented Oct 9, 2015

bgruening Oct 10, 2015

mvdbeek Oct 10, 2015

bgruening Oct 10, 2015

davebx Oct 12, 2015

mvdbeek Oct 12, 2015

mvdbeek commented Oct 11, 2015

bgruening commented Oct 11, 2015

bgruening commented Oct 12, 2015

mvdbeek commented Oct 12, 2015

yhoogstrate commented Oct 12, 2015

mvdbeek commented Oct 12, 2015

bgruening commented Oct 12, 2015

mvdbeek commented Oct 12, 2015

bgruening commented Oct 12, 2015

martenson commented Oct 12, 2015

bgruening commented Oct 12, 2015

mvdbeek commented Oct 13, 2015

mvdbeek commented Oct 16, 2015

WIP: R 3 2 2 and cairo #312

WIP: R 3 2 2 and cairo #312

Conversation

mvdbeek commented Oct 9, 2015

Choose a reason for hiding this comment

yhoogstrate commented Oct 9, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mvdbeek commented Oct 9, 2015

bgruening commented Oct 9, 2015

mvdbeek commented Oct 9, 2015

davebx commented Oct 9, 2015

yhoogstrate commented Oct 9, 2015

bgruening commented Oct 9, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mvdbeek commented Oct 11, 2015

bgruening commented Oct 11, 2015

bgruening commented Oct 12, 2015

mvdbeek commented Oct 12, 2015

yhoogstrate commented Oct 12, 2015

mvdbeek commented Oct 12, 2015

bgruening commented Oct 12, 2015

mvdbeek commented Oct 12, 2015

bgruening commented Oct 12, 2015

martenson commented Oct 12, 2015

bgruening commented Oct 12, 2015

mvdbeek commented Oct 13, 2015

mvdbeek commented Oct 16, 2015