Fetching maven relocation issues is flaky, leading to missing relocations #1165
Comments
|
For io.quarkus:quarkus-webjars-locator there is a relocation information in quarkus-webjars-locator-3.14.2.pom. This is present since quarkus-webjars-locator-3.11.0.pom The last non relocated version is quarkus-webjars-locator-3.10.2.pom As indicated on the page:
Each pom published since
For me the maven API way to check if there is a newer version since For io.quarkus:quarkus-rest I do not see a relocated information in quarkus-rest-3.14.2.pom. |
Yes, that's the bit that fails intermittently in the CI. We just can't fetch the files from maven reliably. We cache to reduce our dependence on the flaky maven APIs, but that also means once we decide a relocation doesn't exist, we cache that incorrect answer. (Fixing that so we only cache correct answers, not incorrect answers, is the intention of this work item.)
The As an example of it working/not-working intermittently, last Wednesday, https://quarkus.io/extensions/io.quarkiverse.reactivemessaging.nats-jetstream/quarkus-reactive-messsaging-nats-jetstream/ gave this:
On Thursday and Friday the relocation information wasn't visible on the live site. I went to take a screencap showing the missing of relocation information today, but the relocation information is back, so I can't screencap the failure case. There's been no code changes in that time, just flaky download endpoints for the information we need, and cache expiries making everything even more non-deterministic. |
|
I see you are doing already a lot and mostly I can't follow what is going on.
I don't know what you are doing, but I can't imagine that fetching the latest version of <dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-webjars-locator</artifactId>
<version>RELEASE</version>
</dependency>can return the Sometimes you can have some caching delay, but not more than 3 months (3.11.0 was released on May 22, 2024) |
|
I did some more investigations that might be interesting… EDIT: to provide some context on my investigations, I did not looked at how this project is doing the lookup of the artifacts in maven central, but I investigated different way how this can be done with other tools I think the major issue is that maven already "resolves" the relocation information. I found online that a The idea is to use the tool to get the latest released version for without having to implement it. 1) getting the effective POM with mavenI do not know how to query the POM file with maven, I thought the effective POM is probably one of the closest artifact I can get With the command: in a project where you have a dummy <project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>g</groupId>
<artifactId>a</artifactId>
<version>1.0-SNAPSHOT</version>
<packaging>pom</packaging>
</project>returns a very long effective pom, but already pointing to the new location: <?xml version="1.0" encoding="UTF-8"?>
<!-- ====================================================================== -->
<!-- -->
<!-- Generated by Maven Help Plugin -->
<!-- See: https://maven.apache.org/plugins/maven-help-plugin/ -->
<!-- -->
<!-- ====================================================================== -->
<!-- ====================================================================== -->
<!-- -->
<!-- Effective POM for project -->
<!-- 'io.quarkus:quarkus-web-dependency-locator:jar:3.14.2' -->
<!-- -->
<!-- ====================================================================== -->
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd" child.project.url.inherit.append.path="false">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-web-dependency-locator-parent</artifactId>
<version>3.14.2</version>
</parent>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-web-dependency-locator</artifactId>
<version>3.14.2</version>
<name>Quarkus - Web Dependency Locator - Runtime</name>
<description>Simplify paths and importmap support for Web dependencies</description>
<url>https://github.com/quarkusio/quarkus</url>
<inceptionYear>2020</inceptionYear>
...==> Basically when you ask for the effective POM of a relocated artifact, you get the effective relocated POM. 2) jbang
|
The longest we have our cache set to hold information for is about 10 days, and that's only for data that we expect not to change at all. The reason the version number is old is that you're looking at the pre-relocation version of the page (you can tell from the URL, which is The "old" page should have a little banner at the top saying 'there's a new version of this jar', with a link to the new ones, and the "new" version should say "this used to have a different name," with a link to the old one. We also would normally only show the "new" page on the front page listing. That's not happening, because the maven endpoint is flaky and was having a bad day when we populated the cache, so we weren't able to get the relevant xml out of maven central. At some point, in a non-deterministic way (or if I improve the caching logic to propagate errors around so it never caches 'error' results), both pages will show what they're supposed to. |
This is where I think I disagree with this analysis.
If you would do a With gradle this is the artifact with coordinates And because you are doing a good job in the quarkus project, you even have a message pointing to the release note or blog post indicating why the artifact was relocated. But you need the |
See, for example, https://quarkus.io/extensions/io.quarkus/quarkus-webjars-locator/ which should be a relocation but today is not. Similarly, https://quarkus.io/extensions/io.quarkus/quarkus-rest/.
We cache to reduce flakiness of the maven API but I think one side effect of this is that errors and missing data persist for longer.
The text was updated successfully, but these errors were encountered: