Investigate failing test tests/remote/test_firefox.py::test_tinderbox_scraper

[salilmishra23]

The test is currently skipped as for some arguments the request timeout (Given enough time it returns HTTP 504).

Logs - https://github.com/mozilla/mozdownload/actions/runs/3041139681/jobs/4897959420#step:5:362

Failing arguments -

{'branch': 'mozilla-central', 'platform': 'linux64'},
{'branch': 'mozilla-central', 'platform': 'win32'},
{'branch': 'mozilla-central', 'platform': 'win64'},

Link to the test -

mozdownload/tests/remote/test_firefox.py

Line 104 in 104a536

@pytest.mark.skip(reason="https://github.com/mozilla/mozdownload/issues/625")

[whimboo]

The problem can be seen as well when manually opening the affected folders:

• https://archive.mozilla.org/pub/firefox/tinderbox-builds/mozilla-central-linux64/
• https://archive.mozilla.org/pub/firefox/tinderbox-builds/mozilla-central-win32/
• https://archive.mozilla.org/pub/firefox/tinderbox-builds/mozilla-central-win64/

It's happening because there are basically too many builds in those directories. And at the end cloudflare fails with a 504 ERROR. As such I would suggest that we use beta or even release tinderbox builds instead for which the number of builds is way lesser.

[whimboo]

I've done some testing locally with the following results:

When using mozilla-beta:

tests/remote/test_firefox.py::test_tinderbox_scraper[args0] PASSED                                                                                            [  2%]
tests/remote/test_firefox.py::test_tinderbox_scraper[args1] PASSED                                                                                            [  4%]
tests/remote/test_firefox.py::test_tinderbox_scraper[args2] PASSED                                                                                            [  6%]
tests/remote/test_firefox.py::test_tinderbox_scraper[args3] PASSED                                                                                            [  8%]
tests/remote/test_firefox.py::test_tinderbox_scraper[args4] FAILED                                                                                            [ 10%]
tests/remote/test_firefox.py::test_tinderbox_scraper[args5] PASSED                                                                                            [ 12%]
tests/remote/test_firefox.py::test_tinderbox_scraper[args6] PASSED                                                                                            [ 14%]
tests/remote/test_firefox.py::test_tinderbox_scraper[args7] FAILED

When using mozilla-release:

tests/remote/test_firefox.py::test_tinderbox_scraper[args0] PASSED                                                                                            [  2%]
tests/remote/test_firefox.py::test_tinderbox_scraper[args1] PASSED                                                                                            [  4%]
tests/remote/test_firefox.py::test_tinderbox_scraper[args2] PASSED                                                                                            [  6%]
tests/remote/test_firefox.py::test_tinderbox_scraper[args3] PASSED                                                                                            [  8%]
tests/remote/test_firefox.py::test_tinderbox_scraper[args4] PASSED                                                                                            [ 10%]
tests/remote/test_firefox.py::test_tinderbox_scraper[args5] PASSED                                                                                            [ 12%]
tests/remote/test_firefox.py::test_tinderbox_scraper[args6] PASSED                                                                                            [ 14%]
tests/remote/test_firefox.py::test_tinderbox_scraper[args7] PASSED

I do not really see a benefit in having to use mozilla-central builds here given that we haven't had a naming change of binaries for a very long time. Also if one would come up we will be informed right in time.

As such I will provide a patch that changes the parameterization of the tests from mozilla-central to mozilla-release.