Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mb/tidy-up #98

Merged
merged 30 commits into from
Feb 11, 2025
Merged

mb/tidy-up #98

merged 30 commits into from
Feb 11, 2025

Conversation

sesquideus
Copy link
Contributor

Tidying up and adding more tests wherever applicable.

@sesquideus sesquideus self-assigned this Feb 6, 2025
@sesquideus
Copy link
Contributor Author

I added test classes, so that every recipe is at least somewhat covered, and ironed out everything that was failing (except those that depended on external data).

Also changed the Product hierarchy to use "private" _attributes and corresponding properties, mostly to be consistent with Inputs.

@sesquideus sesquideus changed the title Mb/tidy up mb/tidy-up Feb 7, 2025
@sesquideus
Copy link
Contributor Author

Now this branch should be up to date with METIS_Pipeline_Test_Datamb/tidy-up. All tests pass/xfail for me locally.

I also refactored the way Products are collected and registered: it is somewhat simpler, with less repetition and hopefully reduces the effective cross-section™ for bugs too.

@sesquideus sesquideus requested a review from chyan26 February 7, 2025 22:52
@sesquideus sesquideus marked this pull request as ready for review February 8, 2025 08:50
Comment on lines +26 to +32
@pytest.fixture(autouse=True, scope="module")
def reset_edps():
def inner():
os.system("edps -shutdown")
os.system("rm -rf /tmp/EDPS_Data/*")

return inner
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not so comfortable with having any os.system("rm -rf code in the repository.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, it's dangerous. Right now I cannot think of a better way of resetting the state though.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yesterday I fixed the bug that the EDPS won't start if you delete your EDPS_Data directory, and I created this test:

https://github.com/AstarVienna/edps/blob/4b77d742c8de65ebde80eedf8c9d0ba822ed172e/test/tests/repository_tests.py#L446

(Created a fork of the EDPS to highlight that.) There an entire temporary EDPS_Data directory is created, and deleted after the test. Perhaps we can do something similar. I don't fully understand what is going on, I just cargo-culted that test.

(Which, if my fix is accepted, would mean that you would not need that /* in this part of the code.)

Maybe we can keep the code as-is for now. But I just wanted to note here that I don't like it. Which means by extension now I'm also on the line if something ever goes wrong with this code..

Comment on lines 85 to 86
edps -w metis.metis_wkf -i $SOF_DATA -c
edps -w metis.metis_wkf -i $SOF_DATA | tee edps.stdout.txt
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should not be necessary to revert this change right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, and it should be OK now.

@hugobuddel
Copy link
Contributor

This branch contains #89 , so should we merge that first? It is hard to disentangle what is going on

@sesquideus
Copy link
Contributor Author

Finally I was able to bring it locally to the same state as on GitHub: EDPS has 21 completed, 3 failing. However I still cannot see the actual source of the problem, EDPS logs are not very verbose.

assert output.returncode == 0
assert output.stderr == b''
assert re.findall('[eE]rror', message) == []
assert re.findall('FAILED', message) == []
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand half of this, so I'm in no position to yay/nay it

Copy link
Contributor

@astronomyk astronomyk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apart from the conversation regarding os.removing the whole EDPS folder - which I don't know enough to comment on - I'm happy with the PR

@sesquideus
Copy link
Contributor Author

I think we're done here. I self-reviewed all the tests and tried to make them as consistent as possible.

@sesquideus sesquideus merged commit ae97870 into main Feb 11, 2025
2 checks passed
@sesquideus sesquideus deleted the mb/tidy-up branch February 11, 2025 21:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants