Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow blessing if test only failed GENERATE #4760

Open
wants to merge 8 commits into
base: master
Choose a base branch
from
+47 −7
Open

Allow blessing if test only failed GENERATE #4760

Changes from 1 commit
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
ab0664e
bless_test_results now allows blessing if test only failed GENERATE.
samsrabin Mar 6, 2025
049f197
Move check for only_failed_generate to minimize testing impact.
samsrabin Mar 6, 2025
89531b3
Resolve StopIterationError in test_is_bless_needed_overall_fail(). No…
samsrabin Mar 6, 2025
31f7ce0
Black-reformat test_unit_bless_test_results.py.
samsrabin Mar 6, 2025
ab66ca0
only_failed_generate bugfix.
samsrabin Mar 6, 2025
11a5553
Bugfix in is_hist_bless_needed().
samsrabin Mar 12, 2025
ae21da1
Fix test_is_bless_needed_overall_fail.
samsrabin Mar 12, 2025
1e0e112
Add test_is_bless_needed_generate_fail.
samsrabin Mar 12, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 13 additions & 1 deletion CIME/bless_test_results.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -489,6 +489,18 @@ def is_hist_bless_needed(

run_result = ts.get_status(RUN_PHASE)

# Sometimes a test might fail only during the generate phase; e.g., if the user doesn't have
# write permissions in the baseline directory. We still want to bless those tests.
if ts.get_status(GENERATE_PHASE) == TEST_FAIL_STATUS:
only_failed_generate = True
for p in ALL_PHASES:
if p == GENERATE_PHASE:
continue
phase_result = ts.get_status(p)
if phase_result is TEST_FAIL_STATUS:
only_failed_generate = False
break

if run_result is None:
broken_blesses.append((test_name, "no run phase"))
logger.warning("Test '{}' did not make it to run phase".format(test_name))
Expand All @@ -501,7 +513,7 @@ def is_hist_bless_needed(
)
)
needed = False
elif overall_result == TEST_FAIL_STATUS:
elif overall_result == TEST_FAIL_STATUS and not only_failed_generate:
broken_blesses.append((test_name, "test did not pass"))
logger.warning(
"Test '{}' did not pass due to phase {}, not safe to bless, test status = {}".format(
Expand Down
Loading