Warning: Expected one CSV file...

[vxnuaj]

Hey! Consistently getting the following error when running python3 get_spider2snow_submission_data.py --experiment_suffix gpt-4o-mini-test1 --results_folder_name ../../spider2-snow/evaluation_suite/gpt-4o-mini-test1

Warning: Expected one CSV file in ../../spider2-snow/evaluation_suite/gpt-4o-mini-test1/sf_bq416, found 0
Warning: Expected one CSV file in ../../spider2-snow/evaluation_suite/gpt-4o-mini-test1/sf_bq096, found 0
Warning: Expected one CSV file in ../../spider2-snow/evaluation_suite/gpt-4o-mini-test1/sf_local039, found 0
Warning: Expected one CSV file in ../../spider2-snow/evaluation_suite/gpt-4o-mini-test1/sf_bq444, found 0

...

Warning: Expected one CSV file in ../../spider2-snow/evaluation_suite/gpt-4o-mini-test1/sf_ga014, found 0
Warning: Expected one CSV file in ../../spider2-snow/evaluation_suite/gpt-4o-mini-test1/sf_ga013, found 0
Warning: Expected one CSV file in ../../spider2-snow/evaluation_suite/gpt-4o-mini-test1/sf_bq081, found 0
Warning: Expected one CSV file in ../../spider2-snow/evaluation_suite/gpt-4o-mini-test1/sf_bq203, found 0
Warning: Expected one CSV file in ../../spider2-snow/evaluation_suite/gpt-4o-mini-test1/sf_bq211, found 0

Wondering where these CSV files were supposed to come from?

Pretty sure this is causing erroneous eval of 4o-mini, as

Final score: 0.0, Correct examples: 0, Total examples: 281
Real score: 0.0, Correct examples: 0, Total examples: 547

doesn't seem right.

[vxnuaj]

Update -- Ran it on the example submissions folder, went all smooth -- it seems that the final eval seems to be valid and that 4o-mini just sucks... lmao