Fix issue where Evaluation Results don't load when individual Evaluation Scenarios fail

[jp-agenta]

Description

• Catch None-related issue in Optional[...] field: EvaluationResult.correct_answers
• Adds logger.error(e) for exception in evaluation router

Related Issues
Not linked yet.

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
agenta	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jul 4, 2024 4:13pm

[mmabrouk]

Thanks @jp-agenta for the PR.
lgtm

I see this issue happen again and again (a non-handled output / edge case from the backend breaking the frontend). Any ideas how can we make sure this issue does not happen in the future? Integration tests with missing information? An explicit definition of the API using something like postman?

[jp-agenta]

I'd say (non-exhaustive list):

1. Consistent (and monotonically growing) automation and manual testing.
2. Proper testing coverage (BDD, RTM, API docs, Web/CLI/SDK docs)
3. Testing not only for E2E from the user/browser perspective but also the API from an endpoint perspective.
4. BDD for E2E testing (functional, security, etc)
5. Splitting tests : smoke (fast), sanity (partial), regression (full), acceptance (new) -- automated and/or manual
6. Automated API documentation (incl. strong typing) -- OpenAPI ?
7. API-driven design is key when there are multiple interfaces (web, cli, sdk(s))

[jp-agenta]

QA tests

(Opens even with failed scenario n°2)

[mmabrouk]

Thanks @jp-agenta !