[Hotfix]: resolve ChatCompletionMessage object is not subscriptable

[aybruhm]

Description

This PR resolves the ChatComletionMessage is not subscriptable error when running auto ai critique evaluation.

Additional Information

Full Error Traceback:

[2024-06-07 13:39:28,371: DEBUG/ForkPoolWorker-1] Result: evaluator_config=ObjectId('666309d686a85ea3911dbc99') result=Result(type='error', value=None, error=Error(message='Error during Auto AI Critique', stacktrace="'ChatCompletionMessage' object is not subscriptable"))
[2024-06-07 13:39:28,403: ERROR/ForkPoolWorker-1] Task agenta_backend.tasks.evaluations.evaluate[67ad4369-e935-42e2-b400-fc962af27510] raised unexpected: TypeError('expected string or bytes-like object')
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/celery/app/trace.py", line 453, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/celery/app/trace.py", line 736, in __protected_call__
    return self.run(*args, **kwargs)
  File "/app/agenta_backend/tasks/evaluations.py", line 324, in evaluate
    aggregated_results = loop.run_until_complete(
  File "/usr/local/lib/python3.9/asyncio/base_events.py", line 647, in run_until_complete
    return future.result()
  File "/app/agenta_backend/tasks/evaluations.py", line 368, in aggregate_evaluator_results
    result = aggregation_service.aggregate_ai_critique(results)
  File "/app/agenta_backend/services/aggregation_service.py", line 21, in aggregate_ai_critique
    match = re.search(r"\d+", result.value)
  File "/usr/local/lib/python3.9/re.py", line 201, in search
    return _compile(pattern, flags).search(string)
TypeError: expected string or bytes-like object

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
agenta	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jun 7, 2024 2:34pm

[mmabrouk]

Thanks @aybruhm !
Any clue why the tests are failing?

[aybruhm]

Thanks @aybruhm ! Any clue why the tests are failing?

Yup, it turns out that the issue was because of the AGENTA_TEMPLATE_REPO change I made in the test compose.