[hf inference] ASR remote inference model parser impl #1020
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Ankush-lastmile
force-pushed
the
pr1020
branch
from
a91c116 to
4d6e47c
Compare
Ankush-lastmile
changed the title
Ankush-lastmile
force-pushed
the
pr1020
branch
from
4d6e47c to
7574133
Compare
Ankush-lastmile
requested review from
saqadri,
rholinshead,
suyoglastmileai,
jonathanlastmileai and
rossdanlm
as code owners
Ankush-lastmile
force-pushed
the
pr1020
branch
from
7574133 to
3a8d37c
Compare
rholinshead
requested changes
Jan 25, 2024
.../src/aiconfig_extension_hugging_face/remote_inference_client/automatic_speech_recognition.py
Outdated
Show resolved
Hide resolved
.../src/aiconfig_extension_hugging_face/remote_inference_client/automatic_speech_recognition.py
Outdated
Show resolved
Hide resolved
.../src/aiconfig_extension_hugging_face/remote_inference_client/automatic_speech_recognition.py
Outdated
Show resolved
Hide resolved
.../src/aiconfig_extension_hugging_face/remote_inference_client/automatic_speech_recognition.py
Outdated
Show resolved
Hide resolved
Comment on lines
216
to
219
| if len(inputs) > 1: | ||
| raise ValueError( | ||
| f"Multiple audio inputs are not supported for the HF Automatic Speech Recognition Inference api. Please specify a single audio input attachment." | ||
| ) |
There was a problem hiding this comment.
Ah, ok, that's what I was thinking. Instead of doing this, we should just make the validate_and_retrieve function return a single value, not array
There was a problem hiding this comment.
updated. refactpred validate_and_retrieve returns a single value, not an array
|
|
||
| # HuggingFace Automatic Speech Recognition outputs should only ever be string | ||
| # format so shouldn't get here, but just being safe | ||
| return json.dumps(output_data, indent=2) |
There was a problem hiding this comment.
Based on your comment on the image_2_text one, maybe this should raise a ValueError instead?
There was a problem hiding this comment.
Yep, updated. Had previously copy pasted what you had for consistency.
Ankush-lastmile
force-pushed
the
pr1020
branch
from
3a8d37c to
a0f6f12
Compare
Testplan,Same as in original pr description, output had the same output so omitting the screenshot |
Ankush-lastmile
force-pushed
the
pr1020
branch
from
a0f6f12 to
a6dfbdb
Compare
rholinshead
reviewed
Jan 25, 2024
| f"Attachment has no mime type. Specify the audio mimetype in the aiconfig" | ||
| ) | ||
|
|
||
| if not attachment.mime_type.startswith("audio/"): |
There was a problem hiding this comment.
Should be "audio" without trailing slash since we default to just "audio" and this would invalidate that. Alternatively, could default to "audio/*" but not sure if that will work the same
There was a problem hiding this comment.
ah I see what you mean. Technically this doesn't break anything yet but will be needed.
Updated, thanks for catching
rholinshead
approved these changes
Jan 25, 2024
Implementation of the HuggingFaceAutomaticSpeechRecognition Model parser using the inference endpoint to run inference. Python API takes in bytes as well as path, skip binary for now. Very similar to #1018 ## Testplan <img width="1000" alt="Screenshot 2024-01-24 at 10 37 05 PM" src="https://github.com/lastmile-ai/aiconfig/assets/141073967/808956ce-e3be-4528-9f34-c8d31d704ddb"> 1. Temporarily add model parser to Gradio Cookbook model parser registry. ``` asr = HuggingFaceAutomaticSpeechRecognitionRemoteInference() AIConfigRuntime.register_model_parser( asr, asr.id() ) ``` 2. run AIConfig Edit on Gradio example `python3 -m 'aiconfig.scripts.aiconfig_cli' edit --aiconfig-path=cookbooks/Gradio/huggingface.aiconfig.json --parsers-module-path=cookbooks/Gradio/hf_model_parsers.py --server-mode=debug_servers`
Ankush-lastmile
force-pushed
the
pr1020
branch
from
a6dfbdb to
02e43fd
Compare
|
update the mimetype validation to check |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
[hf inference] ASR remote inference model parser impl
Implementation of the HuggingFaceAutomaticSpeechRecognition Model parser using the inference endpoint to run inference. Python API takes in bytes as well as path, skip binary for now.
Very similar to #1018
Testplan
python3 -m 'aiconfig.scripts.aiconfig_cli' edit --aiconfig-path=cookbooks/Gradio/huggingface.aiconfig.json --parsers-module-path=cookbooks/Gradio/hf_model_parsers.py --server-mode=debug_servers