Audio PR 4 - File List Reader and Meta-data reader #23
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Owner
swetha097
commented
Aug 2, 2023
•
swetha097
commented
- Add support for file_list_reader to read source files from file_list instead of folder_path (similar to file_source_reader)
- Add support for meta-data reader (label_reader_file_list) to read and add labels to map from file_list.txt
snehaa8
reviewed
Aug 3, 2023
| FileListReader(); | ||
|
|
||
| private: | ||
| //! opens the folder containnig the images |
| _curr_file_idx = 0; | ||
| } | ||
|
|
||
|
|
| continue; | ||
| } | ||
| _in_batch_read_count++; | ||
| _in_batch_read_count = (_in_batch_read_count%_batch_count == 0) ? 0 : _in_batch_read_count; |
There was a problem hiding this comment.
Please add space before and after % if its the norm in MIVisionX too.
| if(_in_batch_read_count > 0 && _in_batch_read_count < _batch_count) | ||
| { | ||
| replicate_last_image_to_fill_last_shard(); | ||
| LOG("FileReader ShardID [" + TOSTR(_shard_id) + "] Replicated " + _folder_path+_last_file_name + " " + TOSTR((_batch_count - _in_batch_read_count) ) + " times to fill the last batch") |
There was a problem hiding this comment.
change
_folder_path+_last_file_name
to
_folder_path + _last_file_name
| continue; | ||
| } | ||
| _in_batch_read_count++; | ||
| _in_batch_read_count = (_in_batch_read_count%_batch_count == 0) ? 0 : _in_batch_read_count; |
SundarRajan28
requested changes
Aug 7, 2023
| @@ -27,7 +27,7 @@ THE SOFTWARE. | |||
| /// \param rocal_context | |||
| /// \param source_path path to the folder that contains the dataset or metadata file | |||
| /// \return RocalMetaData object, can be used to inquire about the rocal's output (processed) tensors | |||
There was a problem hiding this comment.
Add docs for file_list_path
| @@ -0,0 +1,54 @@ | |||
| /* | |||
| Copyright (c) 2019 - 2023 Advanced Micro Devices, Inc. All rights reserved. | |||
There was a problem hiding this comment.
Change this 2023 since audio support has been added in this year and this is a new file. Change the copyright year for all new files to 2023
| void reset() override; | ||
|
|
||
| //! Returns the name of the latest file opened | ||
| std::string id() override { return _last_id;}; |
| file_path = _folder_path + "/"+ file_path; | ||
|
|
||
|
|
||
| // std::cout << "\n entry Path String" << entry_path << '\n'; |
There was a problem hiding this comment.
Remove all debug statements in this file
| std::vector<std::string> entry_name_list; | ||
| // open_subdirectory(_folder_path.c_str()); | ||
| auto ret = Reader::Status::OK; | ||
| std::ifstream fp (_file_list_path); |
There was a problem hiding this comment.
Remove space before (
| // std::cout << "\n entry Path String" << entry_path << '\n'; | ||
|
|
||
| // std::cout << filesys::is_regular_file(entry.path()) << "\n"; | ||
| if (filesys::is_regular_file(file_path )) |
There was a problem hiding this comment.
Remove extra spaces before and after ()
There was a problem hiding this comment.
@swetha097 Please resolve the spacing issues in all new files added to the branch
…el_reader_file_list.cpp & .h files)
….com/swetha097/MIVisionX into swetha/audio_pr4/meta_data
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.