Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Compression support #39

Closed
3 tasks done
cifkao opened this issue Dec 30, 2020 · 5 comments
Closed
3 tasks done

Compression support #39

cifkao opened this issue Dec 30, 2020 · 5 comments
Labels
enhancement New feature or request

Comments

@cifkao
Copy link
Contributor

cifkao commented Dec 30, 2020 •
edited by salu133445
Loading

  • Allow passing open files (TextIO) instead of paths to save_* and load_* methods/functions to facilitate compression.
  • Add support for some common compression method(s) (e.g. gzip, lzma) for JSON and (maybe) YAML via an optional compression parameter.
  • Make save and load recognize compressed file extensions, e.g. .json.gz

For discussion and efficiency benchmark, see #36.
@salu133445 salu133445 added the enhancement New feature or request label Dec 30, 2020
@salu133445
Copy link
Owner

Opened files are now supported by save_json and save_yaml (8a485b1).

@salu133445
Copy link
Owner

Opened files are now supported by load_json and load_yaml (bcf7a39).

@salu133445
Copy link
Owner

Gzipped JSON and YAML files (.json.gz and .yaml.gz) are now supported by load and save (27d26b6).

@salu133445
Copy link
Owner

Another TODO is to expose the compressed argument to the dataset module (e.g., muspy.Dataset.convert).

salu133445 added a commit that referenced this issue Jan 15, 2021
@salu133445
Support kwargs in FolderDataset.convert (#39)
646cadc
@salu133445
Copy link
Owner

Most features are now supported.

@cifkao cifkao mentioned this issue Apr 1, 2021
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@cifkao @salu133445