Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update type map caching #2057

Open
wants to merge 10 commits into
base: dev
Choose a base branch
from
Open

Update type map caching #2057

wants to merge 10 commits into from

Conversation

stephprince
Copy link
Contributor

@stephprince stephprince commented Mar 12, 2025

Motivation

Fix issues related to type map caching:

This PR saves the type map pickle file in the user cache directory instead of the pynwb installation location, adds a catch for access errors when writing the cache files, and adds an option to disable caching.

A couple of questions related to implementation:

  1. platformdirs has a built-in versioning argument that creates distinct version folders in the cache directory. The current draft of this PR uses this approach to replace the version indicator file. A pro is that someone working with multiple versions of pynwb would have the appropriate typemap cached for each version. A con is multiple cache files would be saved, especially when developing (We might want to recommend developers use the environment variable to disable the cache in general). Any other considerations I'm missing or preference for which approach to use?
    2. I added a reset_namespaces function, since it is difficult to test type map caching because unpickleable extension namespaces are loaded as part of the test suite. This could partially address unload_namespace #1009, but may require some further testing to make sure everything is working as expected. I can move that to a separate PR if needed, I think it would just limit the tests here.

Edit: I think any changes to resetting/unloading namespaces would need further discussion and should be moved to a separate PR. I'll remove the tests that require these changes for now.

TODO

  • try to replicate original errors in singularity/apptainer

How to test the behavior?

import pynwb

Checklist

  • Did you update CHANGELOG.md with your changes?
  • Have you checked our Contributing document?
  • Have you ensured the PR clearly describes the problem and the solution?
  • Is your contribution compliant with our coding style? This can be checked running ruff check . && codespell from the source directory.
  • Have you checked to ensure that there aren't other open Pull Requests for the same change?
  • Have you included the relevant issue number using "Fix #XXX" notation where XXX is the issue number? By including "Fix #XXX" you allow GitHub to close issue #XXX when the PR is merged.

Copy link

codecov bot commented Mar 12, 2025

Codecov Report

Attention: Patch coverage is 73.68421% with 5 lines in your changes missing coverage. Please review.

Project coverage is 92.75%. Comparing base (2888ad0) to head (393b6e5).

Files with missing lines Patch % Lines
src/pynwb/__init__.py 73.68% 5 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##              dev    #2057      +/-   ##
==========================================
- Coverage   92.85%   92.75%   -0.10%     
==========================================
  Files          28       28              
  Lines        2756     2761       +5     
  Branches      716      714       -2     
==========================================
+ Hits         2559     2561       +2     
- Misses        127      131       +4     
+ Partials       70       69       -1     
Flag Coverage Δ
integration 73.63% <36.84%> (-0.07%) ⬇️
unit 82.79% <68.42%> (+0.03%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@stephprince stephprince marked this pull request as ready for review March 12, 2025 22:58
@stephprince stephprince requested a review from rly March 12, 2025 22:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
1 participant