Add Python 3.13 compatibility

[JSCU-CNI]

This PR aims to implement a TargetPath 3.13 variant. Fixes #895 and currently WIP.

• Add 3.13 variant
• Fix scandir
• Fix glob
• Fix rglob
• Do we still need flavours?
• ...

Resources:

• https://docs.python.org/3.13/whatsnew/3.13.html#pathlib
• https://github.com/python/cpython/blob/v3.13.0/Lib/pathlib/_local.py
• https://github.com/python/cpython/blob/v3.12.7/Lib/pathlib.py
• https://github.com/python/cpython/blob/v3.13.0/Lib/glob.py
• https://github.com/python/cpython/blob/v3.12.7/Lib/glob.py

[Schamper]

Fixed up the rest, I believe only flow.record needs to add compatibility with 3.13 (path type).

[codecov]

Codecov Report

Attention: Patch coverage is 96.01594% with 10 lines in your changes missing coverage. Please review.

Project coverage is 78.34%. Comparing base (78f434f) to head (b5322f4).
Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
dissect/target/helpers/compat/path_313.py	95.40%	8 Missing ⚠️
dissect/target/helpers/compat/path_common.py	50.00%	1 Missing ⚠️
dissect/target/target.py	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #897      +/-   ##
==========================================
+ Coverage   78.24%   78.34%   +0.10%     
==========================================
  Files         336      337       +1     
  Lines       29673    29686      +13     
==========================================
+ Hits        23217    23258      +41     
+ Misses       6456     6428      -28     

Flag	Coverage Δ
unittests	78.34% <96.01%> (+0.10%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[Schamper]

Depends on fox-it/flow.record#154

[JSCU-CNI]

That looks great @Schamper, nice work!

[JSCU-CNI]

Seems like some tests are still failing:

$ docker run --rm -it -v .:/data python:3.13 bash
root@4d52f1e090ab:/# useradd user && su user
user@a2523f6f5315:/data$ pip install -e .[dev] && pip install pytest pytest-cov coverage docutils
user@a2523f6f5315:/data$ pip install git+https://github.com/fox-it/flow.record.git@py313-compat
user@a2523f6f5315:/data$ pytest tests -vv
...
FAILED tests/loaders/test_ova.py::test_ova_loader - dissect.target.exceptions.FileNotFoundError: /st.ovf
FAILED tests/plugins/apps/vpn/test_wireguard.py::test_wireguard_plugin_global_log - AssertionError: assert './etc/wireguard/wg0.conf' == 'etc/wireguard/wg0.conf'
  
  - etc/wireguard/wg0.conf
  + ./etc/wireguard/wg0.conf
  ? ++
FAILED tests/plugins/apps/webserver/test_iis.py::test_plugins_apps_webservers_iis_access_iis_format - TypeError: can't compare offset-naive and offset-aware datetimes
FAILED tests/plugins/os/windows/test_clfs.py::test_clfs_plugin - AttributeError: 'TargetPath' object has no attribute '_flavour'
FAILED tests/test_container.py::test_open_fallback_fh - AttributeError: 'RawContainer' object has no attribute 'size'
FAILED tests/test_container.py::test_reset_file_position - AttributeError: 'MockContainer' object has no attribute 'size'
FAILED tests/tools/test_dump.py::test_execute_pipeline[jsonlines-gzip] - TypeError: can't compare offset-naive and offset-aware datetimes
FAILED tests/tools/test_dump.py::test_execute_pipeline[jsonlines-bzip2] - TypeError: can't compare offset-naive and offset-aware datetimes
FAILED tests/tools/test_dump.py::test_execute_pipeline[jsonlines-None] - TypeError: can't compare offset-naive and offset-aware datetimes
FAILED tests/tools/test_dump.py::test_execute_pipeline[msgpack-gzip] - TypeError: can't compare offset-naive and offset-aware datetimes
FAILED tests/tools/test_dump.py::test_execute_pipeline[msgpack-bzip2] - TypeError: can't compare offset-naive and offset-aware datetimes
FAILED tests/tools/test_dump.py::test_execute_pipeline[msgpack-None] - TypeError: can't compare offset-naive and offset-aware datetimes
FAILED tests/tools/test_dump.py::test_execute_pipeline_limited[5] - TypeError: can't compare offset-naive and offset-aware datetimes
FAILED tests/tools/test_dump.py::test_execute_pipeline_limited[15] - TypeError: can't compare offset-naive and offset-aware datetimes
FAILED tests/tools/test_dump.py::test_execute_pipeline_limited[None] - TypeError: can't compare offset-naive and offset-aware datetimes
========================================== 15 failed, 1907 passed, 1 skipped, 8 warnings in 34.07s ==========================================
user@a2523f6f5315:/data$ 

[Schamper]

I didn't run the entire test suite yet with the flow.record fixes, so that probably still requires some looking into 😄

[JSCU-CNI]

Looks good, all tests seem to be working now. We mitigated some new 3.13 DeprecationWarnings in 2ceb21c. Two remain as it seems pathlib.Path.is_reserved() will be deprecated from 3.15 and onwards. What do you suggest we do here?

tests/helpers/test_fsutil.py::test_target_path_is_reserved
  /data/tests/helpers/test_fsutil.py:316: DeprecationWarning: pathlib.PurePath.is_reserved() is deprecated and scheduled for removal in Python 3.15. Use os.path.isreserved() to detect reserved paths on Windows.
    assert not path_fs.path("CON").is_reserved()

tests/helpers/test_fsutil.py::test_target_path_is_reserved
  /data/tests/helpers/test_fsutil.py:317: DeprecationWarning: pathlib.PurePath.is_reserved() is deprecated and scheduled for removal in Python 3.15. Use os.path.isreserved() to detect reserved paths on Windows.
    assert not path_fs.path("foo").is_reserved()

[Schamper]

I saw that deprecation warning as well when looking at the pathlib changes, but the alternative is not really a good fit for Dissect. We don't really use that function anyway beside a unit test, so maybe remove it from the unit test and add a fsutil.isreserved() for completeness sake?

[JSCU-CNI]

[..] so maybe remove it from the unit test and add a fsutil.isreserved() for completeness sake?

b860576 looks like a good solution.

1932 passed, 1 skipped in 33.90s

Seems like this PR is good to go :)

[JSCU-CNI]

We performed some more testing on 3.13 and have not come across any breaking issues.

[JSCU-CNI]

I think the tests for 3.13 fail because the CI does not install the pre-release version of flow.record.

[Schamper]

We have to bump that version anyway, so we'll just have to wait for a new release.

[Schamper]

Can be merged after a new release of flow.record.

[JSCU-CNI]

Tests should pass now that flow.record 3.19 has been released.