Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Do not attempt to rewrite non-source files #5490

Merged
merged 1 commit into from
Jun 25, 2019
Merged

Do not attempt to rewrite non-source files #5490

merged 1 commit into from
Jun 25, 2019

Conversation

asottile
Copy link
Member

Resolves #5488

I couldn't find a good way to reproduce this in a test, so here's my "minimal reproduction":

$ tree
.
├── setup.py
└── x
    ├── basic_c_module.c
    ├── __init__.py
    └── y.py

1 directory, 4 files
$ find -type f | xargs tail -n 99
==> ./setup.py <==
from setuptools import setup, Extension

setup(
    name='foo',
    packages=['x'],
    ext_modules=[Extension('x.basic_c_module', ['x/basic_c_module.c'])],
    entry_points={'pytest11': ['x=x.y']},
)

==> ./x/basic_c_module.c <==
#include <Python.h>

static PyObject* _hello_world(PyObject* self) {
    return PyUnicode_FromString("hello world");
}

static struct PyMethodDef methods[] = {
    {"hello_world", (PyCFunction)_hello_world, METH_NOARGS},
    {NULL, NULL}
};

#if PY_MAJOR_VERSION >= 3
static struct PyModuleDef module = {
    PyModuleDef_HEAD_INIT,
    "basic_c_module",
    NULL,
    -1,
    methods
};

PyMODINIT_FUNC PyInit_basic_c_module(void) {
    return PyModule_Create(&module);
}
#else
PyMODINIT_FUNC initbasic_c_module(void) {
    Py_InitModule3("basic_c_module", methods, NULL);
}
#endif

==> ./x/y.py <==
import x.basic_c_module

==> ./x/__init__.py <==

I then did

pip install .
pytest x

Which resulted in:

$ pytest x
Traceback (most recent call last):
  File "/tmp/pytest/venv/bin/pytest", line 11, in <module>
    load_entry_point('pytest', 'console_scripts', 'pytest')()
  File "/tmp/pytest/src/_pytest/config/__init__.py", line 55, in main
    config = _prepareconfig(args, plugins)
  File "/tmp/pytest/src/_pytest/config/__init__.py", line 200, in _prepareconfig
    pluginmanager=pluginmanager, args=args
  File "/tmp/pytest/venv/lib/python3.6/site-packages/pluggy/hooks.py", line 289, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/tmp/pytest/venv/lib/python3.6/site-packages/pluggy/manager.py", line 87, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/tmp/pytest/venv/lib/python3.6/site-packages/pluggy/manager.py", line 81, in <lambda>
    firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
  File "/tmp/pytest/venv/lib/python3.6/site-packages/pluggy/callers.py", line 203, in _multicall
    gen.send(outcome)
  File "/tmp/pytest/src/_pytest/helpconfig.py", line 89, in pytest_cmdline_parse
    config = outcome.get_result()
  File "/tmp/pytest/venv/lib/python3.6/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/tmp/pytest/venv/lib/python3.6/site-packages/pluggy/callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "/tmp/pytest/src/_pytest/config/__init__.py", line 661, in pytest_cmdline_parse
    self.parse(args)
  File "/tmp/pytest/src/_pytest/config/__init__.py", line 869, in parse
    self._preparse(args, addopts=addopts)
  File "/tmp/pytest/src/_pytest/config/__init__.py", line 815, in _preparse
    self.pluginmanager.load_setuptools_entrypoints("pytest11")
  File "/tmp/pytest/venv/lib/python3.6/site-packages/pluggy/manager.py", line 292, in load_setuptools_entrypoints
    plugin = ep.load()
  File "/tmp/pytest/venv/lib/python3.6/site-packages/importlib_metadata/__init__.py", line 90, in load
    module = import_module(match.group('module'))
  File "/tmp/pytest/venv/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "/tmp/pytest/src/_pytest/assertion/rewrite.py", line 143, in exec_module
    exec(co, module.__dict__)
  File "/tmp/pytest/venv/lib/python3.6/site-packages/x/y.py", line 1, in <module>
    import x.basic_c_module
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "/tmp/pytest/src/_pytest/assertion/rewrite.py", line 134, in exec_module
    source_stat, co = _rewrite_test(fn)
  File "/tmp/pytest/src/_pytest/assertion/rewrite.py", line 283, in _rewrite_test
    tree = ast.parse(source, filename=fn)
  File "/usr/lib/python3.6/ast.py", line 35, in parse
    return compile(source, filename, mode, PyCF_ONLY_AST)
ValueError: source code string cannot contain null bytes

After fixing:

$ pytest x
============================= test session starts ==============================
platform linux -- Python 3.6.7, pytest-4.6.1.dev136+ga54e2e19f.d20190625, py-1.8.0, pluggy-0.12.0
rootdir: /tmp/pytest, inifile: tox.ini
plugins: foo-0.0.0
collected 0 items                                                              

========================= no tests ran in 0.01 seconds =========================

@nicoddemus
Copy link
Member

I understand this is complicated to reproduce, let's go ahead with the fix then! Thanks!

@nicoddemus nicoddemus closed this Jun 25, 2019
@nicoddemus nicoddemus reopened this Jun 25, 2019
@codecov
Copy link

codecov bot commented Jun 25, 2019

Codecov Report

Merging #5490 into master will decrease coverage by 2.29%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff            @@
##           master    #5490     +/-   ##
=========================================
- Coverage   96.07%   93.78%   -2.3%     
=========================================
  Files         117      117             
  Lines       25542    25542             
  Branches     2473     2473             
=========================================
- Hits        24540    23954    -586     
- Misses        698     1259    +561     
- Partials      304      329     +25
Impacted Files Coverage Δ
src/_pytest/assertion/rewrite.py 94.21% <ø> (-0.62%) ⬇️
testing/test_argcomplete.py 20.37% <0%> (-61.12%) ⬇️
testing/test_pdb.py 47.31% <0%> (-51.69%) ⬇️
src/_pytest/_argcomplete.py 27.27% <0%> (-45.46%) ⬇️
src/_pytest/debugging.py 69.63% <0%> (-20.42%) ⬇️
testing/test_conftest.py 90.15% <0%> (-9.47%) ⬇️
testing/test_pathlib.py 92.1% <0%> (-7.9%) ⬇️
src/_pytest/pytester.py 84% <0%> (-6.97%) ⬇️
src/_pytest/unittest.py 89.38% <0%> (-5.59%) ⬇️
testing/test_collection.py 95.23% <0%> (-4.58%) ⬇️
... and 20 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a54e2e1...b991810. Read the comment docs.

@nicoddemus
Copy link
Member

Re-opening to trigger CI again.

@nicoddemus nicoddemus merged commit aaf4031 into pytest-dev:master Jun 25, 2019
@nicoddemus
Copy link
Member

Thanks!

@asottile asottile deleted the only_source_files branch June 25, 2019 23:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Failure trying to rewrite a .so file
2 participants