Feature request: Enable pytest for executable scripts without ".py" extension

[duerrp]

First of all, let me say that I love pytest! Thanks for your great work.

As far as I understand it is considered standard practice to name executable python scripts without the ".py" extension (e.g., pip, pytest, etc.) - at least on linux. Normally such scripts do not contain much code, but it might still be useful to be able to test them with pytest, which is currently not possible.

Test case:

pyenv virtualenv 3.6.5 foo
pyenv activate foo
pip install pytest
echo "#/usr/bin/env python python\n\ndef test_bla():\n    assert True" > ./foo
pytest ./foo                                                                               

Actual result:

============================================================================== test session starts ==============================================================================
platform linux -- Python 3.6.5, pytest-3.6.0, py-1.5.3, pluggy-0.6.0
rootdir: /home/duerpp, inifile:
collecting 0 items                                                                                                                                                              
========================================================================= no tests ran in 0.00 seconds ==========================================================================
ERROR: not found: /home/duerrp/foo
(no name '/home/duerrp/foo' in any of [])

Expected:

============================================================================== test session starts ==============================================================================
platform linux -- Python 3.6.5, pytest-3.6.0, py-1.5.3, pluggy-0.6.0
rootdir: /home/duerrp, inifile:
collected 1 item                                                                                                                                                                

foo .                                                                                                                                                                     [100%]

=========================================================================== 1 passed in 0.01 seconds ============================================================================

The expected behavior could be implemented using a plugin, but I personally think it should be solved by pytest itself...

[pytestbot]

GitMate.io thinks possibly related issues are #103 (Move py.code to pytest.code), #2975 (Pytest 3.3: tests being skipped without failing), #1956 (Can Pytest be used for Python scripts?), #2268 (FEATURE: run pytest in Jupyter notebooks), and #1411 (Pytest stops).

[RonnyPfannschmidt]

this should absolutely be a plugin - as far as consistent python packaging is concerned - scripts come from entrypoints of modules and there is no excuse not to have a test module

[anarcat]

this should absolutely be a plugin - as far as consistent python packaging is concerned - scripts come from entrypoints of modules and there is no excuse not to have a test module

I am constantly writing small scripts that I don't want to bother to build an entire package for. They're just scripts: I drop them in ~/bin/foo and they work. The only reason pytest can't run them is because they don't have a .py extension.

I currently work around this problem by creating a hardlink to the script (a symlink doesn't work for some reason), but that's really just silly:

anarcat@angela:bin(master)$ ln -sf spaced spaced.py
anarcat@angela:bin(master)$ pytest-3 spaced
============================= test session starts ==============================
platform linux -- Python 3.7.3, pytest-3.10.1, py-1.7.0, pluggy-0.8.0
rootdir: /home/anarcat/bin, inifile:
plugins: remotedata-0.3.1, openfiles-0.3.2, doctestplus-0.2.0, cov-2.6.0, arraydiff-0.3, betamax-0.8.1
collecting ... 
========================= no tests ran in 0.02 seconds =========================
ERROR: not found: /home/anarcat/bin/spaced
(no name '/home/anarcat/bin/spaced' in any of [])

[4]anarcat@angela:bin(master)$ pytest-3 spaced.py 
============================= test session starts ==============================
platform linux -- Python 3.7.3, pytest-3.10.1, py-1.7.0, pluggy-0.8.0
rootdir: /home/anarcat/bin, inifile:
plugins: remotedata-0.3.1, openfiles-0.3.2, doctestplus-0.2.0, cov-2.6.0, arraydiff-0.3, betamax-0.8.1
collecting ... 
========================= no tests ran in 0.02 seconds =========================
ERROR: not found: /home/anarcat/bin/spaced
(no name '/home/anarcat/bin/spaced' in any of [])

[4]anarcat@angela:bin(master)$ ln -f spaced spaced.py
anarcat@angela:bin(master)$ pytest-3 spaced.py 
============================= test session starts ==============================
platform linux -- Python 3.7.3, pytest-3.10.1, py-1.7.0, pluggy-0.8.0
rootdir: /home/anarcat/bin, inifile:
plugins: remotedata-0.3.1, openfiles-0.3.2, doctestplus-0.2.0, cov-2.6.0, arraydiff-0.3, betamax-0.8.1
collected 2 items                                                              

spaced.py ..                                                             [100%]

=========================== 2 passed in 0.04 seconds ===========================

This shouldn't be necessary... I understand you want to have this be a plugin but I don't really know where to start to write one for this. And I did read the documentation on how to write plugins - it's unclear to me that I could actually change the "this is not a module" behavior at all from those examples.

I'm aware of the existence of pytest-console-scripts, but that's not exactly what I want here: I don't want to test the program as a script: I have real test_foo functions in there that are properly designed and are detected by pytest when the file has the right extension.

I find this problem especially confusing because the pytest commandline usage says this:

$ pytest-3 --help
usage: pytest-3 [options] [file_or_dir] [file_or_dir] [...]

positional arguments:
  file_or_dir

... ie. that it should just be able to work directly on a file (not a module, mind you, a file).

Is there some magic trick I'm missing here or I'm the only idiot out there writing tests for his small silly non-packaged programs? :)

[RonnyPfannschmidt]

It's in most cases good to write tests, it's just not a core pytest concern to collect small executables

Personally I just create packages for utility scripts and call it a day

[RonnyPfannschmidt]

As for writing a plugin, the collect_file hook should be a good starting point, unfortunately you will need to create a new kind of Module collector to sort out the finer details

Sorry for the terse answer, I'm on mobile on a train ride

[blueyed]

btw: at least the error message could be improved maybe:

ERROR: not found: /home/anarcat/bin/spaced
(no name '/home/anarcat/bin/spaced' in any of [])

IIRC a possible workaround is to use -opython_files="*" or something similiar.

[anarcat]

IIRC a possible workaround is to use -opython_files="*" or something similiar.

anarcat@angela:bin(master)$ pytest-3 -opython_files='*' spaced
===================================================================================== test session starts =====================================================================================
platform linux -- Python 3.7.3, pytest-3.10.1, py-1.7.0, pluggy-0.8.0
rootdir: /home/anarcat/bin, inifile:
plugins: remotedata-0.3.1, openfiles-0.3.2, doctestplus-0.2.0, cov-2.6.0, arraydiff-0.3, betamax-0.8.1
collecting ... 
================================================================================ no tests ran in 0.01 seconds =================================================================================
ERROR: not found: /home/anarcat/bin/spaced
(no name '/home/anarcat/bin/spaced' in any of [])

[blueyed]

Oh yeah, remembered it wrong: #4476

[anarcat]

okay, so i should follow #4476 is what you're saying? :)