pytest.mark.parametrize cannot locate argument name when there is another decorator

[archibate]

This is a feature request.

Description

When pytest.mark.parametrize is combined with another decorator that takes (*args, **kwargs), it can't find the correct argument name, see examples below.

Not working

import pytest

def mydeco(foo):
  def wrapped(*args, **kwargs):
    print('wrapped!')
    return foo(*args, **kwargs)

  return wrapped

@pytest.mark.parametrize('x', [1, 2, 3])
@mydeco
def test_func(x):
  assert x == 233

(opengl) [bate@archit taichi]$ pytest asas.py
============================================= test session starts =============================================
platform linux -- Python 3.8.1, pytest-5.3.4, py-1.8.1, pluggy-0.13.1
rootdir: /home/bate/Develop/taichi
plugins: xdist-1.31.0, forked-1.1.3
collected 0 items / 1 error                                                                                   

=================================================== ERRORS ====================================================
__________________________________________ ERROR collecting asas.py ___________________________________________
In wrapped: function uses no argument 'x'
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
============================================== 1 error in 0.06s ===============================================

Working

import pytest

@pytest.mark.parametrize('x', [1, 2, 3])
def test_func(x):
  assert x == 233

(opengl) [bate@archit taichi]$ pytest asas.py    
============================================= test session starts =============================================
platform linux -- Python 3.8.1, pytest-5.3.4, py-1.8.1, pluggy-0.13.1
rootdir: /home/bate/Develop/taichi
plugins: xdist-1.31.0, forked-1.1.3
collected 3 items                                                                                             

asas.py FFF                                                                                             [100%]

================================================== FAILURES ===================================================
________________________________________________ test_func[1] _________________________________________________

x = 1

    @pytest.mark.parametrize('x', [1, 2, 3])
    def test_func(x):
>     assert x == 233
E     assert 1 == 233

asas.py:5: AssertionError
________________________________________________ test_func[2] _________________________________________________

x = 2

    @pytest.mark.parametrize('x', [1, 2, 3])
    def test_func(x):
>     assert x == 233
E     assert 2 == 233

asas.py:5: AssertionError
________________________________________________ test_func[3] _________________________________________________

x = 3

    @pytest.mark.parametrize('x', [1, 2, 3])
    def test_func(x):
>     assert x == 233
E     assert 3 == 233

asas.py:5: AssertionError
============================================== 3 failed in 0.03s ==============================================

Why we need this?

@k-ye was trying to apply a test for all architectures using our @ti.all_archs decorator. It must be the last decorator to function so that ti.init() could be called for each test.
Then, we want to use parametrize and failed due to the reason shown above.
Discussion: taichi-dev/taichi#527 (comment)

Possible solutions:

Thanks to @k-ye:

def parametrize(argnames: str, argvalues):
  # @pytest.mark.parametrize only works for canonical function args, and doesn't
  # support *args or **kwargs. This makes it difficult to play along with other
  # decorators like @ti.all_archs. As a result, we implement our own.
  argnames = [s.strip() for s in argnames.split(',')]
  def iterable(x):
    try:
      _ = iter(x)
      return True
    except:
      return False

  def decorator(test):
    def wrapped(*test_args, **test_kwargs):
      for vals in argvalues:
        if isinstance(vals, str) or not iterable(vals):
          vals = (vals, )
        kwargs = {k: v for k, v in zip(argnames, vals)}
        assert len(kwargs.keys() & test_kwargs.keys()) == 0
        kwargs.update(test_kwargs)
        test(*test_args, **kwargs)
    return wrapped
  return decorator

Related commits: https://github.com/taichi-dev/taichi/pull/527/files/11cb31b4fa1a2836ae015d5a463e4a0245b65346..2d6591825c6253bd2a01df8b3d4e2e71ed64c7a

[The-Compiler]

It works correctly if your decorator is well-behaved and uses functools.wraps:

import functools

import pytest

def mydeco(foo):
  @functools.wraps(foo)
  def wrapped(*args, **kwargs):
    print('wrapped!')
    return foo(*args, **kwargs)

  return wrapped

@pytest.mark.parametrize('x', [1, 2, 3])
@mydeco
def test_func(x):
  assert x == 233

I don't think this is something pytest should work around.

[archibate]

Thank you! It was really my problem not using decoration correctly...

[k-ye]

Thanks! I wonder if this is documented somewhere, or is @functools.wraps a known trick to preserve the function signature info? A quick search on Stackoverflow and I found the closest answer to be using @decorator.decorator: https://stackoverflow.com/a/12200860/12003165

[RonnyPfannschmidt]

its the documented standard way to keep the signature/metadata of a function

[The-Compiler]

Hmm, the Python documentation indeed doesn't say much about it. FWIW here's an article about it: https://lerner.co.il/2019/05/05/making-your-python-decorators-even-better-with-functool-wraps/

[k-ye]

OK, thank you all for the input!