--newest-first forgets about existing nodeids when collection is filtered

[blueyed]

--newest-first (added via #3034) forgets about cached nodeids when e.g. run with some file as argument.

Code:

pytest/src/_pytest/cacheprovider.py

Lines 269 to 290 in 3f5622c

	class NFPlugin(object):
	""" Plugin which implements the --nf (run new-first) option """
	def __init__(self, config):
	self.config = config
	self.active = config.option.newfirst
	self.cached_nodeids = config.cache.get("cache/nodeids", [])
	def pytest_collection_modifyitems(self, session, config, items):
	if self.active:
	new_items = OrderedDict()
	other_items = OrderedDict()
	for item in items:
	if item.nodeid not in self.cached_nodeids:
	new_items[item.nodeid] = item
	else:
	other_items[item.nodeid] = item
	items[:] = self._get_increasing_order(
	six.itervalues(new_items)
	) + self._get_increasing_order(six.itervalues(other_items))
	self.cached_nodeids = [x.nodeid for x in items if isinstance(x, pytest.Item)]

I think it should not forget about those - this makes it likely consider all others as new when running without args then again.

It should probably just add to the known items, and have a mechanism to garbage-collect them from time to time, but that is not really critical I suppose.

/cc @feuillemorte

[blueyed]

While at it nodeids could be renamed to known_nodeids maybe also, and a timestamp for its discovery and last collection (or something similar) could be added to the data then.