Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Faster which/command lookup implementation #98

Merged
merged 2 commits into from
Nov 14, 2013

Conversation

jlisee
Copy link
Contributor

@jlisee jlisee commented Nov 13, 2013

Taken from the twister.python.procutils.which this function speeds up the
importing of commands by a factor of 6, and which itself by a several
hundred times.

This is an example fix for issue #97.

On the master branch:

jlisee:$ time python -c "from plumbum.cmd import grep, wc, cat, head, ls, touch, mkdir, rm, echo, dd"

real  0m0.646s
user  0m0.588s
sys   0m0.048s

With this fix

jlisee:$ time python -c "from plumbum.cmd import grep, wc, cat, head, ls, touch, mkdir, rm, echo, dd"

real  0m0.061s
user  0m0.024s
sys   0m0.016s

Taken from the twister.python.procutils.which this function speeds up the
importing of commands by a factor of 6, and which itself by a several
hundred times.
This new verison is based on the original plumbum version.  The main speed boost is
from not enumerating every single file in each directory on the PATH.  We just build
the full path and rely on the OS to tell us if the file exists and we can run it.
@jlisee
Copy link
Contributor Author

jlisee commented Nov 14, 2013

The new version actual passes the tests! I had to fire up a vagrant VM and run the travis-ci instructions to run the tests. It would be nice to be able to run tests without installing the package, but I wasn't able to figure that out.

@tomerfiliba
Copy link
Owner

@jlisee - thanks!

by the way, when you submit a pull request travis automatically tests it, so there's no need to fire up a local travis. also, you can just run the tests with nose.

tomerfiliba added a commit that referenced this pull request Nov 14, 2013
Faster which/command lookup implementation
@tomerfiliba tomerfiliba merged commit f654be6 into tomerfiliba:master Nov 14, 2013
tomerfiliba pushed a commit that referenced this pull request Nov 14, 2013
@jlisee jlisee deleted the import-speedup branch November 17, 2013 15:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants