Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature Request]: A directory in the repo with numbers from different architectures #85

Open
goldsteinn opened this issue Apr 2, 2021 · 4 comments

Comments

@goldsteinn
Copy link

It would allow anyone w/o having to have the older/newer hardware to compare. It would also make this repo an amazing resource for quickly checking numbers.

@nemequ
Copy link
Collaborator

nemequ commented Apr 2, 2021

I would suggest a separate repo instead.

@goldsteinn
Copy link
Author

What about in the "results" page on the wiki?

@travisdowns
Copy link
Owner

travisdowns commented Apr 10, 2021

It would allow anyone w/o having to have the older/newer hardware to compare. It would also make this repo an amazing resource for quickly checking numbers.

Something like this was definitely one of my original goals. I have received results from various systems, but mostly haven't published them.

What about in the "results" page on the wiki?

In fact, that was the original idea. Note that page is empty: it's because I originally thought the wiki was hierarchical, so I would put all the results "under" this page, but that's not how it works at all. Ultimately I uploaded a single set of Zen results.

So I like the idea, but here are some reasons this might not happen or not happen soon:

  • Due to changes in my life situation I have less time for hobby programming.
  • I dislike the idea of storing freeform text results and that's currently what uarch-bench outputs. I want to fix this for other reasons (e.g. to allow json output), but I wouldn't want to collect a bunch of freeform results first and then have to convert them.
  • I'm always adding benchmarks, so old results would get out of date.
  • It would be nice to have some kind of table-based display where you could "diff" two sets or results or show them side by side or whatever, more than just a bunch of riles in a repo, but this takes time (see point 1).
  • The tests need a bit of work to be stable on a larger variety of hardware, e.g., I don't do many checks to avoid frequency scaling and other things that would throw the results off. I already have some dodgy results because of that (it's noticeable when things that should be an integer number of cycles all show up 0.89 cycles or whatever). Even if I can't avoid it, I should have extra columns to identify when this happened, e.g., a column showing the start (instantaneous) and end frequencies, as well as the average frequency across the test, etc.
  • The tests are quite x86 specific because most of them are written in asm. I do write some tests in C++ but probably not enoguh to make this useful on anythings except x86.

That said, I'm definitely open to the idea, especially if I get some help. I know there are people willing to provide results on all kinds of hardware.

If you're looking for collections of this type of info out there already, check out 7-cpu. They have low level details and sometimes freeform commentary for a lot of CPU types (it's not the same for every CPU). See for example Zen 2.

@travisdowns
Copy link
Owner

@goldsteinn - I added one result from Zen 3 to the "results" dir.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants