-
Notifications
You must be signed in to change notification settings - Fork 91
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add a faster midi parsing backend #112
Comments
This looks promising! Also, the way I saw the switch was to use the new c++ backend by default when reading / loading files, but still supporting |
That's great. It's easier to support other types of midi events, while writing back will lead to heavy workload. I'll try to implement these functions. And is there any high level interface you need for better integration? Currently, we merely add some functions like time_shift, clip, sort, to meet our own needs, and the pianoroll conversion function is quite casual. We are trying to make a more friendly interface in python level. |
I totally get it. :) I didn't touch c/c++ in a long time, but if there is a way I can help don't hesitate to tell me!
For MidiTok, the needs are mainly to parse the MIDI data with respect to the General MIDI 2 specifications, so I guess you already cover them well. 🙌 |
Thanks for your feedback. I'll answer some of the questions, while @lzqlzzq (the author of minimidi, also the co-author of symusic) will answer the others, especially those about architecture.
By the way, I have added Marker, Lyrics, and Pitch Bend to symusic (haven't upload to pypi) and minimid, but I have some problems with |
Thanks for your attention and feedback on
|
Additionally, the conversion between Of course, this kind of conversion will cause a certain degree of from miditoolkit import MidiFile
midi = MidiFile("midi file path")
ticks_per_quarter = midi.ticks_per_beat
# conversion
start_midi_tick: int = midi.instruments[0].notes[0] # start time of an example note
start_quarter: float = start_midi_tick / ticks_per_quarter
# For writing back, we just need to set a new quantization factor, the same meaning as ticks per quarter
q = 960 # for example
start_quantized = int(start_quarter * q) |
Hi guys, thank you for the quick feedback! I finally got time to run a more complete "benchmark", here is the script: https://gist.github.com/Natooz/1a39413220d038e0ac3981f864039370
The good news is that almost all the notes are retrieved exactly the same, with a few exception after time signature changes. Here are the saved files. failed_midis.zip I'll answer to each points respectively now.
Another thing, while writing the benchmark I had to manually recreate the lists of notes from the tracks in order to sort them. Again, I want to thank you for developing this and getting to MidiTok. I truly think that once achieved (i.e. all MIDI data parsed and written back as original), Symusic + minimidi can durably replace miditoolkit / pretty_midi and maybe even Mido for some usages. |
|
@Natooz I have refactored almost all the code using template, which allows users to choose the time_unit by themselves. The new version of However, it might be easy for you to explore the new version: from symusic import Score
score = Score("path to your midi", time_unit="tick")
for track in score.tracks:
for note in track.notes:
cur_tick = note.time # Here, for all the event, I use time attribute to represent the time stamp (instead of start for Note)
duration = note.duration |
@Yikai-Liao This is great! I added "TODOs" notes in the code where I found things that could be improved, namely:
When I run it on the same MIDI files, I get the same errors for the notes, and one file has errors with tempos, and the last one has a key signature mismatch, its
I also tested on the multitrack files, and got more errors but I think we can keep on working the "one_track" ones for the moment. I did not have time to really explore the code of Symusic, I'll try to do so and maybe send PR if I find things where I can help. |
Thanks for your reply. We are grateful for your help in testing it! Maybe there's something wrong with my expression, I mean that even if I tweak some of the bindings, debugging tool might be able to help you to see the new interface. Also, after the refactoring, the code is a lot more abstract and indeed harder to read. Btw, I think I should also prepare more for the my 12.10 gre test these days |
@Yikai-Liao @lzqlzzq thank you for the active development of symusic and minimidi!
I updated the test script to handle the symusic update, and left some todo notes for things that could benefit from some improvement (in symusic and/or just the test script itself).
For the markers and time signatures, I don't know if we should filter them as mido might do. As the library plays the role of a parser, it is intended to retrieve the data exactly as it is within the file without altering (filtering) it even if it is invalid our outside the recommended values, leaving this last task up to the user. But maybe a nice feature would be to offer users a ‘filter_unrecommended_values‘ argument when loading the MIDI, that would make checks when parsing time signatures, markers, lyrics or other concerned events, and not parse those containing values outside the recommended values in the General MIDI specifications. In the coming days I'll open a PR here for the integration of Symusic. This will mark either the 2.2.0 or 3.0.0 version, depending on the amount of changes. The plan is to remove miditoolkit for the requirements, replace it with symusic, but still support |
Thanks for your test. Simple Parsing Error and Design Problems
@dataclass(frozen=True)
class ScoreFactory:
__core_classes = CoreClasses(core.ScoreTick, core.ScoreQuarter, smt.ScoreSecond)
def __call__(self, file: Union[str, Path], ttype: smt.GeneralTimeUnit = TimeUnit.tick):
if isinstance(file, str) or isinstance(file, Path):
return self.from_file(file, ttype)
def from_file(self, path: Union[str, Path], ttype: smt.GeneralTimeUnit = TimeUnit.tick) -> smt.Score:
a = self.__core_classes.dispatch(ttype)
return a.from_file(path) Hard Problems
Future
|
|
I have begun the switch to symidi, it's going well but there are still a few things blocking its full completion:
Additionally, the I'll take a look at those if I have time before you do. |
Also it would be great to implement the midi = Score(midi_path)
midi2 = Score(midi_path)
midi.tracks[0].notes[0] == midi.tracks[0].notes[0]
# True
midi.tracks[0].notes[0] == midi2.tracks[0].notes[0]
# False |
|
Great, thank you for the very prompt reaction! For the first bullet point, I was referring to the base type of the objects. It seems to need to be addressed when using pybind: https://stackoverflow.com/questions/76239369/in-pybind11-is-it-possible-to-determine-whether-a-pyobject-is-a-pydict Also for the |
Well, I do understand the part about |
Of course, here is an example in MidiTok, in the |
Like this? from symusic import Score
from symusic.core import ScoreTick
s = Score("path")
assert isinstance(s, Score) # fail, but you want to make is pass
assert isinstance(s, ScoreTick) # pass |
Yes that's it! |
Well, it's hard to make On the other hand, the purpose of putting all the things from c++ to core is that, I don't want to expose those annoying Maybe there is a better encapsulation method? What about writing a isinstance(alignwith maybe a better name) method for all those factory class? |
I agree with you that it's nicer to not expose Maybe an isinstance for Now, do you think we could have interfaces to create |
Doesn't the current version work? @dataclass(frozen=True)
class NoteFactory:
__core_classes = CoreClasses(core.NoteTick, core.NoteQuarter, core.NoteSecond)
def __call__(
self,
time: smt.TimeDtype,
duration: smt.TimeDtype,
pitch: int,
velocity: int,
ttype: smt.GeneralTimeUnit = TimeUnit.tick,
) -> smt.Note:
"""
Note that `smt.TimeDtype = Union[int, float]`, and Note constructor requires `int` or `float` as time.
So Type Checker like MyPy will complain about the type of `time` argument.
However, float and int can be converted to each other implicitly.
So I just add a `# type: ignore` to ignore the type checking.
"""
return self.__core_classes.dispatch(ttype)(time, duration, pitch, velocity) # type: ignore |
It's working for notes, tempos etc |
Wonderful! 🤩 |
I have begun the new tests, right now 50 out of 136 tests on single track tokenization - detokenization are passing! 🙌 On the meantime, would it be possible to have a |
ok,I'll add it |
@Yikai-Liao there is still one test not passing, that might come from symusic's writing - loading (but not sure 100%). I'll try to add tests to the library now in order to make sure. Also, while when I load back a midi saved with symusic I can retrieve all the tracks with their programs, when I open it with a DAW (Logic Pro), all the tracks have the same program, being the same as the first one, except for drums tracks. There may be a channel mismatch or something else in the writing process? Here are two examples: Archive.zip |
well,in symusic, there is no channel information. there is only program currently. So for convenience, when writing midi back, I just set all track except for drum to channel 0. I'm not sure will this cause a problem. |
I dumped those two midi files with symusic, and asked someone else to load them to Logic Pro and FL studio. He said that, they are loaded correctly. |
Great that's good news! I prefer something being wrong just on my machine, now time to find what. FYI I encounter this bug with symusic 0.2.1, macOS 12.7 intel |
I meant that maybe the MIDI might have been dumped differently from the different platforms? Or maybe this might just come from Logic Pro? I can't find where to see the version. I'll be able to test on my second machine in a few hours, which has an other version installed. |
Music.zip |
I have the same problem with these files, so the issue must come from my installation (Logic 10.7.4). I'll test on my desktop when I'll get home. But I assume there is no issue/bug within symusic then. |
I have the same issue on my other machine, with Logic Pro 10.6.0. |
Maybe you could try fl studio, which also works as excepted in my test. |
It can be opened normally with signal. I don't know what to think about it, I'll just leave it for later maybe and focus on more important matters (tests, integration...) |
This issue is stale because it has been open for 30 days with no activity. |
Leaving this open until the v3 is finally released! |
I'm (finally 😅) about to release the V3! I took more time than I expected, but I also changed/improved a lot of things. There are still a few things that would need to be fixed (especially tokenization tests not passing on a few files from the MMD dataset, using bindings for the tokenization part), I leave it for later, the benefits we have right now are already huge! Before releasing, do you plan to make updates of symusic / minimidi in the coming days? I'm asking so that I can directly set the minimum symusic version to the very latest. Also, I'll make some promotion for symusic on twitter. If you have accounts, I can link to them when crediting you. |
MIDI file reading:
Symusic is 320 faster for Maestro, 364 for MMD, 423 for POP909. MIDI preprocessing#### Maestro Dataset
MetaMIDI Dataset
POP909 dataset
Tokenization (MIDI already loaded)#### Maestro Dataset
MetaMIDI Dataset
POP909 dataset
|
Now, the newest version of symusic is 0.3.2, which fixed some bugs we just found. We will try to complete the (now still empty) documentation in a subsequent 0.3.* update. We do have a more clear plan for the next 2 large versions now. We plans to add support for We have previously written a command line tool for midi audio synthesis based on soundfont that is not open source. It's not quite finished, but it serves as a good base for symusic's next development goals. The support for abc notation is more complicated, because writing a syntax parser from scratch is more difficult, the parse part of abc.js uses more than 6000 lines of js code. Perhaps using the existing abc2midi command line tool is a better choice, and it's also fast actually. Anyway, there are still determinations to be made about the exact technical route of abc notation. BTW, I don't get a twitter account now. Maybe I should sign up for one. |
That's noted, thank you for the info! Ultimately, a dedicated C parser will always be faster, but I of course acknowledge the difficulty of writing one from scratch. Tell me if you plan to create an account ;) |
My brand new twitter account: RealYikaiLiao |
End-to-end tokenization benchmark (MIDI read + tokenization)Performed on a set of 50 MIDIs with an intel i7-6700K CPU. Maestro Dataset
MetaMidi Dataset
POP909 Dataset
|
Closing this as the V3 is finally released! :) |
Recently, I have writen a lightweight midi parsing library (writen in c++ with pybind11), wich is significantly faster than miditoolkit.
https://github.com/Yikai-Liao/symusic
Midi parsing speed has been the bottleneck of preprocessing for a long time, and I want to deal with this problem.
Please tell me if you need any help or if i need add new features to the library. (Note that not all MIDI events are supported currently. For example, we ignored lyrics and pitch bend. Wel will add these features in the future if you need them. )
Here is the midi parsing benchmark:
mido
is writen in pure python, and only parses midi files toevent level
pretty_midi
andmiditoolkit
is based onmido
, and parse midi files tonote level
The text was updated successfully, but these errors were encountered: