Serialize and deserialize bytes without length

[cauebs]

I and a friend are implementing a binary network protocol and wanted to serialize and deserialize structs to and from raw bytes, without padding. Some of these structs contain a payload field at the end, which can be arbitrarily large, but is always the last to be read or written. The problem we now faced is that bincode always writes the length (in serialize_bytes) and expects it to be present when deserializing as well. I can't think of a good way to offer this option, considering the limitations of the traits provided by serde, but I really think this should be possible. Any thoughts?

[cauebs]

In case anyone reading this is having the same problem, we found a library that is probably going to simplify a lot of things: https://github.com/libpnet/libpnet

[TyOverby]

A few random thoughts:

• Bincode wasn't intended for this purpose, so there's no easy answer here
• It might be possible to do it with bincode (@dtolnay keeps surprising me with the things that Serde - and bincode - can do.
• Maybe this is a common enough ask that someone (me?) should make a crate just for this kind of usage

[JoshMcguigan]

Hey @TyOverby, I am reading through bincode to learn more about serde (thanks for your work on this btw), and I'd just like to confirm my understanding of something here.

Initially my thought was that bincode could omit encoding the length before a serde seq type if that type would be the last thing serialized. I think the problem here is that the serializer does not know whether or not a given seq is the last thing to be serialized? In that case, enabling this type of functionality would require a change not just to bincode but also to serde?

If you don't mind sharing, what would your approach be for implementing this as a separate crate? Would you work entirely outside the serde ecosystem?

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.