Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/deserde #413

Merged
merged 81 commits into from
Oct 24, 2021
Merged
Changes from 1 commit
Commits
Show all changes
81 commits
Select commit Hold shift + click to select a range
b9e3a80
a fresh new start
ZoeyR Jun 19, 2021
22ed10b
reorganize de/enc modules
ZoeyR Jun 19, 2021
b1ebece
First version of the bincode serialization spec
VictorKoenders Sep 13, 2021
a68c95d
Added documentation on String/&str
VictorKoenders Sep 13, 2021
a899cb7
Fixed internal links in spec.md
VictorKoenders Sep 13, 2021
ee83f30
Started working on bincode_derive
VictorKoenders Sep 13, 2021
a6a52a8
Removed derive/Cargo.lock
VictorKoenders Sep 13, 2021
85693e0
Enable CI for the deserde branch
VictorKoenders Sep 13, 2021
5cdb578
Made the encoder respect intencoding, implemented FixintEncoding
VictorKoenders Sep 19, 2021
f61fdeb
Renamed blob to slice
VictorKoenders Sep 19, 2021
8dbf723
Moved Encoder to its own file
VictorKoenders Sep 19, 2021
23221ce
Turned Config::INT_ENCODING back into a const, made a ton of varint_e…
VictorKoenders Sep 19, 2021
3deff50
Disabled rust 1.41 CI check
VictorKoenders Sep 19, 2021
54580ee
Re-enabled tarpaulin
VictorKoenders Sep 19, 2021
719edd9
Split Error into EncodeError and DecodeError
VictorKoenders Sep 19, 2021
9519dd2
Added decode_unsigned
VictorKoenders Sep 19, 2021
ff80a8d
Added unit tests for decoding errors
VictorKoenders Sep 19, 2021
f45080b
Added unit tests for encode_signed
VictorKoenders Sep 19, 2021
4ca934a
Add generic bound support to derive
ZoeyR Sep 20, 2021
419e26f
Add derive for Decodeable
ZoeyR Sep 20, 2021
5aa3031
Add derive support for tuple structs
ZoeyR Sep 20, 2021
410abd8
Add enum encode/decode derive
ZoeyR Sep 20, 2021
b5e4b33
Exclude derive from coverage
ZoeyR Sep 20, 2021
588dfab
Made clippy also check _derive, fixed clippy warnings
VictorKoenders Sep 21, 2021
8c25ee6
Added decode_ fns
VictorKoenders Sep 22, 2021
caa95c5
Added config options for endian and int_encoding, added full coverage…
VictorKoenders Sep 22, 2021
620cf86
Added support for slices, str, fixed size arrays. Added lifetime to D…
VictorKoenders Sep 22, 2021
c6cb968
Enable non-slice readers
ZoeyR Sep 22, 2021
4782e48
Moved all feature flags to src/features, made the CI run tests on all…
VictorKoenders Sep 22, 2021
f631913
Clean up the borrow crimes
ZoeyR Sep 22, 2021
cb23078
Removed `allow` attributes, fixed some warnings
VictorKoenders Sep 25, 2021
484a9ce
Make bincode_derive 0 dependencies (#409)
VictorKoenders Oct 12, 2021
55fe28a
Added alloc and std tests
VictorKoenders Oct 12, 2021
1763642
Started working on documentation
VictorKoenders Oct 12, 2021
38c7d2d
Removed unneeded Config trait constraints
VictorKoenders Oct 12, 2021
c9ffe28
Added documentation for the src/de module
VictorKoenders Oct 14, 2021
5ae3117
Added documentation for the src/enc module
VictorKoenders Oct 14, 2021
62081c0
Added documentation for the src/featyres module
VictorKoenders Oct 14, 2021
b3cbd8c
Added documentation for the src/ module. Added warning for missing do…
VictorKoenders Oct 14, 2021
41da862
Added support for char encoding/decoding
VictorKoenders Oct 14, 2021
3425380
Added support for the bool type
VictorKoenders Oct 14, 2021
986f9dc
Added support for tuples with up to 8 values
VictorKoenders Oct 14, 2021
4792d15
Added an option to encode a fixed array length or to skip it
VictorKoenders Oct 14, 2021
94f67f5
Added support for Option<T> and Result<T, U>
VictorKoenders Oct 16, 2021
c827d58
Fixed newline issues in project
VictorKoenders Oct 16, 2021
549129a
Added support for Vec<T>, Box<T> and Box<[T]>
VictorKoenders Oct 16, 2021
30bd9e1
Added support for Cell and RefCell
VictorKoenders Oct 16, 2021
04ff6a6
Added support for Cow, Rc and Arc
VictorKoenders Oct 16, 2021
b383e0f
Added support for BinaryHeap, BTreeMap, BTreeSet and VecDeque
VictorKoenders Oct 16, 2021
66fa711
Added support for CStr and CString
VictorKoenders Oct 16, 2021
5f26eb0
Added support for String, Mutex<T> and RwLock<T>
VictorKoenders Oct 16, 2021
3d35f30
Added core::time::Duration and std::time::SystemTime
VictorKoenders Oct 16, 2021
fd203ab
Added support for Path and PathBuf
VictorKoenders Oct 16, 2021
70a8403
Added support for Range<T>, RangeInclusive<T> and Bound<T>
VictorKoenders Oct 16, 2021
10b2e31
Added support for NonZero* types
VictorKoenders Oct 16, 2021
40f90e9
Added support for IpAddr, Ipv4Addr, Ipv6Addr, SocketAddr, SocketAddrV…
VictorKoenders Oct 16, 2021
5e36fd9
Added support for atomic integers
VictorKoenders Oct 16, 2021
1c0d643
Added badges to the functions to indicate which features they require
VictorKoenders Oct 17, 2021
466bb3d
Refactor and rename encoders
ZoeyR Oct 17, 2021
4c9cbdf
Added warning on unused lifetimes, fixed warnings
VictorKoenders Oct 17, 2021
2e6474f
Made the test stage throw an error on warnings
VictorKoenders Oct 17, 2021
53f9d19
Made the varint decode_signed module report the correct errors
VictorKoenders Oct 17, 2021
4f01bd8
Config rewrite (#412)
ZoeyR Oct 17, 2021
1895a0b
Merge branch 'trunk' into feature/deserde
ZoeyR Oct 17, 2021
dfe8649
Processed some of the feedback
VictorKoenders Oct 18, 2021
34dd6cd
Added more documentation to derive/src/generate
VictorKoenders Oct 19, 2021
64d6447
Removed a panic in the derive/src/generate module, moved it to derive…
VictorKoenders Oct 19, 2021
8e10a07
Included spec.md into cargo's documentation, fixed the issues, change…
VictorKoenders Oct 19, 2021
103fe99
Made the Decode of [T; N] properly drop all instances of T when an er…
VictorKoenders Oct 19, 2021
50fb8d6
Replaced the decode implementation of [T; N] with the implementation …
VictorKoenders Oct 21, 2021
eb8625d
Added dedicated error for `std::path::Path` encoding. Fixed broken li…
VictorKoenders Oct 21, 2021
0ff5199
Fixed a copy-paste error where RwLock would report to be a Mutex if i…
VictorKoenders Oct 21, 2021
d769016
Changed bincode version to 2.0.0-alpha.0
VictorKoenders Oct 21, 2021
0eda6fc
Made all encode and decode functions always require a Config, removed…
VictorKoenders Oct 21, 2021
55eedc8
Joined the 2 Sealed traits into a single one
VictorKoenders Oct 21, 2021
f68ab2a
Updated documentation of Encode, added an example to lib.rs
VictorKoenders Oct 21, 2021
bdb8e89
Updated to edition 2021
VictorKoenders Oct 21, 2021
b4b3240
Optimize performance of slice writing
ZoeyR Oct 22, 2021
99806d3
Optimize performance of decoding u8 arrays
ZoeyR Oct 22, 2021
5f570c3
Reintroduce varint optimizations
ZoeyR Oct 24, 2021
157e60f
Fixed clippy lint, removed function that was only used in 1 place
VictorKoenders Oct 24, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
First version of the bincode serialization spec
VictorKoenders committed Sep 13, 2021
commit b1ebecea2cf73e554eff01e0a7f873bdca704970
103 changes: 103 additions & 0 deletions docs/spec.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
# Serialization specification
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ZoeyR could you review this document to see if I missed something?


*NOTE*: Serialization is done by `bincode_derive` by default. If you enable the `serde` flag, serialization is done by `serde-derive` instead. `serde-derive` has the same guarantees as `bincode_derive` for now.

Related issue: https://github.com/serde-rs/serde/issues/1756#issuecomment-689682123

## Basic types

Boolean types are encoded with 1 byte for each boolean type, with `0` being `false`, `1` being true. Whilst deserilizing every other value will throw an error.

All basic numeric types will be encoded based on the configured [IntEncoding](#IntEncoding).

All floating point types will take up exactly 4 (for `f32`) or 8 (for `f64`) bytes.

All tuples have no additional bytes, and are encoded in their specified order, e.g.
```rs
let tuple = (u32::min_value(), i32::max_value()); // 8 bytes
let encoded = bincode::encode_to_vec_with_options(&tuple, Options::default().with_fixint_encoding()).unwrap();
assert_eq!(encoded.as_slice(), &[
0, 0, 0, 0, // 4 bytes for first type: u32
255, 255, 255, 255 // 4 bytes for second type: i32
]);
```

## IntEncoding
Bincode currently supports 2 different types of `IntEncoding`:

### VarintEncoding
Encoding an unsigned integer v (of any type excepting u8) works as follows:

1. If `u < 251`, encode it as a single byte with that value.
1. If `251 <= u < 2**16`, encode it as a literal byte 251, followed by a u16 with value `u`.
1. If `2**16 <= u < 2**32`, encode it as a literal byte 252, followed by a u32 with value `u`.
1. If `2**32 <= u < 2**64`, encode it as a literal byte 253, followed by a u64 with value `u`.
1. If `2**64 <= u < 2**128`, encode it as a literal byte 254, followed by a u128 with value `u`.

See the documentation of [VarintEncoding](https://docs.rs/bincode/latest/bincode/config/struct.VarintEncoding.html) for more information.

### FixintEncoding

- Fixed size integers are encoded directly
- Enum discriminants are encoded as u32
- Lengths and usize are encoded as u64

See the documentation of [FixintEncoding](https://docs.rs/bincode/latest/bincode/config/struct.FixintEncoding.html) for more information.

## Enums

Enums are encoded with their variant first, followed by optionally the variant fields. The variant index is based on the `IntEncoding` during serilization.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should explicitly mention that it's serialized as a u32


Both named and unnamed fields are serialized with their values only, and therefor encode to the same value.

```rs
#[derive(bincode::Serialize)]
pub enum SomeEnum {
A,
B(u32),
C { value: u32 },
}

// SomeEnum::A
let encoded = bincode::encode_to_vec_with_options(&SomeEnum::A, Options::default().with_fixint_encoding()).unwrap();
assert_eq!(encoded.as_slice(), &[
0, 0, 0, 0, // first variant, A
// no extra bytes because A has no fields
]);

// SomeEnum::B(0)
let encoded = bincode::encode_to_vec_with_options(&SomeEnum::B(0), Options::default().with_fixint_encoding()).unwrap();
assert_eq!(encoded.as_slice(), &[
0, 0, 0, 1, // first variant, B
0, 0, 0, 0 // B has 1 unnamed field, which is an u32, so 4 bytes
]);

// SomeEnum::C { value: 0u32 }
let encoded = bincode::encode_to_vec_with_options(&SomeEnum::C { value: 0u32 }, Options::default().with_fixint_encoding()).unwrap();
assert_eq!(encoded.as_slice(), &[
0, 0, 0, 2, // first variant, C
0, 0, 0, 0 // C has 1 named field which is a u32, so 4 bytes
]);
```

# Collections

Collections are encoded with their length value first, following by each entry of the collection. The length value is based on your `IntEncoding`.

```rs
let list = vec![
0u8,
1u8,
2u8
];

let encoded = bincode::encode_to_vec_with_options(&list, Options::default().with_fixint_encoding()).unwrap();
assert_eq!(encoded.as_slice(), &[
0, 0, 0, 0, 0, 0, 0, 3, // length of 3u64
0, // entry 0
1, // entry 1
2, // entry 2
]);
```

This also applies to e.g. `HashMap`, where each entry is a [tuple](#Basic%20types) of the key and value.