Skip to content

Commit 81a8a87

Browse files
alercahalercah
committed
Union current state. II. Validity.
Closes #352.
1 parent 36661ec commit 81a8a87

File tree

1 file changed

+236
-6
lines changed

1 file changed

+236
-6
lines changed

‎reference/src/validity/unions.md

+236-6
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,243 @@
11
# Validity of unions
22

3-
**Disclaimer**: This chapter is a work-in-progress. What's contained here
4-
represents the consensus from issue [#73]. The statements in here are not (yet)
5-
"guaranteed" not to change until an RFC ratifies them.
3+
**Disclaimer**: This chapter is a work-in-progress.
4+
What's contained here represents the consensus from [various issues][union
5+
discussion].
6+
The statements in here are not (yet) "guaranteed" not to change until an RFC
7+
ratifies them.
68

7-
## Validity of unions with zero-sized fields
9+
[union discussion]: https://github.com/rust-lang/unsafe-code-guidelines/blob/master/active_discussion/unions.md
810

9-
A union containing a zero-sized field can contain any bit pattern. An example of such
10-
an union is [`MaybeUninit`].
11+
**Note**: For ease of reading the examples, the hypothetical type `Padded<T>` is
12+
used, which behaves identically to `T` except that writing to it clobbers any
13+
subsequent padding.
14+
This could actually be accomplished using an overaligned newtype struct, but
15+
that would make the examples harder to read.
16+
Additionally, layout of union fields is put in some comments; in these comments,
17+
the notation `n*pad` means "`n` padding bytes".
1118

19+
## Value model
20+
21+
The possible values of unions are not defined in terms of the values of the
22+
fields, but rather, a union's possible values are lists of bytes.
23+
The [representation relation] is trivial in both directions, except for [padding
24+
bytes][padding byte] which are uninitialized in all values.
25+
26+
<details><summary><b>Rationale</b></summary>
27+
28+
The following examples must be supported, and therefore impose constraints on union value behaviour. The simplest solution, by far, is to treat the value representation of unions as merely being lists of bytes. While we do not discuss every possible angle here, it should be rapidly clear from just these two examples that any other example is significantly more complicated. Please trust us that every other alternatives that sounds even half-reasonable has been examined and has some critical flaw or another.
29+
30+
**We cannot require all fields, or even only one field, to be valid at all times.**
31+
32+
```rust
33+
union Padding {
34+
left: (Padded<u8>, u16), // [[u8, 1*pad], u16]
35+
right: (u16, Padded<u8>), // [u16, [u8, 1*pad]]
36+
}
37+
let p = Padding{left: (0, 0)}; // resulting bytes: [0, uninit, 0, 0]
38+
p.right.1 = 1; // resulting bytes: [0, uninit, 1, uninit]
39+
fn f(_: Padding) {}
40+
f(p);
41+
```
42+
43+
By the end of this example, the resulting union has no valid fields, because every field contains uninit, non-padding bytes. So therefore no field is even fully initialized, despite the fact that `p` was fully initialized as can be witnessed by the fact that the compiler allows it to be moved into `f`. This is all stable, Safe Rust, so these semantics being sound are a hard constraint.
44+
45+
**Unions must preserve provenance.**
46+
47+
```rust
48+
union Provenance<'a> {
49+
raw: *const u32,
50+
reference: &'a u32,
51+
}
52+
let x: u32;
53+
let u = Provenance(raw: &x);
54+
let y = unsafe{ *u.reference };
55+
```
56+
57+
We must be able to carry provenance between the `raw` and `reference` fields in order for the assignment via `u.reference` to be valid. While this uses Unsafe Rust, this code is "obviously" sound. And therefore the union must be able to maintain provenance between the two pointers---even should the pointers be nested deeply within structs.
58+
59+
[representation relation]: ../glossary.md#representation-relation
60+
[padding byte]: ../glossary.md#padding-byte
61+
62+
</details>
63+
64+
### Niches
65+
66+
## Valid values for #[repr(C)] and Raw-repr unions
67+
68+
`#[repr(C)]` and [Raw-repr][raw repr] unions can take on any byte value.
69+
70+
<details><summary><b>Rationale</b></summary>
71+
72+
The purpose of the Raw-repr is to provide these semantics, which are easy to raeson about. Furthermore, C programmers are used to being able to treat unions like bags of bytes, more or less, and Rust programmers are similarly used to the same with `#[repr(C)]` unions. Therefore, they should both accept any arbitrary byte pattern.
73+
74+
</details>
75+
76+
## Possible niche values
77+
78+
The presence of padding bytes, and writes to individual fields in general, makes niches hard to come by in unions. A niche representation of a union would have to not only be invalid for every single one of its fields, but also impossible to construct in Safe Rust with any combination of writes to any of its fields.
79+
80+
For reprs other than `#[repr(C)]` and the [Raw-repr], values not constructible from safe Rust are consequently [**TBD**][#73] whether or not they are valid. The following example assumes that `#[repr(Rust)]` is not the Raw-repr:
81+
82+
```rust
83+
#[repr(Rust)]
84+
union U {
85+
a: (u16, u16),
86+
b: u32,
87+
}
88+
MaybeUninit<U>::uninit().assume_init(); // Unsound: assumes that U can be uninit
89+
fn get_b(u: U) -> u32 {
90+
unsafe { u.b } // Unsound: assumes that U cannot be uninit
91+
}
92+
let u: U;
93+
get_b(u); // Compile error: u is not initialized.
94+
```
95+
96+
Because all bytes of `U` must be initialized for the value to be valid, and this is enforced by the compiler's initialization checks, it might be tempting to assume that `U`'s bytes must always be defined, but this is not a valid assumption. It is equally invalid, however, to assume that `U`'s bytes can be undefined.
97+
98+
<details><summary><b>Rationale</b></summary>
99+
100+
We have not yet reached consensus on whether or not we wish to leave the door open for the possibility that unions with safe field access, or `#[repr(transparent)]` unions with no ZSTs, contain niches:
101+
102+
```rust
103+
#[repr(transparent)]
104+
union U { b: bool };
105+
assert_eq!(size_of::<Option<U>>(), 1); // Requires a niche, which in turn requires that U must be initialized to be valid.
106+
```
107+
108+
We are **not** describing this case as unspecified, but instead as TBD."Constructible with Safe Rust" is a poorly-defined and very complex invariant, which falls short of the UCG's goals of easily checked, easy to understand (such as it were) semantics, and therefore we are not comfortable leaving the language in this state on an indefinite basis.
109+
110+
The main saving grace here is that `#[repr(Rust)]` unions are presently nearly impossible to use correctly anyways, because they do not even guarantee fields at offset 0.
111+
112+
</details>
113+
114+
## Validity of sometimes-padding bytes
115+
116+
We can say that a byte is *sometimes padding* for a union `U` if there is *some* inhabited field `f` such that the byte is either padding for `f` or not a part of `f`.
117+
118+
In that case, the byte will be uninitialized in the value `U{f: /* some value */ }`. By the [monotonicity property], therefore, all sometimes-padding bytes can contain any byte value, be it undefined or any bit pattern with any provenance. Likewise, if multiple bytes are padding for the same field, then they can take on any possible combination of byte values between them.
119+
120+
It follows that a union containing an inhabited zero-sized field can contain any bit pattern whatsoever, because all bytes are sometimes-padding bytes. An example of such
121+
an union is [`MaybeUninit<T>`], which is a union of `T` and `()`.
122+
123+
As per the previous section, however, just because a byte is a sometimes-padding byte does not mean it can always safely be set to uninitialized (or any other value), if this can produce a value not reachable from Safe Rust.
124+
125+
For instance, the following is presently unsound (assuming that `#[repr(Rust)]` is not the [Raw-repr]), even assuming that all fields are placed at offset 0:
126+
127+
```rust
128+
struct B {
129+
130+
}
131+
#[repr(Safe)]
132+
Union u {
133+
b: bool, // [bool, 1*pad]
134+
u: u16, // [u16]
135+
}
136+
let u = U{u: (0xff00, 0}; // resulting bytes: [0xff, 0]
137+
unsafe { (&mut u.b as *mut u8 as *mut MaybeUninit<u8>).offset(1).write(MaybeUninit::uninit()) }; // resulting bytes: [0xff, uninit]
138+
```
139+
140+
This value is impossible to reach in Safe Rust: the only way to write uninit to the padding is to write to the boolean field. Writing to the integer field must initialize
141+
142+
## Safety invariants of unions
143+
144+
Unions currently provide *no* safety invariants of any kind. Without a documented safety invariant for a union type, code cannot make any assumptions about a union passed in from untrusted code, other than that it has a valid value, and it cannot pass a union value to untrusted code unless it could do so in purely Safe Rust.
145+
146+
In particular, regardless of the union's repr, it is not safe to assume that a union's field can be safely accessed, even if it seems "obviously" safe.
147+
148+
```rust
149+
// Crate a
150+
pub union U {
151+
pub i: i32
152+
}
153+
// Crate b
154+
pub fn get_i(u: a::U) -> i32 {
155+
// Safe: u.i cannot be uninit in Safe Rust.
156+
unsafe { u.i } // UNSOUND!
157+
}
158+
```
159+
160+
Making this field access safe would require additional an safety invariant that can be understood by the compiler. The UCG WG does not oppose such a safety invariant, but believes it should be opt-in, and an RFC for such a feature is beyond our remit.
161+
162+
<details><summary><b>Rationale</b></summary>
163+
164+
At first blush, it may appear that the crate `b` is entitled to assume that it is being called from Safe Rust, or from unsafe Rust following the rules of Safe Rust. It then seems to follow that `u.i` must always be initialized, since the only safe way to create a value of type `U` is to initialize it with a value for `i`.
165+
166+
One might analogize this to the corresponding code with a struct:
167+
168+
```rust
169+
// Crate a
170+
pub struct S {
171+
pub i: i32
172+
}
173+
// Crate b
174+
pub fn get_i(s: a::S) -> i32 {
175+
// Safe: s.i cannot be uninit in Safe Rust.
176+
unsafe { s.i } // Sound.
177+
}
178+
```
179+
180+
This struct code, however, is absolutely sound, even in the absence of a safety invariant documented by `S`, because of `S`'s validity invariant: for `S` to be valid, all its fields must be valid, and therefore `i` must be initialized. If it weren't, the definition of `get_i` wouldn't be the problem: the caller would be committing UB by passing an uninitialized `S`. Consequently, the `unsafe` block is redundant.
181+
182+
But for the union type `U`, its validity invariant is not transitive to its fields. `u.i` has no guarantee of validity for `U` to be valid.
183+
184+
Okay, so what about a field with a safety invariant that is stricter than the validity invariant?
185+
186+
```rust
187+
// Crate a
188+
pub struct S<'a> {
189+
pub s: &'a str
190+
}
191+
// Crate b
192+
pub fn get_s(s: a::S<'_>) -> String {
193+
// Safe: s.s must be UTF-8 in Safe Rust.
194+
unsafe { String::from_utf8_unchecked(s.s.as_bytes().to_owned()) } // Sound.
195+
}
196+
```
197+
198+
Now we are relying on a safety invariant separate from the validity invariant: that `str` must be UTF-8. So isn't this like our union example, where we're relying on the safety invariant that `i32` can't be uninit? No, because union fields are unsafe.
199+
200+
Consider the following three types:
201+
202+
```rust
203+
static invalid_utf8: [u8; 1] = [0xff];
204+
pub struct Sound<'a> {
205+
s: &'a str
206+
}
207+
pub struct Unsound<'a> {
208+
pub s: &'a str
209+
}
210+
pub union AlsoSound<'a> {
211+
pub s: &'a str
212+
}
213+
impl<'a> Sound<'a> {
214+
pub fn new() -> Self {
215+
Self { s: str::from_utf8_unchecked(&invalid_utf8) }
216+
}
217+
}
218+
impl<'a> Unsound<'a> {
219+
pub fn new() -> Self {
220+
Self { s: str::from_utf8_unchecked(&invalid_utf8) }
221+
}
222+
}
223+
impl<'a> AlsoSound<'a> {
224+
pub fn new() -> Self {
225+
Self { s: str::from_utf8_unchecked(&invalid_utf8) }
226+
}
227+
}
228+
```
229+
230+
One struct type is `Sound`, the other is `Unsound`, and the only difference between the two is that `Unsound`'s field is `pub`. This lets us get to the heart of how safety invariants work for fields: untrusted Safe Rust cannot be allowed to get its hands on a `str` with invalid UTF-8. If it could do that, it could pass it off to arbitrary unsafe Rust that *does* assume that the `str` has UTF-8, such as the `str::chars()` method. And that will cause UB.
231+
232+
Thus, every struct implicitly has a safety invariant that all of its `pub` fields are safe.
233+
234+
The union `AlsoSound`, is identical to `Unsound` except for being a union, but it is sound. And the reason it doesn't break the rules is that union field access is unsafe. Safe Rust can call `Unsound::new().s.chars()`, but neither `Sound::new().s.chars()` nor `AlsoSound::new().s.chars()`.
235+
236+
It follows that unions have no safety invariants on their fields, even `pub` fields, except for those that are explicitly documented.
237+
238+
</details>
239+
240+
[Raw-repr]: ../layout/unions.md#raw-repr
12241
[#73]: https://github.com/rust-lang/unsafe-code-guidelines/issues/73
13242
[`MaybeUninit`]: https://doc.rust-lang.org/std/mem/union.MaybeUninit.html
243+
[monotonicity property]: https://github.com/RalfJung/minirust/blob/master/lang/values.md#generic-properties

0 commit comments

Comments
 (0)