First draft: caller unsafe

[agocke]

No description provided.

[richlander]

How about for a binary? Should we set UnverifiableCodeAttribute or similar?

[agocke]

Good question. In this design it’s allowed to define a RequiresUnsafe method without an unsafe block if you never actually call the method. If you did, there would be an unsafe block and the assembly would have unverifiable code. I’m not sure if defining such a method, but never using it except through other callers unsafe methods, should be unverifiable.

My inclination is yes: anything that contains unsafe code should be marked, even if it’s not called from safe code.

[jaredpar]

The sentence says project. That should be very simple to achieve: does the project have AllowUnsafeBlocks or not? That is the barrier today for allowing unsafe usage and that should not change. If the desire is to detect in the binary of we had this attribute or not then yes we'd need to add more metadata.

[richlander]

It would be great to see an example of this, a method that is unsafe with the CallerUnsafe attribute. I'd like to see examples of obligations that can only be represented in a comment and how they are discharged in a caller. I'm certain that readers would find such an example very informative.

[richlander]

ArrayPool.Rent has two sides to uninitialized memory. As the new "renter", I cannot see signs of the old renter and the older renter cannot see signs of what I'm doing.

Staying with renters ... we want to ensure that old and new renters cannot post to social media about the bizarre paint choices of the other ... assuming that the landlord paints the walls to an unappealing beige between tenants, as a matter of policy (and the door keys are never exchanged until the paint has dried).

[richlander]

It might be worth some time thinking forward to policies that organizations with strict security policies might have given these new mechanisms.

For example, I could imagine audit being expanded to block on restore of packages with unsafe code, either absolutely or allowing unsafe code from only certain publishers. Similarly, I could imagine CI breaks on new unsafe blocks.

Unsafe code is a reality, in all general purpose programming languages. It's a question of what the default posture is and how that gets implemented in everyday workflows.

Some of that is out of scope of this doc. It's entirely possible that "easy way to identify" covers all the cases. It's worth a bit more thought. The workflows around safety don't end at the compiler.

[jkotas]

I agree we need an attribute like this to encode the fact that the callers must be called unsafe.

However, I think we should strongly consider changing the rules for unsafe keyword so that it works more sensible and more similar to Rust. If one sees void unsafe M() { ... }, it is natural to think that the method signature is unsafe and that the callers must be unsafe. I had to explain many times that it is not the case today. I am proposing that we change this to mean that the method is annotated with [RequiresUnsafe] void unsafe M() { ... }.

If we do not do this, we will have to be explaining why unsafe in C# is non-sensical and different from Rust. When we have to be explaining, we are failing.

[agocke]

I agree that if we were starting from scratch, the retrofit is simpler. Starting from where we are now, I’m not so convinced.

Let’s say we add a new project flag that makes all uses of unsafe on members or types mean RequiresUnsafe. That implies that both the producer and consumer need to opt-in to see any changes. Producer is not hard — the core libraries will be updated to set the flag and the next .NET version will produce appropriate metadata.

The consumer side is not simple. Because the scope is all-or-nothing, consumers will have to opt-in to the breaking change. Because the breaking change severity is so high, we will never be able to enable the feature by default. Because the feature is all-or-nothing we will also have no way to force errors for new code, even if there’s no breaking change risk.

Conversely, if we go with the attribute approach we gain two degrees of freedom. We can both opt-in individual methods, and we can force new errors without consumer involvement. That would allow us to make new unsafe APIs always error.
We could also potentially consider breaking changes for specific API surface.

Even six years later, adoption of nullable isn’t universal, and I’m not sure it ever will be at this point. This feature is far less prominent that nullable and I expect even fewer users will go out of their way to enable it.

Lastly, from an operational perspective, if we need the unsafe modifier to cause an attribute to be emitted, we’re now forced to implement this feature in the compiler and we’ll have to do language design review for the language changes. The analyzer approach is lighter weight and could be implemented by any team with spare bandwidth.

[jkotas]

Yes, I agree that the adoption will take a while. There is a good chance that opt-in into this feature will become mandatory for Microsoft 1P code at some point that would increase the adoption significantly.

I do not think enabling errors for new APIs and keeping existing APIs as warnings is interesting. I do not expect that we are going to grow unsafe API surface a lot, so the number of APIs on the error plan would be minimal in foreseeable future. If we wanted to create groups of APIs (I am not a fan - it makes the story more complicated), it should probably be on the same plan as Obsolete.

we’ll have to do language design review for the language changes

I think it is a good thing. unsafe has been a part of the language since forever, and any extensions of it should be reviewed starting from that context. We really need to build fully aligned and coherent story here. The lack of alignment between language, runtime and libraries is why we are looking into this.

[jaredpar]

That implies that both the producer and consumer need to opt-in to see any changes

I think this is the only rational way to approach the problem. The number of APIs that require unsafe to call is going to increase quite significantly compared to today. Don't see any way to do this without the ability for the consumer to opt in separately from the producer. And I think that is totally fine. As long as it's easy for us to check in build systems and productions if a build / binary has opted into the new version of unsafe then we can use tooling / policy to push parties to move to the new model. For 3P customers who are uninterested in the new unsafe they can remain as they are today without changes.

[jkotas]

Is the idea going to be that WarnByDefault is effectively going to mean: API introduced before .NET 10 (or whatever release we are going to skip this in)? I do not think we want to introduce a wart like this that will have to live in the API surface forever. Also, it is probably going to problematic for projects that multi-target.

I think that it would be better to deal with the compatibility concern at project level, similar how it was done for nullability. My proposal would be to extend existing AllowUnsafeBlocks property (or introduce a new msbuild property) with a new value that opts-in the project into a new unsafe policy where RequiresUnsafe annotations are respected, unsafe in the method signature implies RequiresUnsafe annotation, etc.

[rolfbjarne]

All P/Invokes would also be unsafe, even ones that don’t need the unsafe keyword.

Even a P/Invoke with no return value or parameters is unsafe:

[DllImport ("libc")]
static extern void strlen();

This will crash when called.

[richlander]

I don't think of operations that reliably crash as unsafe. I would assert that stack overflow is safe in .NET because StackOverflowException is uncatchable and reliably causes a crash/fail fast.

Crashing is the same as a fuse that trips (and cuts the power), which prevents home wiring from transitioning from warm to burny temps. I love analogies since they enable us to see our problems from a distance in another domain. Do you agree with this one?

That said, in previous conversations I believe we consider all P/Invokes (as you suggest) as unsafe. It is hard to consider them otherwise. There is no way to "discharge obligations" at a static extern method signature.

[jkotas]

In this document memory safety is strictly defined as: safe code can never acquire a reference to memory that is not managed by the application.

This does not look like a sufficient definition. Consider the following example:

M(Random.Shared.Next());

void M(int x)
{
    unsafe
    {
        *(int*)x = 42;
    }
}

The safe code never acquired a reference to memory that is not managed by the application in this example, but this code has major memory safety issue.

[agocke]

You’re right, the definition needs to be more precise. It’s along the lines of: the above definitions are for valid code. All code must meet those definitions to be considered valid. Unsafe code is merely code that doesn’t have validity checked and enforced by the compiler/runtime. I’ll need to wordsmith this. (And yes, I’m aware this is retreading our definitions of “verifiable”, but I’m not sure we want to import that legacy wholesale).

[jkotas]

@jaredpar started writing a draft for this as well some time ago: https://github.com/jaredpar/csharplang/tree/unsafe .

(It proposes to reuse the unsafe keyword that I think is much cleaner design.)

[jaredpar]

This is the critical part.