[.NET] Add binary for web assembly to NuGet package

[Jack-Edwards]

.NET 6 has supposedly made it very simple for Blazor WASM projects to refer to native code in the browser. Would it be possible to include a WASM binary in the libsodium NuGet package?

For reference, Steve Sanderson explains how SQLite recently accomplished this with their package: https://www.youtube.com/watch?v=lP_qdhAHFlg&t=724s

[jedisct1]

libsodium can easily be compiled to WebAssembly using the dist-build/wasm32-wasi.sh script. It uses Zig as a compiler, so that works on most platforms including Windows.

Now for the "include the wasm binary in the NuGet package" part, I have no idea how to do that.

[Jack-Edwards]

I'm having some difficulty compiling to WASM using the dist-build/wasm32-wasi.sh script in the stable branch.

• Ubuntu 20.04.
• Fresh installs of make and clang through apt.
• Installed zig from the snap store, using --classic --beta flags.
• Invoked dist-build/wasm32-wasi.sh from the root of the repo.

The process runs to completion, but every test fails. Logs attached.

Looking at the diff for #1174 , would including a WASM binary in the NuGet package only be a matter of adding another procedure to the dotnetcore-yml file and adding the output as a reference to the libsodium.pkgproj file?

logs.zip

[jedisct1]

Running webassembly modules requires a webassembly runtime.

So, in order to run the test suite, a runtime such as wasmer, wavm or wasmedge needs to be installed.

[Jack-Edwards]

Installing a runtime did the trick. I am referring to output in my project's .csproj file like this:

<ItemGroup>
   <NativeFileReference Include="Native\libsodium.a" />
</ItemGroup>

However, when I make a call to randombytes_buf through the libsodium-core C# language binding, I receive this error message: missing function: arc4random_buf. Not sure what to make of it yet.

[Jack-Edwards]

Libsodium requires either the emscripten or wasi environments, as it's not even possible to generate random numbers in standalone WebAssembly.

Source

If this comment still describes the current state of Web Assembly, then is trying to get a native version of libsodium (libsodium.a) running in the browser a fool's errand? Do I need to be pointed in the direction of emscripten.sh and the libsodium.js that it outputs? Given the issue I encountered with arc4random_buf not being found.

Edit: It appears the error I encountered appears as early as linking the libsodium.a from the wasm32-wasi script to my .net project:

warning : undefined symbol: arc4random (referenced by top-level compiled C/C++ code)
warning : undefined symbol: arc4random_buf (referenced by top-level compiled C/C++ code)

build-warnings.txt

[jedisct1]

randombytes_buf is part of WASI, a set of common functions for WebAssembly.

Linking a WASI implementation would be the best way to go. And there has to be ways to do that in .NET.

[Jack-Edwards]

dotnet/runtime#74352

[Jack-Edwards]

There has been some discussion on the linked issue. Sounds complicated.

Will you elaborate on how we get from arc4random to randombytes? I think I understand that arc4random is a native function capable of producing random bytes, but that doesn't appear to be available in the browser.

[Jack-Edwards]

Linking a WASI implementation would be the best way to go. And there has to be ways to do that in .NET.

Is this what you are referring to by "linking a WASI implementation"? Someone would have implemented their own "random" code that works in the browser and I would instruct Libsodium to use that implementation?

If a non-default implementation is being used (see randombytes_set_implementation()), randombytes_stir() must be called by the child after a fork() call.

Source: https://doc.libsodium.org/generating_random_data

[jedisct1]

Not sure why you keep referring to randombytes_buf() and arc4random()?

WASI adds the following functions to a WebAssembly environment: https://github.com/WebAssembly/WASI/blob/main/phases/snapshot/docs.md

There's indeed a function that returns random data, that is absolutely required for libsodium.

But is it the only one? I expect at least clock_time_get() to be also required. Probably more, and that can change in the future.

[jedisct1]

Maybe wasmer-js can help?

[Jack-Edwards]

I think your latest comments really help clear this up for me. I was struggling to understand why WASI was relevant in the context of a WASM application running in the browser. My understanding of WASI is that it was a technology intended for non-browser scenarios.

But it sounds like the libsodium wasm-wasi build is intended for both browser and non-browser scenarios alike, assuming a WASI implementation is present (or linked, as you commented earlier).

I'm eager to see if wasmer-js can help. I assume I just need to initialize the library on the webpage and everything else should just begin to work? wasmer-js initialization

Thank you for your help. There are a few other folks who would also like to get libsodium working as a native dependency in their Blazor WASM projects; they have been commenting on the nsec C# repo It will be nice to get this working.

[Jack-Edwards]

Here is the latest on using wasmer-js.

I don't think I can "load" the libsodium.a output of the wasm32-wasi.sh script into the wasmer-js wasi runtime. Wasmer-js expects a .wasm file, not an archive file. If I try to load the file anyway, I get this error:

Uncaught (in promise) CompileError: wasm validation error: at offset 4: failed to match magic number

It seems like the next option would be to somehow get the entire Blazor WASM application loaded into the wasmer-js wasi runtime. This way I can continue referring to libsodium.a as if it were library code. Still above my head, but I'll keep digging.

[Jack-Edwards]

Commenting here since @ektrah wants to use this issue as the central location for discussing a Blazor WASM solution.

I really don't think using the wasm32-wasi package is the right way to go for Blazor WASM. It would have been really convenient to use the libsodium.a file from the wasm32-wasi build script as a "NativeFileReference" in Blazor, but that's not possible due to the WASI dependency. Blazor WASM is primarily a browser framework and I think that's where most of the demand is coming from.

wasmer-js could work if we were really inclined, but I don't see any benefit to that over using libsodium-js since both methods are going to require JSInterop on every single call into libsodium (or so I think; I'm not equipped to write alternative bindings for .wasm modules).

I started working on a small proof-of-concept to show how we could wrap a .NET class library over libsodium-js, but I'm having trouble working with the buf data type. It looks like JSInterop just doesn't know what to do. See here: JSInterop output when we try to get a buf result

Maybe JsonElement isn't the right type to deserialize to, but that's where .NET was leading me. The same problem occurs when I try providing a List<byte>, List<uint>, byte[], or uint[] to any function with a buf argument. The browser tells me I have provided something of the wrong type.

Running the example should be as simple as setting up npm and running the demo project.

[Jack-Edwards]

I got buf to byte[] working!

For reference: dotnet/runtime#76672

[Jack-Edwards]

@jedisct1 Thanks for putting up with me. I'd be fine closing this issue now that I have a path forward and discovered the impossibility of asking you (or other core maintainers) to provide a nuget package for Blazor WASM. I'll continue working on my "BlazorSodium" project for now.