Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[.NET] Add binary for web assembly to NuGet package #1204

Closed
Jack-Edwards opened this issue Aug 7, 2022 · 16 comments
Closed

[.NET] Add binary for web assembly to NuGet package #1204

Jack-Edwards opened this issue Aug 7, 2022 · 16 comments

Comments

@Jack-Edwards
Copy link

.NET 6 has supposedly made it very simple for Blazor WASM projects to refer to native code in the browser. Would it be possible to include a WASM binary in the libsodium NuGet package?

For reference, Steve Sanderson explains how SQLite recently accomplished this with their package: https://www.youtube.com/watch?v=lP_qdhAHFlg&t=724s

@jedisct1
Copy link
Owner

jedisct1 commented Aug 7, 2022

libsodium can easily be compiled to WebAssembly using the dist-build/wasm32-wasi.sh script. It uses Zig as a compiler, so that works on most platforms including Windows.

Now for the "include the wasm binary in the NuGet package" part, I have no idea how to do that.

@Jack-Edwards
Copy link
Author

Jack-Edwards commented Aug 8, 2022

I'm having some difficulty compiling to WASM using the dist-build/wasm32-wasi.sh script in the stable branch.

  • Ubuntu 20.04.
  • Fresh installs of make and clang through apt.
  • Installed zig from the snap store, using --classic --beta flags.
  • Invoked dist-build/wasm32-wasi.sh from the root of the repo.

The process runs to completion, but every test fails. Logs attached.

Looking at the diff for #1174 , would including a WASM binary in the NuGet package only be a matter of adding another procedure to the dotnetcore-yml file and adding the output as a reference to the libsodium.pkgproj file?

logs.zip

@jedisct1
Copy link
Owner

jedisct1 commented Aug 9, 2022

Running webassembly modules requires a webassembly runtime.

So, in order to run the test suite, a runtime such as wasmer, wavm or wasmedge needs to be installed.

@Jack-Edwards
Copy link
Author

Installing a runtime did the trick. I am referring to output in my project's .csproj file like this:

<ItemGroup>
   <NativeFileReference Include="Native\libsodium.a" />
</ItemGroup>

However, when I make a call to randombytes_buf through the libsodium-core C# language binding, I receive this error message: missing function: arc4random_buf. Not sure what to make of it yet.

@Jack-Edwards
Copy link
Author

Jack-Edwards commented Aug 21, 2022

Libsodium requires either the emscripten or wasi environments, as it's not even possible to generate random numbers in standalone WebAssembly.

Source

If this comment still describes the current state of Web Assembly, then is trying to get a native version of libsodium (libsodium.a) running in the browser a fool's errand? Do I need to be pointed in the direction of emscripten.sh and the libsodium.js that it outputs? Given the issue I encountered with arc4random_buf not being found.

Edit: It appears the error I encountered appears as early as linking the libsodium.a from the wasm32-wasi script to my .net project:

warning : undefined symbol: arc4random (referenced by top-level compiled C/C++ code)
warning : undefined symbol: arc4random_buf (referenced by top-level compiled C/C++ code)

build-warnings.txt

@jedisct1
Copy link
Owner

randombytes_buf is part of WASI, a set of common functions for WebAssembly.

Linking a WASI implementation would be the best way to go. And there has to be ways to do that in .NET.

@Jack-Edwards
Copy link
Author

dotnet/runtime#74352

@Jack-Edwards
Copy link
Author

Jack-Edwards commented Sep 6, 2022

There has been some discussion on the linked issue. Sounds complicated.

Will you elaborate on how we get from arc4random to randombytes? I think I understand that arc4random is a native function capable of producing random bytes, but that doesn't appear to be available in the browser.

@Jack-Edwards
Copy link
Author

Linking a WASI implementation would be the best way to go. And there has to be ways to do that in .NET.

Is this what you are referring to by "linking a WASI implementation"? Someone would have implemented their own "random" code that works in the browser and I would instruct Libsodium to use that implementation?

If a non-default implementation is being used (see randombytes_set_implementation()), randombytes_stir() must be called by the child after a fork() call.

Source: https://doc.libsodium.org/generating_random_data

@jedisct1
Copy link
Owner

Not sure why you keep referring to randombytes_buf() and arc4random()?

WASI adds the following functions to a WebAssembly environment: https://github.com/WebAssembly/WASI/blob/main/phases/snapshot/docs.md

There's indeed a function that returns random data, that is absolutely required for libsodium.

But is it the only one? I expect at least clock_time_get() to be also required. Probably more, and that can change in the future.

@jedisct1
Copy link
Owner

Maybe wasmer-js can help?

@Jack-Edwards
Copy link
Author

I think your latest comments really help clear this up for me. I was struggling to understand why WASI was relevant in the context of a WASM application running in the browser. My understanding of WASI is that it was a technology intended for non-browser scenarios.

But it sounds like the libsodium wasm-wasi build is intended for both browser and non-browser scenarios alike, assuming a WASI implementation is present (or linked, as you commented earlier).

I'm eager to see if wasmer-js can help. I assume I just need to initialize the library on the webpage and everything else should just begin to work? wasmer-js initialization

Thank you for your help. There are a few other folks who would also like to get libsodium working as a native dependency in their Blazor WASM projects; they have been commenting on the nsec C# repo It will be nice to get this working.

@Jack-Edwards
Copy link
Author

Jack-Edwards commented Sep 26, 2022

Here is the latest on using wasmer-js.

I don't think I can "load" the libsodium.a output of the wasm32-wasi.sh script into the wasmer-js wasi runtime. Wasmer-js expects a .wasm file, not an archive file. If I try to load the file anyway, I get this error:

Uncaught (in promise) CompileError: wasm validation error: at offset 4: failed to match magic number

It seems like the next option would be to somehow get the entire Blazor WASM application loaded into the wasmer-js wasi runtime. This way I can continue referring to libsodium.a as if it were library code. Still above my head, but I'll keep digging.

@Jack-Edwards
Copy link
Author

Commenting here since @ektrah wants to use this issue as the central location for discussing a Blazor WASM solution.

I really don't think using the wasm32-wasi package is the right way to go for Blazor WASM. It would have been really convenient to use the libsodium.a file from the wasm32-wasi build script as a "NativeFileReference" in Blazor, but that's not possible due to the WASI dependency. Blazor WASM is primarily a browser framework and I think that's where most of the demand is coming from.

wasmer-js could work if we were really inclined, but I don't see any benefit to that over using libsodium-js since both methods are going to require JSInterop on every single call into libsodium (or so I think; I'm not equipped to write alternative bindings for .wasm modules).

I started working on a small proof-of-concept to show how we could wrap a .NET class library over libsodium-js, but I'm having trouble working with the buf data type. It looks like JSInterop just doesn't know what to do. See here: JSInterop output when we try to get a buf result

Maybe JsonElement isn't the right type to deserialize to, but that's where .NET was leading me. The same problem occurs when I try providing a List<byte>, List<uint>, byte[], or uint[] to any function with a buf argument. The browser tells me I have provided something of the wrong type.

Running the example should be as simple as setting up npm and running the demo project.

@Jack-Edwards
Copy link
Author

I got buf to byte[] working!

For reference: dotnet/runtime#76672

@Jack-Edwards
Copy link
Author

@jedisct1 Thanks for putting up with me. I'd be fine closing this issue now that I have a path forward and discovered the impossibility of asking you (or other core maintainers) to provide a nuget package for Blazor WASM. I'll continue working on my "BlazorSodium" project for now.

Repository owner locked and limited conversation to collaborators Feb 22, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants