cmd/compile: relax wasm/wasm32 function import signature type constraints

@johanbrandhorst

Background

#59149 removed the package restrictions on the use of go:wasmimport, but established strict constraints on the types that can be used as input and result parameters. The motivation for this was that supporting rich types between the host and the client would require sophisticated and expensive runtime type conversions because of the mismatch between the 64 bit architecture of the client and the 32 bit architecture of the host.

With the upcoming 32 bit wasm port, this problem goes away, as both client and host will use 32 bit pointers. However, we can also support a limited set of types on 64 bit platforms, where no runtime conversion is necessary.

Proposal

Relax the constraints on types that can be used as input and result parameters with the go:wasmimport compiler directive. The exact allowed types would depend on whether wasm or wasm32 is used.

We define the "small integer types" group as the set of types described by [u]int[8|16]. The following types would be allowed as input parameters to any wasmimport/wasmexport function:

bool
int32, uint32, int64, uint64
float32, float64
string
uintptr, unsafe.Pointer, *T where T is an allowed parameter type or one of the small integer types.
*struct. All fields of the *struct must be allowed parameter types, struct, [...]T, or one of the small integer types.
- If the struct has any fields, it must embed structs.HostLayout (see structs: add HostLayout "directive" type #66408).
- Any struct fields must also embed structs.HostLayout (recursively).
- *T, unsafe.Pointer, uintptr, and string types are only allowed in *struct on wasm32.
*[...]T where T is an allowed type or one of the small integer types.

All input parameter types except string are also allowed as result parameter types.

The following types would remain disallowed as input and output parameter types:

chan T
complex64, complex128
func
interface
map[T]U
[]T
struct
[...]T

The conventions established for use of pointers in CGO will be required when using pointers with wasmimport/wasmexport, e.g. the host can read Go memory, can write pointerless data (like the content of a byte buffer) but cannot write Go pointers to Go memory, and cannot hold on to Go pointers unless they are pinned.

Discussion

Compatibility guarantees

The Go spec does not specify the struct layout and leaves it up to implementations to decide. As such, we cannot provide a guaranteed ABI without having to change the spec or force future layout changes to provide runtime conversion of data. This proposal suggests making it clear to users through documentation that there are no guarantees of compatibility across versions of the Go compiler.

Type conversion rules

The following conversion rules would be automatically applied by the compiler for the respective parameter type:

Go Type	Parameter type (per Wasm spec)
`bool`	`i32`
`int32, uint32` `int64, uint64`	`i32, i32` `i64, i64`
`float32, float64`	`f32, f64`
`string`	Assigned to two call parameters as a `(i32, i32)` tuple of (pointer, len). Only allowed for input parameters.
`uintptr`, `unsafe.Pointer`, `T`, `struct`, `*[...]T`	`i32, i32, i32, i32, i32`

Strings

Strings are not allowed as result parameters as Wasm practically does not allow more than 1 result parameter.

Supporting `GOARCH=wasm`

The wasm architecture uses 64 bit pointers and integer sizes. As the host uses 32 bit pointers, this makes it impossible to allow certain types without costly runtime conversions, such as *struct types containing pointer fields. Since string types are also pointer types, *struct types containing string fields are also disallowed.

Supporting `[u]int`, `[u]int8`, `[u]int16` as concrete parameters

The [u]int types are problematic as the size of them are not precisely defined, and may cause confusion when used with strictly 32 bit or 64 bit integers. The [u]int8 and [u]int16 types are problematic because we would be forced to automatically convert them to/from the i32 wasm representation, with potential loss of precision or overflow. They are still allowed as pointer type, array elements and struct fields.

Supporting slices, maps

Both slices and maps are disallowed because of the uncertainty around the memory underlying these types and interactions with struct and array rules. Users who wish to use slices can manually use (&slice, len(slice)) or unsafe.Pointer. There is no clear way to support passing or returning map data from the host other than by using unsafe.Pointer and making assumptions about the underlying data.

Related proposals

struct.Hostlayout

#66408 proposes a way for users to request that struct layout is host compatible. Our proposal depends on the definitions put forward in this proposal for struct parameters.

Future work

WASI Preview 2 (AKA WASI 0.2)

WASI Preview 2 defines its API in terms of the Component Model, with a rich type system and an IDL language, WIT. The Component Model also defines a Canonical ABI with a specification for lifting and lowering Component Model types into and out of linear memory. This proposal does not attempt to define the ABI for any hypothetical wasip2 target, and would leave such decisions for any future wasip2 proposal.

Supporting `struct` and `[...]T` by value

A previous version of this proposal included support for passing struct and [...]T types by value by expanding each field recursively into call parameters. This was removed in favor of a simpler initial implementation but could be re-added if users require it.

Contributors

@johanbrandhorst, @evanphx, @achille-roussel, @dgryski, @ydnar

CC @cherrymui @golang/wasm

@gabyhelp's overview of this issue: #66984 (comment)

"This follows the C struct value semantics" is just a hair vague; are 8-byte quantities (float64, int64, uint64) stored at a 4-byte or 8-byte alignment? It was my understanding (and the purpose of #66408) to specify a ~~4-byte~~ alignment for fields of those types when they occur in structs passed to wasm32 (tagged structs.HostLayout).

(edited to note error, the host alignment for 8-byte integers and floats is 8 bytes).

Ideally 8-byte values would always be 8-byte aligned in the wasm32 port.

@dr2chase

@dr2chase Looking at what clang does, it uses 8-byte alignment on 64bit quantities so we'd match that.

You are right, I got it backwards. But that is what you are expecting for anything that has pointers-to-it passed to the wasm host platform, yes?

Thanks for the proposal! A few questions:

8-byte alignment for 64-bit values, as mentioned above. cmd/compile: create GOARCH=wasm32 #63131 doesn't seem to have a definitive answer, and currently it seems the CL doesn't implement 64-bit alignment.
If we don't always align 64-bit value to 8 bytes (which differs from current all 32-bit architectures we support and probably requires quite some work), we should align 64-bit value to 8 bytes when structs.HostLayout is specified. So structs: add HostLayout "directive" type #66408 is very related.
structs and arrays. What is the ABI specification exactly? The C ABI on, say ELF AMD64, is pretty complex for passing structs and arrays. Small fields may be packed into one word. Large structs may be passed indirectly (stored on stack, passing a pointer to the callee). Do we have a specification for this?
string. What does a string look like on Wasm/WASI side? I couldn't find its specification on WASI P1 doc. On Component Model doc https://github.com/WebAssembly/component-model/blob/main/design/mvp/CanonicalABI.md (which I guess is for WASI P2, not P1?), it specifies string is two i32, which is similar to Go's string, which is good. But also it allows three encodings, UTF-8, UTF-16, and "latin1+utf16" differentiated by a high bit. The second and third encoding are not compatible with Go strings. Do we require UTF-8 encoding? Or we don't allow passing Go strings directly?

Besides, for structs, arrays of structs, and pointer to structs, I would suggest we allow only structs with structs.HostLayout to be passed. The reason is that in the Go spec we don't require struct fields to be laid out in memory in source order, and it may well change in a future Go release. structs.HostLayout specifies a fixed layout. Structs without that marker can change. This gives a clear way to say which structs should have a fixed layout, which are okay to change.

Thanks.

Two other questions, first:

type w32thing struct {
    _ structs.HostLayout
    a uint8
    b uint16
}

Is this laid out a_bb or is it aaaabbbb? What sizes do I use for struct fields? I assume it is the smaller ones, but I wanted to verify this else it would be a problem.

Second, passing pointers to 8-byte primitive types to the host will be tricky unless those references come from fields in structures tagged with HostLayout -- otherwise, they may not be aligned. So

type wx struct {
   _ structs.HostLayout
  x int64
}
func f(x int64, w wx) {
  someWasmFunc(&x) // might not work, x might not be 8-byte aligned
  someWasmFunc(&w.x) // this will work because w is a wx and its x field is 8-byte aligned
  someOtherWasmFunc(&w) // if it used *wx for its parameter type instead of *int64
}

@dr2chase

Thanks for the quick feedback! I've tried to answer each question:

structs and arrays. What is the ABI specification exactly? The C ABI on, say ELF AMD64, is pretty complex for passing structs and arrays. Small fields may be packed into one word. Large structs may be passed indirectly (stored on stack, passing a pointer to the callee). Do we have a specification for this?

The specification falls out of the table of transformations (I think?). There current plan isn't to introduce any sort of magic around large structs or field packing. Structs fields are added as call parameters, from the first field to the last, according to the conversion rules for the type of the field. Examples:

type foo struct {
    a int
    b string
    c [2]float32
}

With a function signature of

//go:wasmimport some_module some_function
func wasmFunc(in foo) int

Would roughly translate to (in WAT format)

// $a is of type `i32` holding the value of `a`
// $b_addr is of type `i32` and is a pointer to the start of the bytes for the Go string `b`
// $b_len is of type `i32` and is the length in bytes to read from `$b_addr` to get the whole string
// $c_0 is of type `f32` and is the value of `c[0]`
// $c_1 is of type `f32` and is the value of `c[1]`
call $some_function (local.get $a) (local.get $b_addr) (local.get $b_len) (local.get $c_0) (local.get $c_1)

Struct fields would be expanded into call parameters before subsequent fields at the same level.

What does a string look like on Wasm/WASI side?

For wasip1, we will treat Go string parameters simply as a (*byte, int) tuple. There will be no encoding constraints, just as with regular Go strings. To the Wasm host, it will look identical to using struct { a *byte; b int } as a parameter. For wasip2, those constraints would have to be considered in a hypothetical future wasip2 proposal.

Making structs.HostLayout required for structs, arrays of structs and pointers to structs

This sounds like a great idea, and we should also extend it to pointers to 8 byte sized primitive types to guarantee alignment, as suggested by @dr2chase's last question. This would avoid any question around alignment issues for pointers. It hurts the ergonomics a little bit but that's a price worth paying, I think.

type w32thing struct {
_ structs.HostLayout
a uint8
b uint16
}

Is this laid out a_bb or is it aaaabbbb? What sizes do I use for struct fields? I assume it is the smaller ones, but I wanted to verify this else it would be a problem.

I'm a little confused by the question to be honest. If this type was used as an input to a Wasm call, it would look like this:

// $a is of type `i32`
// $b is of type `i32`
call $some_function (local.get $a) (local.get $b)

I suppose that might mean the memory looks like this: a___bb__? We're not passing a pointer to the struct or the fields, so we'd need to copy the values into locals, which will be of type i32 (I think)? Admittedly my grasp of this exact part of the code is a bit weak so I appreciate corrections.

Thanks for the response!

Structs fields are added as call parameters, from the first field to the last, according to the conversion rules for the type of the field.

This sounds like a reasonable choice. Is this ABI specified anywhere in Wasm/WASI docs? Or the Wasm side has to define the function taking parameters element-wise?

For wasip1, we will treat Go string parameters simply as a (*byte, int) tuple. There will be no encoding constraints, just as with regular Go strings. To the Wasm host, it will look identical to using struct { a *byte; b int } as a parameter.

This sounds reasonable as well. Is it specified anywhere in Wasm/WASI docs?

Thanks.

This sounds like a reasonable choice. Is this ABI specified anywhere in Wasm/WASI docs? Or the Wasm side has to define the function taking parameters element-wise?

I don't know about this being an official ABI so much as just a consequence of the Wasm spec around function calls and how we can apply Go semantics to it. We're limited to the i32, i64, f32 and f64 value types, and the call instruction takes a function index and arguments from the stack. In order to simulate pass-by-value for structs, we have to flatten each field to one of the allowed value types.

This sounds reasonable as well. Is it specified anywhere in Wasm/WASI docs?

Not sure there's a doc anywhere, but practically, definitions like path_create_directory, which take a string parameter, use this pattern: https://cs.opensource.google/go/go/+/refs/tags/go1.22.2:src/syscall/fs_wasip1.go;l=230.

compiler: add link to golang/go#66984

I guess my question is whether a pointer-to-struct is ever passed from Go to the WASM platform, and therefore, what expectations the WASM side has about the layout of the fields of that structure. structs.HostLayout is intended to obtain those expectations, but (1) do we even need to do this? We thought we did, and (2) we need to know what the expectations are. I think it was just that 64-bit floats and ints get 64-bit alignment.

I don't think this is for specifying the layout that gets passed to a WASM call if the struct is passed by value.

Change https://go.dev/cl/633776 mentions this issue: cmd/compile: document wasmexport directive, update permitted types for wasmimport

doc/next: add release notes for Wasm changes

cmd/compile: document wasmexport directive, update permitted types fo…

Change https://go.dev/cl/638881 mentions this issue: cmd/compile: improve compiler directive docs

cmd/compile: improve compiler directive docs

@johanbrandhorst

# AI-generated issue overview

Go Wasm Import/Export Type Constraints Relaxation

This proposal suggests relaxing the type constraints for the go:wasmimport compiler directive, which allows importing functions from Wasm modules into Go. Currently, strict limitations exist on the types allowed as input and result parameters due to the mismatch between 64-bit and 32-bit architectures between Go and Wasm hosts. The arrival of the 32-bit Wasm port (issue) mitigates this issue. The proposal aims to broaden the set of permissible types for both wasm and wasm32 architectures. (@johanbrandhorst, issue)

The proposal defines a set of "small integer types" ([u]int[8|16]) and lists several types that would be allowed as input parameters: bool, int32, uint32, int64, uint64, float32, float64, string, uintptr, unsafe.Pointer, pointers to allowed types, pointers to structs (with restrictions based on the architecture and requirement to embed structs.HostLayout), and pointers to arrays of allowed types. All input parameter types except string would also be allowed as result parameter types. Several types, like channels, complex numbers, functions, interfaces, maps, and slices would remain disallowed. (@johanbrandhorst, issue)

The proposal also acknowledges that no compatibility guarantees can be provided across Go compiler versions due to Go's unspecified struct layout. Conversion rules between Go types and Wasm parameter types are outlined. The rationale for disallowing strings as result parameters is due to Wasm's limitation on result parameters. The proposal further discusses the challenges of supporting GOARCH=wasm, which uses 64-bit pointers, and explains why certain types like slices and maps remain disallowed. The proposal also touches upon future considerations, such as WASI Preview 2 and the potential for supporting structs and arrays by value. (@johanbrandhorst, issue)

Discussion Themes

Alignment and structs.HostLayout: Discussion centered on the alignment of 8-byte quantities within structs passed to wasm32. The consensus reached was to use 8-byte alignment for 64-bit types in structs passed to the Wasm host platform when they're marked with structs.HostLayout. This tag signifies host compatibility and is crucial for ensuring proper alignment. (@dr2chase, comment; @ydnar, comment; @evanphx, comment; @dr2chase, comment)
String Handling and ABI Specifications: Questions arose about the representation and handling of strings in Wasm/WASI and the exact ABI specifications for structs and arrays. For wasip1, strings are treated as a (*byte, int) tuple without encoding constraints. The ABI for structs and arrays involves flattening them into individual call parameters. (@cherrymui, comment; @johanbrandhorst, comment)
Passing Structs and Arrays by Value: Discussion arose on whether or not to support passing structs and arrays by value, which the Component Model Canonical ABI supports, while the Basic C ABI has different constraints on this. There was agreement to simplify the initial implementation by removing support for passing structs and arrays by value. This feature could be revisited later if needed. (@johanbrandhorst, comment; @aykevl, comment; @cherrymui, comment)
Pointer Passing and GC Safety: Concerns about pointer passing and GC safety, particularly when returning Go pointers from wasmimport functions or passing them to wasmexport functions were discussed. It was agreed to adopt conventions similar to CGO to ensure memory safety and proper GC behavior. This includes disallowing the host from writing Go pointers to Go memory and requiring pinning of Go pointers held by the host. (@ydnar, comment; @johanbrandhorst, comment; @cherrymui, comment)
Allowed Types for GOARCH=wasm: Discussion also covered the allowed types for GOARCH=wasm, which differs from wasm32 in integer and pointer sizes. The proposal aims to provide some relaxations for wasm as well but with more restrictions due to the 64-bit pointer size mismatch with the host. (@cherrymui, comment; @johanbrandhorst, comment)
int and uint Types: Concerns were raised about allowing int and uint due to their platform-dependent sizes. The proposal was updated to disallow these types to avoid ambiguity. Similar concerns about uint8 and uint16 led to their removal as allowed parameter types but they are still permitted as pointer types or struct fields. (@aclements, comment; @ydnar, comment; @johanbrandhorst, comment)

Resolution

The proposal was accepted. The final version clarifies allowed types, string handling, pointer passing conventions, and the differences in support between wasm and wasm32 architectures. The implementation for wasm was completed in Go 1.24 (comment), while the wasm32 implementation will follow upon the port's completion.

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

@johanbrandhorst

All the mentioning changes have been merged. @johanbrandhorst could this issue be already resolved?

Yep, I think we can close this!

Navigation Menu

Repository navigation

cmd/compile: relax wasm/wasm32 function import signature type constraints #66984

Background

Proposal

Discussion

Compatibility guarantees

Type conversion rules

Strings

Supporting `GOARCH=wasm`

Supporting `[u]int`, `[u]int8`, `[u]int16` as concrete parameters

Supporting slices, maps

Related proposals

struct.Hostlayout

Future work

WASI Preview 2 (AKA WASI 0.2)

Supporting `struct` and `[...]T` by value

Contributors

93 remaining items

Go Wasm Import/Export Type Constraints Relaxation

Discussion Themes

Resolution

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

cmd/compile: relax wasm/wasm32 function import signature type constraints #66984

Description

Background

Proposal

Discussion

Compatibility guarantees

Type conversion rules

Strings

Supporting GOARCH=wasm

Supporting [u]int, [u]int8, [u]int16 as concrete parameters

Supporting slices, maps

Related proposals

struct.Hostlayout

Future work

WASI Preview 2 (AKA WASI 0.2)

Supporting struct and [...]T by value

Contributors

Activity

dr2chase commented on Apr 23, 2024

ydnar commented on Apr 23, 2024

evanphx commented on Apr 23, 2024

dr2chase commented on Apr 23, 2024

cherrymui commented on Apr 23, 2024

dr2chase commented on Apr 23, 2024

johanbrandhorst commented on Apr 25, 2024

cherrymui commented on Apr 25, 2024

johanbrandhorst commented on Apr 25, 2024

dr2chase commented on May 6, 2024

93 remaining items

gopherbot commented on Dec 5, 2024

gopherbot commented on Dec 27, 2024

gabyhelp commented on Jan 15, 2025

Go Wasm Import/Export Type Constraints Relaxation

Discussion Themes

Resolution

dsseng commented on Jul 29, 2025

johanbrandhorst commented on Jul 31, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Supporting `GOARCH=wasm`

Supporting `[u]int`, `[u]int8`, `[u]int16` as concrete parameters

Supporting `struct` and `[...]T` by value