CG-10-22.md

Agenda for the October 22 video call of WebAssembly's Community Group

• Where: Virtual meeting
• When: 2024-10-22, 16:00-17:00 UTC (2024-10-22, 9am-10am PDT, 18:00-19:00 CEST)
• Location: link on W3C calendar or Google Calendar invitation

Registration

No registration is required for VC meetings. The meeting is open to CG members only.

Agenda items

1. Opening
2. Proposals and discussions
3. Announcement: tentatively planning next in-person CG + Research Day in Fastly SF office Feb 11-13 (issue: meetings/#1680) (Luke Wagner, 5 minutes)
4. Discussion: Module Splitting and Type Duplication (Thomas Lively, 30-45 minutes)
5. Phase 4 poll for memory64 (Sam Clegg, 15 minutes)
6. Closure

Agenda items for future meetings

None

Meeting Notes

Attendees

• Conrad Watt
• Thomas Lively
• Andrew Brown
• Alex Chrichton
• Alon Zakai
• Andreas Rossberg
• Ben Visness
• Brendan Dahl
• Chris Fallin
• Chris Woods
• Dan Gohman
• Dan Philips
• Daniel Lehmann
• Deepti Gandluri
• Francis McCabe
• Heejin Ahn
• Hunter Demeyer
• Jakob Kummerow
• Ricky Vetter
• Jeff Charles
• Linwei Shang
• Johnnie Birch
• Julien Pages
• Manos
• Luke Wagner
• Matthias Liedtke
• Michiel Van Kenhove
• Mingqui Sun
• Oscar Spencer
• Paolo Severini
• Paul Dennis
• Robin Reyler
• Ryan Hunt
• Sam Clegg
• Sean Isom
• Sergey Rubanov
• Yuri Iozzelli
• Yury Delendik
• Zalim Bashorov
• Ilya Rezvov
• Derek Schuff

Proposals and discussions

Announcement: tentatively planning next in-person CG + Research Day in Fastly SF office Feb 11-13 (Luke Wagner, 5 minutes)

LW presenting

LW: tentative 11-13 February, Fastly, see github issue for scheduling discussion (https://github.com/WebAssembly/meetings/issues/1680)

Discussion: Module Splitting and Type Duplication (Thomas Lively, 30-45 minutes)

TL presenting https://github.com/webAssembly/design/issues/1530

AR: why is there a global decrease?

TL: not sure, i didn’t look in detail, but whenever there’s an unexpected code size decrease it’s usually LEBs. I suspect that the split modules just had a lot fewer globals and global imports. All of our strings are imported globals, 14k of those. So the LEBs can get big.

AR: so when you compute the share, you also consider the use size?

TL: yeah the code size is very naive, code size as reported by bloaty. I’m not trying to attribute from other sections

AR: so what you said would only be true if they used globals a lot

TL: right, so the code section sizes are a little wonky because of that,

TL: so the type section is 56% and i want to focus on that (presenting type section numbers)

TL: would like to hear input. One option is type imports. Another if we could have a module reuse a type section from another module that’s already downloaded. That’s weird and unprecedented but it would be ideal for this problem.

CW: if we had repeated sections, it would be very natural to interpret one module as the suffix of another one.

AR: it would be interesting to know, of these indirectly used types, how many are actually needed for type-checking the rest of the module? Or is it sufficient if they are completely opaque?

TL: it should be sufficient to know what reference hierarchy it's in. because they aren’t directly used in any instructions (otherwise they'd be directly used) I think that’s enough to type the module.

AR: I wonder if it would be possible if we had type imports, you could import the transitive types without the definition, would be enough to typecheck with just an abstract bound.

TL: yeah, i think it would the question is how much

CW: which do we think involves less standards work?

JK: you need to identify recgroups though... if the main module has a recgroup (A, B, C), and the secondary module has (A, B, __undefined_placeholder__), how can recgroup identity be established?

AR: you’d always have to specify the entire rec group, you can’t cherry pick from a group, but can pick the transitive definitions. If your hierarchies are deep or indirect dependencies it might help quite a bit.

TL: I suspect the unused fields are the common case. We are optimizing out unused supertypes, but it’s possible they contribute some, but my guess is not.

AR: for super types this wouldn’t work, you’d need the definition to check if the subtyping is sound. But for fields and parameters you could cut them off

TL: and you wouldn’t want to defer that soundness check to instantiation time.

CW: you’d need a way to declare that an abstract type is a subtype of another

AR: you still need to be able to check the structure at the definition type, not just the bound but the full structure.

TL: I'm not totally up on what the current type imports proposal says. AR, do you know whether the proposed mechanism would be sufficient to help with this problem? Or would we need to propose new features there to get started?

AR: if this would help I think it would help even in its current form. You can already express importing abstractly without the definition

TL: if my intuition is right, it would be useful to have a form of import to define a type with some of its fields but leave others abstract. Mix concrete and abstract fields. Would be more compact that having a type import for that field

AR: but how do you relate that to the concrete definition soundly? You don’t want to concretize different ways in different modules. I don’t see how you can do that without imports ensuring they come from the same source

TL: difference between importing types with supertype bounds vs arbitrary constraints on fields. Or not arbitrary but more fine-grained field-level constraints. But yeah I haven’t thought rigorously about the details.

AR: yeah i’m not sure how that would look, it’s not obvious.

CW: could we consider the repeated sections solution? Seems simpler

AR: can you clarify the API for that? Concatenate the modules before you instantiate them?

CW: most naively would be something like instantiateStreaming, that takes a module as an additional parameter, and the new bytes + the (or some) existing sections of the module are instantiated as one big module

AR: The actual binary is not a self-contained module. Other tools would not understand what it is. All tooling would have to do this concatenation to make sense of it.

CW: you can do these games already, e.g. concatenating the bytes in an arraybuffer.

AR but then it's not a module it’s a fragment, you can’t validate that already.

CW: I'm fine calling it that.

AR: the broader point is that this is introducing a whole new thing to the ecosystem. It’s more than hacking an API, it’s a new kind of thing

CW: I think this is a fairly principled extension to repeated sections, to have the sections in another file

AR: we can argue about what’s a hack, but let's not 🙂

CW: i think this could also solve other kinds of separate compilation issues

RH: you’d want a special module fragment interface that’s different from modules. When you compile a module you might want to throw away metadata, but with a fragment you’d want to hold on to everything to use when you compile the real module, so a special interface would be good. Also some people do JIT compilation in the browser, and have to duplicate the type section, so maybe this could help with that. I agree that this is a new thing and we’d want it to have multiple use cases, but not opposed to that.

AR: Some module systems have a notion of one module "including" a different module and extending it. That would be a principled way of doing it, but it would be kind of like a new phase of staging. You would need to resolve the import before validation.

CW: presumably you’d need to declare the type of the module when you import it, so you’d still worry about code size

AR: yeah depending on how self-contained you want to make th is

CW: I'm getting more attached to this suggestion. I think the main technical issue would be the concerns people have raised about repeated sections more generally. Don't now if people have more thoughts about repeated sections generally?

TL: It's less of a hack than what i had thought about oding something ad-hoc just for the type section. So i do like repeated sections better than that. But also i’ve been one of the loudest complaining about repeated sections. So we’d have to rethink that

AR: repeated sections seems like a thing we’ll eventually want anyway, it keep coming up. Maybe i don’t liek abusing it for this but…

TL if you can avoid repeating the type section at all, that’s 0% bloat, as good as you can get and perfectly solves the problem of scaling module splitting. It’s an open question how close to perfect we can get with type imports. My guess is that there would still be code size overhead but maybe it doesn’t scale as poorly as today. Maybe it would be acceptable. If we think repeated sections is a hard battle, then maybe it’s worth looking into type imports first

CW: I would predict that type imports would get into a mess about abstract types and upper bounds, could be intricate design work.

AR: we only need fully abstract types, that seems straightforward

CW: it’s already come up with TL’s ideas about abstract fields. Fi you do the naive thing it might be disappointing in savings

AR: maybe we can try to derive that from the modules we have.

TL: i could see counting the directly used fields, and estimating the size of the type import etc. should be doable. One step more would be just to prototype the type imports and measure directly.

CW: one more augment is that adding a very basic form of abstract type, seems a little unprincipled.

AR: we talked about reading this anyway, e.g. with string builtins. The minimal feature set we discussed would already be sufficient for this

CW: maybe. I’m slightly cynical about that 🙂

TL: we have these 2 directions, they were kind of the directions i had in mind. Are there any radically different ideas for solutions?

SC: not a new idea but if you have an optimized type section, you actually get that LEB savings, but you don't get that from a giant type section

TL: true;

PD: One could use an compression algorithm and transfer multiple modules at once

TL: yeah if you use one type section as a compression dictionary for another than that could work. It does go against the goal of doing module splitting, avoiding shipping the module twice. Using one module as a dictionary for another could work

JK(chat): one could say that a JS API for "use the type section from that other module" is a special implementation of a transfer-only compression scheme...

FM: the fragment thing, you don't need an official standard to experiment with that, you can just do it. With the array buffer concatenation approach.

CW: i think the downside is you’d lose access to instantiateSTreaming, but it could work for code size comparisons.

FM: i think you’ll lose streaming anyway because you're concatenating streams.

CW: in my vision you’d compileStreaming the first module and the second would get the firs module as a param plus a new streaim.

FM: id have concerns about unintended consequences inthe engines for that. It’s vague, but i think you may end up with other kinds of bloat showing up later

CW: one naive issue would be if the engine could only accomplish this by keeping the uncompiled bytes of the previous module live, could be memory pressure

ZB (chat): TODO

ZB: Being able to skip repeating supertype fields in definitions could help with separate/incremental compilation as well.

[missed some comments]

RH: One thing about the module fragment idea,.... Was that 20% of the bloat due to imports?

TL: yeah bout 18% from increased exports. This is not something i think the CG needs to worry about right now because the way we add exports in wasm-split is pretty naive. We take every exportable item in the primary nodule and exports just in case; smarter would be to only use things that are used in the secondary module. Better still would be to move the definition into the secondary module if possible. E.g. move a whole global to the secondary. Today we aren’t doing that. So i think we can still improve exports by improving the tools rather than the spec right now

CW: If you did these would that reduce the type size too? Make types dead in certain modules?

TL: This would help by removing type from primary module, reducing duplication by at most 1. But types appear on average in 10 modules, so this would only get us to 9

AR: wrt the hacky solution of concatenation. You can do that right now. There's no need for repeated sections there, ensure that the modules are built with the same type section prefix and put them back together. But you don’t need semantics. I don’t think it will get much cleaner anyway you're just operating on the byte stream and you can do that today.

CW: yeah I think that was a comment francis raised, the streaming interface would be the motivation.

AR: the repeated section doesn’t really buy you anything on top of that semantically, it's only the streaming API

CW: But that’s a lot. Also repeated sections for other purposes like conditional compilation

AR: agree but they are totally unrelated

CW: yeah but i do think that’s a good argument for repeated sections

AR: but repeated sections are irrelevant to the problem

CW: if you want the streaming API they are relevant

TL: could you use a service worker to do that

AR: yes you could do it with an API or whatever

TL: if you add a more high level API that gets a module fragment adn appends, it then you’d want repeated sections because module fragments become a first class thing

CW: if you do it at the byte level, what if they don’t interleave in the right way, e.g. what if you want the type and import section only.

PD: The compression algorithm could take care of some details of matching and piecing together.

TL: this is kind of similar to defining a new wasm module container format on top of the existing format, gets back to the idea of “layer 1 compression: but tailored to this problem.

CW: i think the point about wanting e.g. type and import sections makes the byte level streaming idea not completely comparable, and makes the repeated section idea more relevant

AR: they are both byte level things, first you have the concatenation kdin of interface, and separately you could “Extend” the power of concatenation if you want. They go well together but neither requires the other.

CW: you could have an API that chops out pieces, but i think repeated section is a natural way to think about it

JK (chat): Q about repeated sections: would they have continuous index spaces for things like functions? E.g. the first function section defines function indices 0, 1, 2; second function section defines function indices 4, 5, 6? If so, that would imply a fixed order between the N fragments into which you'd split a module, right? If there's a main module and a set of optional extension modules, it should probably possible to load those in any order.

AR: yeah i don’t think it could work any other way.

CW: but that doesn’t necessarily hurt the use case here, you’d want the whole type section at a time here

AR: yeah that seems fine for this use case

CWoods: what are we solving here? Just download size and time to execution? Why not allow the sections to be decomposed and replace them with URIs and split? If the type section is small, just replace it and download separately? You could point to sections with pseudo modules

CW: that seems like the idea i’m advocating for

AR: there are 2 approaches: a semantic approach, type imports. And a representation approach, concatenation of byte streams, repeated sections. You might want to do that latter in a streaming fashion to make it more efficient to handle the parts.

TL: do we want to talk about memory 64?

SC: the plan was to delay the vote, i can give an update

TL: Then i’d want to look into estimating benefit from type imports, and based on that see if we want to look into an API, repeated sections, fragments etc.

Phase 4 poll for memory64 (Sam Clegg, 15 minutes)

SC presenting

SC: a quick update on memory64. We were going to vote today but don’t plan to right now. We did an update in June and since then we added 64 bit table support to FF and chrome, updated the JS API spec so that the table and memory APIs on the JS side take BigInts rather than numbers. Thanks to Ben we now have updated JS API tests, but V8 hasn’t had a chance to import them and get them passing, so we’ll punt the vote a couple weeks to ensure that we have 2 engines that have 100% tests passing. The spec is basically done but we wanted to be sure about the implementations.

BV(chat): spec should be done and Firefox is passing all spec tests

SC: The letter of the process does say that we need 2 implementations need to pass the tests, so it seems good to wait until we have that 100%

CW: i think it’s good to be principled, hopefully we can get this out, you’ve been working hard on that.

Closure