Skip to content

Implement resource borrow and transfer trampolines#10

Open
juntyr wants to merge 20 commits into
DouglasDwyer:masterfrom
juntyr:resource-transfer
Open

Implement resource borrow and transfer trampolines#10
juntyr wants to merge 20 commits into
DouglasDwyer:masterfrom
juntyr:resource-transfer

Conversation

@juntyr
Copy link
Copy Markdown

@juntyr juntyr commented Feb 5, 2024

This PR implements resource borrows and transfers for the wasm_component_layer. In particular, it:

  • upgrades all dependencies to the latest version (at the time of writing)
    • does not yet differentiate between stable and unstable WIT interfaces (future work)
  • implements the 32bit UTF8 copy transcoder trampoline
    • other transcoders may be implemented in future work as needed
  • implements the always drop, resource transfer own/borrow, and resource enter/exit call trampolines
    • these trampolines are deduplicated per store
  • automatically creates the needed per-instance flags globals on-demand
    • this follows jco's implementation, since creating them up front always misses some

This PR is only a first step, but - as can be seen from the implementation duration - has survived being used in my project for several months. Merging this PR will expose the added functionality to more users who can implement further usecases (e.g. other transcoder trampolines) as required.

Comment thread src/lib.rs
Comment thread src/lib.rs Outdated
Comment thread src/lib.rs
Comment thread src/lib.rs
@DouglasDwyer
Copy link
Copy Markdown
Owner

Thanks for your work! This PR looks to be on the right track so far.

@DouglasDwyer DouglasDwyer linked an issue Feb 6, 2024 that may be closed by this pull request
@juntyr
Copy link
Copy Markdown
Author

juntyr commented Feb 29, 2024

Sorry for the delay - as the stubs were enough to get my use-case running it took a while to find some time to get back to this PR and implement it properly. Now that bytecodealliance/jco#371 has landed I have a good reference for the actual implementation :)

Comment thread src/lib.rs Outdated
let from_table = &mut table_array[from_rid];
let handle_borrow = from_table.get(*handle)?;
let handle_rep = handle_borrow.rep;
// FIXME: wrong condition, should be if from_rid is imported
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@DouglasDwyer I am unsure about how to implement this part (and about whether I understand Jco's code here https://github.com/bytecodealliance/jco/blob/1420aad1679418da6037bb787a9130b88453bcd3/crates/js-component-bindgen/src/intrinsics.rs#L198-L211). Jco checks whether the resource is imported or local to this instance - is this information already stored somewhere / how could I get it?

Copy link
Copy Markdown
Owner

@DouglasDwyer DouglasDwyer Mar 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @juntyr, thanks again for your hard work on this! I do not believe that I store whether the resource is imported or local to an instance anywhere currently.

I would need to do a bit more reading to understand why the handle is not cleared if the resource is local to the instance. Why would the behavior be different?

I am wondering if this check has to do with the "simplification" where local borrows aren't tracked? That simplification isn't implemented at all in this crate right now.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I hadn’t considered that “simplification” before but I think you’re right that this might be the reason for this if statement in Jco. Without it, would a borrow transfer just decrease the lend count in the “from” resource and create a new resource instance for “to”? Is this the purpose of the lend count? If we always create a new resource instance, we also loose the second if statement which just uses the inner representation inside the defining instance, otherwise we might gain layers of indirection which could mess with that instance‘s assumption that the representation is just a pointer?

Copy link
Copy Markdown
Author

@juntyr juntyr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've made some more progress in filling out the trampolines but there are still some unresolved questions and cleanup needed

@m11m m11m mentioned this pull request Jun 3, 2024
@juntyr juntyr force-pushed the resource-transfer branch from 6667c30 to 7117bb0 Compare June 23, 2024 08:32
@juntyr
Copy link
Copy Markdown
Author

juntyr commented Jun 24, 2024

@KayHHH Are you still having the issue from #16?

@juntyr juntyr marked this pull request as ready for review June 24, 2024 08:40
@juntyr juntyr changed the title Implement (most) missing trampolines Implement resource borrow and transfer trampolines Jun 24, 2024
@juntyr
Copy link
Copy Markdown
Author

juntyr commented Jun 24, 2024

@DouglasDwyer I've upgraded dependencies and cleaned up the code a bit. This PR works for my usecase at least, but I'd appreciate a review to see if I missed anything obvious.

@juntyr juntyr requested a review from DouglasDwyer June 24, 2024 09:37
@m11m
Copy link
Copy Markdown

m11m commented Jun 24, 2024

@KayHHH Are you still having the issue from #16?

Sorry I should have left a comment. I ended up restructuring things to avoid the problem, but yes it was still happening.

@juntyr
Copy link
Copy Markdown
Author

juntyr commented Jun 24, 2024

@KayHHH Are you still having the issue from #16?

Sorry I should have left a comment. I ended up restructuring things to avoid the problem, but yes it was still happening.

I think I triggered that bug once before as well but also just worked around it at the time. Do you still have a reproducible example you could share? Perhaps I can find the fix.

@m11m
Copy link
Copy Markdown

m11m commented Jun 24, 2024

@KayHHH Are you still having the issue from #16?

Sorry I should have left a comment. I ended up restructuring things to avoid the problem, but yes it was still happening.

I think I triggered that bug once before as well but also just worked around it at the time. Do you still have a reproducible example you could share? Perhaps I can find the fix.

I can maybe find a commit where it was happening in my project but with how its set up its probably easier to create a separate example from scratch. I'm not home now to do so though.

@DouglasDwyer
Copy link
Copy Markdown
Owner

@juntyr, I skimmed through the code and everything looks good. Unfortunately, my current focus is on other projects, and I don't have time to vet wasm_component_layer changes more thoroughly. Could you possibly:

  • Provide a list in a PR comment describing all of the new things that you've added?
  • Update the project README with the new "Supported Capabilities" that you've added?
  • Add an example which shows string transcoders and/or some of the other trampolines working?

Once those things are finished, I will happily merge this PR :)

@juntyr
Copy link
Copy Markdown
Author

juntyr commented Jun 27, 2024

  • Provide a list in a PR comment describing all of the new things that you've added?
  • Update the project README with the new "Supported Capabilities" that you've added?
  • Add an example which shows string transcoders and/or some of the other trampolines working?

jsantell added a commit to jsantell/wasm_component_layer that referenced this pull request Oct 23, 2024
jsantell added a commit to jsantell/wasm_component_layer that referenced this pull request Oct 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

resource transfer trampolines are unimplemented

3 participants