Skip to content

Add keepAlive - #363

Merged
andrewthad merged 2 commits into
haskell:masterfrom
type-dance:keepalive
Feb 16, 2023
Merged

andrewthad merged 2 commits into
haskell:masterfrom
type-dance:keepalive

Conversation

@TerrorJack

Copy link
Copy Markdown
Contributor

Closes #362.

Comment thread Control/Monad/Primitive.hs Resolved
@andrewthad

Copy link
Copy Markdown
Contributor

Looks good. I noticed that keepAlive# gets imported from GHC.Prim. We want to import that from GHC.Exts instead, assuming it's exported from there (which it should be). And then you can drop the dep on ghc-prim.

TerrorJack and others added 2 commits January 13, 2023 17:24
Co-authored-by: Alexey Kuleshevich <lehins@yandex.ru>
@andrewthad

Copy link
Copy Markdown
Contributor

I'll leave this open for another day or two in case anyone else with more knowledge about this primop has any feedback, and I'll merge if I don't hear anything.

@andrewthad
andrewthad merged commit e9ff8d6 into haskell:master Feb 16, 2023
@mitchellwrosen

Copy link
Copy Markdown

Hey! Just curious, why does this combinator have the type

keepAlive :: PrimBase m => a -> (a -> m r) -> m r

and not

keepAlive :: PrimBase m => a -> m r -> m r

@treeowl

treeowl commented Mar 20, 2023

Copy link
Copy Markdown
Contributor

@mitchellwrosen , that's an extremely good question. It looks like a mistake..... @andrewthad ?

@chessai

chessai commented Mar 20, 2023

Copy link
Copy Markdown
Member

It is analogous to the "with" family of combinators, e.g. withFile. keepAlive x k ensures x is kept alive as long as it is referenced by the body of k. To my knowledge, the signature suggested would not provide better guarantees than the flawed touch.

@treeowl

treeowl commented Mar 20, 2023

Copy link
Copy Markdown
Contributor

Okay, maybe there's a real idea here, but I'm not sure it's expressed well enough. The only reasons I can think of to keep a lifted value alive are to keep a weak reference active or (relatedly) to delay calling an attached finalizer. Getting that right may involve some NOINLINEs and/or lazy innovations to make sure the thing kept alive is the thing the weak pointer attaches to. Maybe this is to help with that? I don't think it goes all the way though.

@chessai

chessai commented Mar 20, 2023

Copy link
Copy Markdown
Member

It is analogous to the "with" family of combinators, e.g. withFile. keepAlive x k ensures x is kept alive as long as it is referenced by the body of k. To my knowledge, the signature suggested would not provide better guarantees than the flawed touch.

For a more thorough explanation, see https://www.haskell.org/ghc/blog/20210607-the-keepAlive-story.html

@treeowl

treeowl commented Mar 20, 2023

Copy link
Copy Markdown
Contributor

@chessai I think we all know what the primop is for....

@chessai

chessai commented Mar 20, 2023

Copy link
Copy Markdown
Member

@chessai I think we all know what the primop is for....

Well, the link goes into a detailed breakdown of why the type is the way it is. So I thought it might be helpful. Maybe I'm missing something though.

@treeowl

treeowl commented Mar 20, 2023

Copy link
Copy Markdown
Contributor

@chessai , I guess you're talking about the situation where the value isn't supposed to be used after its finalizer runs, like a file handle? I don't know if that's the only use case for the primop.

@treeowl

treeowl commented Mar 20, 2023

Copy link
Copy Markdown
Contributor

It is a bit weird to attach finalizers to lifted things, though; this primop is probably more useful in primitive-unlifted.

@treeowl

treeowl commented Mar 20, 2023

Copy link
Copy Markdown
Contributor

@chessai , the type of the function is not analogous to the type of the primop, which is unusual. I can imagine a reason for such a function, but as I said I don't think this realizes that exactly.

@treeowl

treeowl commented Mar 20, 2023

Copy link
Copy Markdown
Contributor

I'd expect the fancy version to look more like

keepAlive x k = unsafeIOToPrim $ primitive $ \s0 -> keepAlive# x s0 $ internal $ unsafePrimToIO $ k (lazy x)

The lazy would help prevent the function argument from being unboxed/reboxed. But I'm not sure this approach will substitute for extreme care using something more like the raw primop.....

@mitchellwrosen

Copy link
Copy Markdown

@chessai Thanks for the information. I (tried to) read the article you linked and I admit I didn't get exact clarity on your response:

keepAlive x k ensures x is kept alive as long as it is referenced by the body of k

I believe the documentation of withForeignPtr, which essentially just wraps keepAlive#, suggests the opposite:

The foreign object is kept alive at least during the whole action, even if it is not used directly inside.

ðŸĪ”

@andrewthad

Copy link
Copy Markdown
Contributor

I agree that the type of keepAlive is essentially wrong. I should have thought about this more before merging. The type should be

keepAlive :: PrimBase m => a -> m r -> m r

because it most closely matches the type of the primop

keepAlive# :: v -> State# RealWorld -> (State# RealWorld -> p) -> p

but with p specialized to (# State# RealWorld, q #). GHC Issue 21868 explains this weird return type, but it's irrelevant to the discussion here. Unfortunately, this has already been released, so changing it is difficult. We could introduce another function with the correct signature, soft deprecate keepAlive, and then either remove it or just change it in the next release of primitive.

@treeowl describes attaching finalizers to lifted things as "weird". I would go one step further and call it "dangerous". But it's something that GHC explicitly allows, so I think it's reasonable to support it.

I think that a useful addition would be

keepMutableByteArrayAlive :: PrimBase m => MutableByteArray s -> m r -> m r

Which would call keepAlive# on the inner MutableByteArray#, not on the wrapper. And it would be good to have a note about why such a specialization exists.

@treeowl

treeowl commented Mar 21, 2023

Copy link
Copy Markdown
Contributor

@andrewthad I think it would probably be best to have a discussion of such things as part of a broader discussion of the fate of primitive-unlifted. There's nothing special about MutableByteArray; one might wish to keep anything UnliftedType alive.

@andrewthad

Copy link
Copy Markdown
Contributor

Since you raised the issue, here are my thoughts. If there's a reasonable way to do it, I think it would be nice to fold pieces of primitive-unlifted into primitive. For example, the PrimUnlifted class is clearly sound and works all the way back to antiquity. So I think it would be appropriate to just move it over to primitive. And that gives enough to build keepAliveUnlifted or whatever we might call it. All the stuff that has a different code path for the pre-ghc-9.4 world I'm less inclined to move into primitive. I've been burned a lot by unsafeCoerce#, sometimes by my own mistakes, sometimes by GHC bugs (since unsafeCoerce# doesn't get tested a whole lot by applications), and sometimes by things that I just never figured out at all.

@treeowl

treeowl commented Mar 21, 2023

Copy link
Copy Markdown
Contributor

PrimUnlifted actually isn't quite safe.

fromUnlifted# x = fromShortByteStringUnsafe (SBS x)

bothers me. I'd be much happier if the Unlifted version of ShortText were a newtype around ByteArray# that allows for invariant checking.

@andrewthad

Copy link
Copy Markdown
Contributor

Yes, the instance for ShortText is not quite right. Used in some of my other projects, I have Data.Text.Short.Unlifted, which I would like to move into the text-short library.

@chessai

chessai commented Mar 21, 2023

Copy link
Copy Markdown
Member

Ah, you're right. I didn't look closely at the primop before commenting, instead working off memory. This is really unfortunate. Could we do an impact analysis to determine how much it's actually used? It's so new, I doubt many are even using it. There are 4 of some of the most active users of primitive in this thread and it just got noticed. Fixing it would have to be a major version bump, unless we soft deprecated and waited, like Andrew said. Let me know if an impact assessment would be useful, and I can do it.

@treeowl

treeowl commented Mar 21, 2023

Copy link
Copy Markdown
Contributor

I think we should fix it and do a major version bump.

@chessai

chessai commented Mar 21, 2023

Copy link
Copy Markdown
Member

I think we should fix it and do a major version bump.

This is my preference as well

@treeowl

treeowl commented Mar 21, 2023

Copy link
Copy Markdown
Contributor

@andrewthad Yeah, maybe we should keep primitive-unlifted as as separate package. The next release of that should drop the pre-9.4 stuff, and we should put some good links in the primitive documentation to help people find the relevant primitive-unlifted stuff.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add keepAlive in addition to touch?

6 participants