You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
They are not supposed to be internal, they are primops. They are used e.g. in primitive (which is where this issue comes from in the first place, see haskell/primitive#375).
They are not supposed to be internal, they are primops.
In my vocabulary, primops and internal are the same thing: something that is not supposed to be used by a median language user, and instead should be used only for some low-level stuff (like primitive).
For what it's worth, I generally agree with @chshersh that primops aren't necessarily intended to be widely consumed. There are many primops which are present only to implement bits of GHC (e.g. unpackClosure#), others which are intended to be exposed only via their friendlier lifted wrappers (e.g. indexInt16OffAddr#), and others that are hard enough to use correctly that a user should expect to review their usage with every new GHC release (e.g. addrToAny#).
That being said, I think keepAlive# and touch# are examples of primops that do have legitimate uses in low-level library code. However, I cannot see any way in which this generalization would break an existing well-typed program. The only way that such a generalization would break is a failure of type inference; however, all uses of keepAlive# and touch# are already specialised to RealWorld.
@bgamari I see. Thanks for the detailed response! I can accept this as an impact assessment and let everyone else to express their concerns and suggestions.
I think this is the case of something that is supposed to be internal but somehow got leaked via reexports from ghc-prim to base.
FWIW entire ghc-prim is re-exported from base, usually from GHC.Exts, that's a proper operation.
The idea is that users should never import ghc-prim themselves: this package is deeply magical in its structure and can (and does) change rapidly without any deprecation period or warning. Using its entities through base:GHC.Exts allows us (at least in theory) to provide some compatibility shims and wrappers.
I have no idea what those two functions do.
* `keepAlive#` docs don't really help
* `touch#` doesn't have documentation at all
I must admit that this is a stumbling block for me as well, despite a non-zero experience in low-level guts of Haskell. It would be immensely helpful, if someone volunteered to improve the documentation.
For those who aren’t familiar with them: both operations are to ensure some data structure stays reachable on the heap.
As for why touch still exists: there exists code out there that largely uses touch correctly in a way that won’t be miscompiled, and there’s no good reason to break it. Otoh its use for new code should absolutely be discouraged.
@cartazio there are also situations where keepAlive# has performance issues, or (at least potentially) where object lifetimes are supposed to be staggered.
While we do strongly recommend that people use keepAlive# whenever possible, sometimes this is simply not the case. Specifically, some programs use ByteArray#s which whose lifetime does not follow the lexical scoping of the program. In these cases one must use touch# (and be extremely careful while doing so).
While the original ticket only addressed keepAlive#, I would argue that it makes sense to treat these operations symmetrically.
(@bgamari you don't have to answer right away, but I have a question: Do these programs use ByteaArray#s in such a way because there is no other way, or could this usage be modified as not to require touch#?)
(@bgamari you don't have to answer right away, but I have a question: Do these programs use ByteaArray#s in such a way because there is no other way, or could this usage be modified as not to require touch#?)
I can certainly envision cases where the structure of a foreign library essentially precludes lexical scoping of a buffer.
Activity
[-]Generalize the type of keepAlive#[/-][+]Generalize the type of keepAlive#, touch#[/+]treeowl commentedon Mar 28, 2023
Yes, please!
chessai commentedon Mar 28, 2023
This would be very useful
phadej commentedon Mar 28, 2023
This feels silly that this change needs CLC proposal, but apparently doesn't need GHC proposal... (after all, this is changing GHC primops!)
chshersh commentedon Mar 30, 2023
I have no idea what those two functions do.
keepAlive#docs don't really helptouch#doesn't have documentation at allI think this is the case of something that is supposed to be internal but somehow got leaked via reexports from
ghc-primtobase.Even though these two functions look like a really internal thing, I see that they have usages across multiple popular packages:
This looks like a change that could break things. So impact assessment is required.
konsumlamm commentedon Mar 30, 2023
They are not supposed to be internal, they are primops. They are used e.g. in
primitive(which is where this issue comes from in the first place, see haskell/primitive#375).chshersh commentedon Mar 30, 2023
In my vocabulary, primops and internal are the same thing: something that is not supposed to be used by a median language user, and instead should be used only for some low-level stuff (like
primitive).bgamari commentedon Mar 30, 2023
@chshersh this blog post may shed some light on the intended usage of these operations.
For what it's worth, I generally agree with @chshersh that primops aren't necessarily intended to be widely consumed. There are many primops which are present only to implement bits of GHC (e.g.
unpackClosure#), others which are intended to be exposed only via their friendlier lifted wrappers (e.g.indexInt16OffAddr#), and others that are hard enough to use correctly that a user should expect to review their usage with every new GHC release (e.g.addrToAny#).That being said, I think
keepAlive#andtouch#are examples of primops that do have legitimate uses in low-level library code. However, I cannot see any way in which this generalization would break an existing well-typed program. The only way that such a generalization would break is a failure of type inference; however, all uses ofkeepAlive#andtouch#are already specialised toRealWorld.chshersh commentedon Mar 30, 2023
@bgamari I see. Thanks for the detailed response! I can accept this as an impact assessment and let everyone else to express their concerns and suggestions.
Bodigrim commentedon Apr 2, 2023
FWIW entire
ghc-primis re-exported frombase, usually fromGHC.Exts, that's a proper operation.The idea is that users should never import
ghc-primthemselves: this package is deeply magical in its structure and can (and does) change rapidly without any deprecation period or warning. Using its entities throughbase:GHC.Extsallows us (at least in theory) to provide some compatibility shims and wrappers.I must admit that this is a stumbling block for me as well, despite a non-zero experience in low-level guts of Haskell. It would be immensely helpful, if someone volunteered to improve the documentation.
@bgamari @andrewthad @chessai @treeowl is there a good reason to touch
touch#? My understanding of https://www.haskell.org/ghc/blog/20210607-the-keepAlive-story.html is that all new development should usekeepAlive#.cartazio commentedon Apr 3, 2023
For those who aren’t familiar with them: both operations are to ensure some data structure stays reachable on the heap.
As for why touch still exists: there exists code out there that largely uses touch correctly in a way that won’t be miscompiled, and there’s no good reason to break it. Otoh its use for new code should absolutely be discouraged.
treeowl commentedon Apr 3, 2023
@cartazio there are also situations where
keepAlive#has performance issues, or (at least potentially) where object lifetimes are supposed to be staggered.cartazio commentedon Apr 3, 2023
Totally. Point being keeping both has very real value and low marginal cost.
Bodigrim commentedon Apr 3, 2023
@cartazio There've been no suggestions to remove
touch#.@treeowl could you please elaborate on performance issues with
keepAlive#with a specific example?treeowl commentedon Apr 3, 2023
@Bodigrim See
Lost optimisation: CPR unboxingincompiler/GHC/CoreToStg/Prep.hs. I may have been wrong about the staggering bit; sorry.bgamari commentedon Apr 5, 2023
While we do strongly recommend that people use
keepAlive#whenever possible, sometimes this is simply not the case. Specifically, some programs useByteArray#s which whose lifetime does not follow the lexical scoping of the program. In these cases one must usetouch#(and be extremely careful while doing so).While the original ticket only addressed
keepAlive#, I would argue that it makes sense to treat these operations symmetrically.Kleidukos commentedon Apr 5, 2023
(@bgamari you don't have to answer right away, but I have a question: Do these programs use
ByteaArray#s in such a way because there is no other way, or could this usage be modified as not to requiretouch#?)bgamari commentedon Apr 5, 2023
I can certainly envision cases where the structure of a foreign library essentially precludes lexical scoping of a buffer.
Bodigrim commentedon Apr 14, 2023
Thanks for the explanations @bgamari.
Dear CLC members, any non-binding opinions on this?
Bodigrim commentedon Apr 17, 2023
Dear CLC members, given that there are no more comments, let's vote on the proposal to generalize
keepAlive#andtouch#to accept any state token, not onlyRealWorld, as detailed in https://gitlab.haskell.org/ghc/ghc/-/merge_requests/10171/diffs. See https://gitlab.haskell.org/ghc/ghc/-/issues/23163 and haskell/primitive#375 for related discussion.@tomjaguarpaw @chshersh @mixphix @hasufell @parsonsmatt @angerman
+1 from me.
mixphix commentedon Apr 17, 2023
I'm in favour!
chshersh commentedon Apr 17, 2023
+1 from me
parsonsmatt commentedon Apr 18, 2023
+1
Bodigrim commentedon Apr 19, 2023
Thanks all, 4 votes are enough to approve.