~sircmpwn/hare-rfc

6 3

[RFC v1] handling allocation failure

Lorenz (xha) <me@xha.li>
Details
Message ID
<ZhE7J8IveVPdVdJw@xha.li>
DKIM signature
missing
Download raw message
                              RFC SUMMARY

hare builtin's should have some way of handling allocation failures.

most apps probably don't want to handle allocation failure but
rather crash. this proposal does not create any performance impact or
additional complications in these cases, except adding a trailing "!"
to builtins that might return nomem.

                         LANGUAGE IMPLICATIONS

nomem becomes a builtin type. it's error flag is always set.  basicly
another void, unfortionately; as autumnull@ already explained in the
previous proposal, rt::nomem (or errors::nomem) doesn't make sense
because you would always have to import it. i think that its fine
to add a builtin for handling errors from builtins.

the return values are changed like so:

	alloc() returns (*T | nomem)
	static(opt) append() returns (void | nomem)
	static(opt) insert() returns (void | nomem)

i think that the special case for alloc(), i.e., returning a type
based on it's argument and type hint, should be fine, because it's
very obvious to the user what is happening and not very hard to
implement.

the static expressions would have the same return values, but return
nomem when there is not enough capacity, not when the allocation
fails. because it's consistent with the non-static variants, the
programmer has to keep less in mind and i think this is also just
logical.

also, i don't think that we should change delete() here. it's not
very likely that a re-allocation fails and even then you can just
continue to use the old allocation, so it doesn't really matter
that much.

                     STANDARD LIBRARY IMPLICATIONS

errors::nomem is removed. it's used nowhere in the standard libary
except once in dirfdfs.

most standard libary functions should be able to handle allocation
failure simply by doing, for example, append()? and having "nomem"
in their "::error" type. users of these functions can then decide
how they want to handle allocation failure.

                         ECOSYSTEM IMPLICATIONS

this will probably break most hare libaries and apps ever written. it
should, however, be pretty easy to write a script that just appends an
"!" to all calls to alloc, insert and append.

                              EXAMPLE CODE

let y = match (alloc(1)) {
case let y: *int =>
	yield y;
case nomem =>
	[...]; // do Something on OOM
};

// could be optimized to "rt::malloc_or_abort()" or something pretty easily
let slice = alloc([1, 2, 3])!;

fn test() ([]int | nomem) = {
	// could be optmized to `if [[unlikely]] (x == null) return nomem else yield x;`
	let x = alloc([...])?;
	return x;
};

                              RELATED RFCS

mostly stolen from autumnull@:
https://lists.sr.ht/~sircmpwn/hare-dev/%3C09e046d7-1e38-1362-4870-1448781f956c@posteo.net%3E
Details
Message ID
<D0FWKN55J28E.FGGSJ2TWXK1Z@sebsite.pw>
In-Reply-To
<ZhE7J8IveVPdVdJw@xha.li> (view parent)
DKIM signature
pass
Download raw message
+1

On Sat Apr 6, 2024 at 8:08 AM EDT, Lorenz (xha) wrote:
> nomem becomes a builtin type. it's error flag is always set.  basicly
> another void, unfortionately

Yeah... I'm not a fan of this, but I've thought about allocation failure
handling quite a bit and I think this is the best approach.

> the return values are changed like so:
>
> 	alloc() returns (*T | nomem)
> 	static(opt) append() returns (void | nomem)
> 	static(opt) insert() returns (void | nomem)

+1, but see below

> the static expressions would have the same return values, but return
> nomem when there is not enough capacity, not when the allocation
> fails. because it's consistent with the non-static variants, the
> programmer has to keep less in mind and i think this is also just
> logical.

I hadn't really thought about this; I just assumed that the static
variants would always return void, but I guess this makes sense? I'm
indifferent about this.

> also, i don't think that we should change delete() here. it's not
> very likely that a re-allocation fails and even then you can just
> continue to use the old allocation, so it doesn't really matter
> that much.

The spec requires that reallocation in delete() will never fail, so yeah
+1 to not making any changes here.

> this will probably break most hare libaries and apps ever written. it
> should, however, be pretty easy to write a script that just appends an
> "!" to all calls to alloc, insert and append.

Many functions which currently can't error would now be able to error,
so I think we should provide a more sophisticated program which can
correctly handle more of this (i.e. hareupdate (sorry for putting
lossless AST off for so long i promise i'll get to it at some point))

Also, we should encourage libraries to use error propagation here
instead of unconditionally aborting.
Lorenz (xha) <me@xha.li>
Details
Message ID
<ZhdgpdBltOep4YgQ@xha.li>
In-Reply-To
<D0FWKN55J28E.FGGSJ2TWXK1Z@sebsite.pw> (view parent)
DKIM signature
missing
Download raw message
thanks for the feedback! unless there is some more feedback, my
plan is to implement this RFC this weekend.

On Tue, Apr 09, 2024 at 05:17:22PM -0400, Sebastian wrote:
> > this will probably break most hare libaries and apps ever written. it
> > should, however, be pretty easy to write a script that just appends an
> > "!" to all calls to alloc, insert and append.
> 
> Many functions which currently can't error would now be able to error,
> so I think we should provide a more sophisticated program which can
> correctly handle more of this (i.e. hareupdate (sorry for putting
> lossless AST off for so long i promise i'll get to it at some point))

i mean - if you can get hareupdate up and running that'd be nice, otherwise
i wouldn't want to create the next RFC that is blocked on the lossless AST.

i can convert the stdlib manually. the nice thing about this is that it
will actually not break silently.
 
> Also, we should encourage libraries to use error propagation here
> instead of unconditionally aborting.

yes, i aggree. +1
Details
Message ID
<D0INUBI1DBJI.1VDC1G2DH8GYI@turminal.net>
In-Reply-To
<ZhE7J8IveVPdVdJw@xha.li> (view parent)
DKIM signature
pass
Download raw message
>                               RFC SUMMARY
>
> hare builtin's should have some way of handling allocation failures.
>
> most apps probably don't want to handle allocation failure but
> rather crash. this proposal does not create any performance impact or
> additional complications in these cases, except adding a trailing "!"
> to builtins that might return nomem.
>
>                          LANGUAGE IMPLICATIONS
>
> nomem becomes a builtin type. it's error flag is always set.  basicly
> another void, unfortionately; 

We're having a bit of an explosion of these :(
It's not obvious what else to do though. At least for alloc it'd be great if we
found a way to actually use the fact that we can signal failure through NULL.


> as autumnull@ already explained in the
> previous proposal, rt::nomem (or errors::nomem) doesn't make sense
> because you would always have to import it. i think that its fine
> to add a builtin for handling errors from builtins.
>
> the return values are changed like so:
>
> 	alloc() returns (*T | nomem)
> 	static(opt) append() returns (void | nomem)
> 	static(opt) insert() returns (void | nomem)
>
> i think that the special case for alloc(), i.e., returning a type
> based on it's argument and type hint, should be fine, because it's
> very obvious to the user what is happening and not very hard to
> implement.
>
> the static expressions would have the same return values, but return
> nomem when there is not enough capacity, not when the allocation
> fails. because it's consistent with the non-static variants, the
> programmer has to keep less in mind and i think this is also just
> logical.

I'm a bit skeptical of this. It's tempting to let the user handle failed static
appends/inserts, but I'm not sure this is actually desired. Can you give a real
world example where this would be useful?


> also, i don't think that we should change delete() here. it's not
> very likely that a re-allocation fails and even then you can just
> continue to use the old allocation, so it doesn't really matter
> that much.

If we want to delete to always succeed, we need to specify that (and fix
rt::unensure's behavior)

>                      STANDARD LIBRARY IMPLICATIONS
>
> errors::nomem is removed. it's used nowhere in the standard libary
> except once in dirfdfs.
>
> most standard libary functions should be able to handle allocation
> failure simply by doing, for example, append()? and having "nomem"
> in their "::error" type. users of these functions can then decide
> how they want to handle allocation failure.
>
>                          ECOSYSTEM IMPLICATIONS
>
> this will probably break most hare libaries and apps ever written. it
> should, however, be pretty easy to write a script that just appends an
> "!" to all calls to alloc, insert and append.

I think it's worth considering special versions - alloc?, alloc!, and the same
for insert and append. The fact that they're special makes it more clear that
there's more than just calls to regular functions and that the values it's
operating on are not really just usual tagged unions. I realize this makes
lexing harder, but I don't think that's very important here, what matters the
most is what's best for the users.
Details
Message ID
<D0INYEI7DJRZ.2ZAP4KJUGRATW@turminal.net>
In-Reply-To
<ZhdgpdBltOep4YgQ@xha.li> (view parent)
DKIM signature
pass
Download raw message
> thanks for the feedback! unless there is some more feedback, my
> plan is to implement this RFC this weekend.

You're of course welcome to implement the rfc before it's approved or even
thoroughly discussed (it can help the discussion, even), but note that we're
moving kinda slowly around here and one week is generally not enough time for
all the relevant people to give feedback, so you may have to change substantial
things in the implementation afterwards.
Lorenz (xha) <me@xha.li>
Details
Message ID
<ZicuZeAh9T-sqkvM@xha.li>
In-Reply-To
<ZhE7J8IveVPdVdJw@xha.li> (view parent)
DKIM signature
missing
Download raw message
*ping*

anyone? a simple "+1" would also be very much appriciated so i know
if i can continue implementing this.

On Sat, Apr 06, 2024 at 02:08:07PM +0200, Lorenz (xha) wrote:
>                               RFC SUMMARY
> 
> hare builtin's should have some way of handling allocation failures.
> 
> most apps probably don't want to handle allocation failure but
> rather crash. this proposal does not create any performance impact or
> additional complications in these cases, except adding a trailing "!"
> to builtins that might return nomem.
> 
>                          LANGUAGE IMPLICATIONS
> 
> nomem becomes a builtin type. it's error flag is always set.  basicly
> another void, unfortionately; as autumnull@ already explained in the
> previous proposal, rt::nomem (or errors::nomem) doesn't make sense
> because you would always have to import it. i think that its fine
> to add a builtin for handling errors from builtins.
> 
> the return values are changed like so:
> 
> 	alloc() returns (*T | nomem)
> 	static(opt) append() returns (void | nomem)
> 	static(opt) insert() returns (void | nomem)
> 
> i think that the special case for alloc(), i.e., returning a type
> based on it's argument and type hint, should be fine, because it's
> very obvious to the user what is happening and not very hard to
> implement.
> 
> the static expressions would have the same return values, but return
> nomem when there is not enough capacity, not when the allocation
> fails. because it's consistent with the non-static variants, the
> programmer has to keep less in mind and i think this is also just
> logical.
> 
> also, i don't think that we should change delete() here. it's not
> very likely that a re-allocation fails and even then you can just
> continue to use the old allocation, so it doesn't really matter
> that much.
> 
>                      STANDARD LIBRARY IMPLICATIONS
> 
> errors::nomem is removed. it's used nowhere in the standard libary
> except once in dirfdfs.
> 
> most standard libary functions should be able to handle allocation
> failure simply by doing, for example, append()? and having "nomem"
> in their "::error" type. users of these functions can then decide
> how they want to handle allocation failure.
> 
>                          ECOSYSTEM IMPLICATIONS
> 
> this will probably break most hare libaries and apps ever written. it
> should, however, be pretty easy to write a script that just appends an
> "!" to all calls to alloc, insert and append.
> 
>                               EXAMPLE CODE
> 
> let y = match (alloc(1)) {
> case let y: *int =>
> 	yield y;
> case nomem =>
> 	[...]; // do Something on OOM
> };
> 
> // could be optimized to "rt::malloc_or_abort()" or something pretty easily
> let slice = alloc([1, 2, 3])!;
> 
> fn test() ([]int | nomem) = {
> 	// could be optmized to `if [[unlikely]] (x == null) return nomem else yield x;`
> 	let x = alloc([...])?;
> 	return x;
> };
> 
>                               RELATED RFCS
> 
> mostly stolen from autumnull@:
> https://lists.sr.ht/~sircmpwn/hare-dev/%3C09e046d7-1e38-1362-4870-1448781f956c@posteo.net%3E
Details
Message ID
<D0S1TURQOBHV.2Y6AQQ5OXLAAF@sebsite.pw>
In-Reply-To
<ZicuZeAh9T-sqkvM@xha.li> (view parent)
DKIM signature
pass
Download raw message
On Mon Apr 22, 2024 at 11:43 PM EDT, Lorenz (xha) wrote:
> *ping*
>
> anyone? a simple "+1" would also be very much appriciated so i know
> if i can continue implementing this.

+1. Even if we end up changing things, having an actual implementation
will be useful, and I can't imagine you'll need to throw it out and
start over or anything.

I still sorta dislike nomem being another "void" type, but I have no
better ideas. It wouldn't bother me nearly as much if we didn't have
`done`, but `done` actually is nearly identical to void, with some minor
differences here and there, and nomem will be pretty much the same
story.
Reply to thread Export thread (mbox)