LLL: alloc issues round-up

[benjaminion]

This is an attempt to move the LLL alloc issue on. It supersedes PRs #2489 and #2463, and resolves Issue #2465.

The proposed change to the generated bytecode is the tightest code I can make that handles both the edge cases (alloc 0) and MSIZE being initially zero.

Notes:

• (alloc 0) does not allocate any memory. This is contra @chriseth, but in line with @axic.
• It uses MLOAD rather than MSTORE8 to increase MSIZE. This is a belt-and-braces guard against memory corruption.
• It handles the edge case of MSIZE being intially zero, which is a problem for @axic's approach in [WIP] LLL: change alloc to use mload and fix alloc(0) #2489.

A bunch of test cases is included that should be comprehensive.

Fixes #2465.

[benjaminion]

Note that, alternatively, alloc could be implemented pretty efficiently as a compiler macro as follows:

  (def 'alloc (n) (raw (msize) (when n (pop (mload (+ (msize) (& (- n 1) (~ 0x1f))))))))

The downside of this is that it removes the opportunity to do memory management between alloc and variables - which seems to be the purpose of the finalise function. The memory management is not currently working, so as it stands it makes no practical difference if we do alloc in the parser or in the macros. But if we want to do the memory management in future, alloc ought to be left in the parser where it currently is.

[axic]

This is (msize) - (alloc..), right? So the resulting value should be how much was allocated.

Can you please add that as a comment somewhere?

[axic]

What is weird though it assumes that execution goes as:

• alloc
• msize
• sub

But the compiler could actually execute the two operands in a different order.

[benjaminion]

Hmm. Good spot. I've updated it to be resistant to changes in argument evaluation order. Ugly but effective.

[axic]

Indentation seems to be off here.

[benjaminion]

Aaaagh... must fix my editor...

[axic]

My only nitpicking is I like comments above statements (as opposed to next to them), but I guess it is more readable in this case, since it fits a single window.

Please include your macro version as a comment in the macro section so it is not forgotten. Let's keep the native version for now and see if we can/should fix finalise.

[axic]

Please add a comment here explaining the rules:

• returns msize prior to expansion
• doesn't allocate in case of N = 0
• rounds N to a multiple of 32
• uses MLOAD to expand, which keeps memory untouched

[benjaminion]

Done.

[benjaminion]

My only nitpicking is I like comments above statements (as opposed to next to them), but I guess it is more readable in this case, since it fits a single window.

Comments were mainly for my benefit; happy to keep them, move them, leave them...

Please include your macro version as a comment in the macro section so it is not forgotten.

Done. I put it at the top as some of the other macros could probably usefully be rewritten to use ALLOC now that it is reliable.

I've squashed the commits into one for convenience.

By the way, I wrote some docs.