Hello all,

I'd like to propose an optional extension for development-focused 
emulators to implement. The purpose is to ease development by adding 
functionality to the System port for protecting memory, printing 
backtraces, and the like.

It hinges on the .System/debug port, adding several magic bytes that, 
when received, trigger certain functionality.

Here are the operations that are currently implemented in Bureaucrat (a 
Uxn language+emulator that I've been working on for a bit):

- 0x40 .System/debug DEO: Prints a backtrace to stdout, iterating over 
the return stack and converting addresses to their respective functions 
(and adding line/column/file info where possible). Does not empty rst.
   - No arguments.
   - Screenshot: 
https://cdn.masto.host/mastodongamedevplace/media_attachments/files/111/739/021/759/050/685/original/e1d57c3feac53fbb.png

- #41 .System/debug DEO: Protect memory region.
   - Signature is (ptr* len* -- )
   - If region is already protected or is contained in an 
already-protected region, emulator prints a backtrace and exits.

- #42 .System/debug DEO: Unprotect memory region.
   - Takes single pointer (short) argument (ptr* -- )
   - Pointer MUST be the beginning of a previously-protected region.
   - If the region isn't protected or it's not the beginning of the 
region, the emulator prints a backtrace and exits.

- #43 .System/debug DEO: Enter privileged mode, where writing to 
protected memory is permitted.

- #44 .System/debug DEO: Exit privileged mode.

- #45 .System/debug DEO: Assert that a region of memory has been 
protected, and crash otherwise.

These commands are, of course, tailored to my needs, but I think most of 
them are generally useful. The backtrace is of course very useful given 
ROMs can have difficulty finding their symbol file (and it wouldn't 
contain line/column info), and memory protection features are great for 
debugging buffer overflows, setting variables to "constant", etc. I've 
personally used memory protection in my allocator, for protecting memory 
chunk headers to quickly locate overflows/corruption.

Do note that roms SHOULD NOT be distributed relying on this extension; 
it's strictly a debugging extension only and not for general use. The 
extra bytes do add up to a larger ROM and the memory protection commands 
are not backwards compatible (since the emulator pops items off the stack).

Some ideas for new/changed features:

- The backtrace command is possibly to specific. Perhaps a revised 
command, that takes an address and destination buffer off the stack and 
pushes debug info for that address, would be a better fit.

- The commands that take arguments are not backwards compatible; 
emulators that don't implement DAMPE wouldn't know to pop them off the 
stack. However, it's possible to make it backwards compatible, by 
specifying that the ROM itself (not the emulator) should pop the 
arguments off the stack, e.g. instead of

	;SPRITES #0190 #41 .System/debug DEO

it would be

	;SPRITES #0190 #41 .System/debug DEO POP2 POP2

Downside is increased bloat, though that probably doesn't matter for 
debugging purposes. Upside is that if a rom is accidentally distributed 
using this extension, it will at least be runnable.

- Command to print all currently protected regions to stdout. Useful for 
debugging, I suppose. Then again, probably better to build this into the 
emulator itself as a keyboard shortcut.

- Multiple levels of protection: read-only, exec-only, exec/read-only, 
etc. Not sure if its usefulness would justify the complexity it would add.

I'd love to hear your feedback on this.

Hi Kiëd,

Development extensions should remain emulator-centric I think, I like 
the idea of expanding what the debug port can do tho, so if we can agree 
which format we want it in, I could add some details on this in the docs 
🙂 Normally, once an application is stable enough, I remove all the 
calls to the debug port before shipping the rom, so I never considered 
that specific one too important to exist across all emulators.

What you use the debug port for, should really be in the /expansion 
port, especially since it creates unpredictable stack depths and doesn't 
follow the opcode's arity. This will allow you to pass an address to a 
memory-safety program stored in memory. That would be the way to go I 
think 🙂

Dll

On 2024-02-06 08:02, Kiëd Llaentenn wrote:
> Hello all,
>
> I'd like to propose an optional extension for development-focused 
> emulators to implement. The purpose is to ease development by adding 
> functionality to the System port for protecting memory, printing 
> backtraces, and the like.
>
> It hinges on the .System/debug port, adding several magic bytes that, 
> when received, trigger certain functionality.
>
> Here are the operations that are currently implemented in Bureaucrat 
> (a Uxn language+emulator that I've been working on for a bit):
>
> - 0x40 .System/debug DEO: Prints a backtrace to stdout, iterating over 
> the return stack and converting addresses to their respective 
> functions (and adding line/column/file info where possible). Does not 
> empty rst.
>   - No arguments.
>   - Screenshot: 
> https://cdn.masto.host/mastodongamedevplace/media_attachments/files/111/739/021/759/050/685/original/e1d57c3feac53fbb.png
>
> - #41 .System/debug DEO: Protect memory region.
>   - Signature is (ptr* len* -- )
>   - If region is already protected or is contained in an 
> already-protected region, emulator prints a backtrace and exits.
>
> - #42 .System/debug DEO: Unprotect memory region.
>   - Takes single pointer (short) argument (ptr* -- )
>   - Pointer MUST be the beginning of a previously-protected region.
>   - If the region isn't protected or it's not the beginning of the 
> region, the emulator prints a backtrace and exits.
>
> - #43 .System/debug DEO: Enter privileged mode, where writing to 
> protected memory is permitted.
>
> - #44 .System/debug DEO: Exit privileged mode.
>
> - #45 .System/debug DEO: Assert that a region of memory has been 
> protected, and crash otherwise.
>
> These commands are, of course, tailored to my needs, but I think most 
> of them are generally useful. The backtrace is of course very useful 
> given ROMs can have difficulty finding their symbol file (and it 
> wouldn't contain line/column info), and memory protection features are 
> great for debugging buffer overflows, setting variables to "constant", 
> etc. I've personally used memory protection in my allocator, for 
> protecting memory chunk headers to quickly locate overflows/corruption.
>
> Do note that roms SHOULD NOT be distributed relying on this extension; 
> it's strictly a debugging extension only and not for general use. The 
> extra bytes do add up to a larger ROM and the memory protection 
> commands are not backwards compatible (since the emulator pops items 
> off the stack).
>
> Some ideas for new/changed features:
>
> - The backtrace command is possibly to specific. Perhaps a revised 
> command, that takes an address and destination buffer off the stack 
> and pushes debug info for that address, would be a better fit.
>
> - The commands that take arguments are not backwards compatible; 
> emulators that don't implement DAMPE wouldn't know to pop them off the 
> stack. However, it's possible to make it backwards compatible, by 
> specifying that the ROM itself (not the emulator) should pop the 
> arguments off the stack, e.g. instead of
>
>     ;SPRITES #0190 #41 .System/debug DEO
>
> it would be
>
>     ;SPRITES #0190 #41 .System/debug DEO POP2 POP2
>
> Downside is increased bloat, though that probably doesn't matter for 
> debugging purposes. Upside is that if a rom is accidentally 
> distributed using this extension, it will at least be runnable.
>
> - Command to print all currently protected regions to stdout. Useful 
> for debugging, I suppose. Then again, probably better to build this 
> into the emulator itself as a keyboard shortcut.
>
> - Multiple levels of protection: read-only, exec-only, exec/read-only, 
> etc. Not sure if its usefulness would justify the complexity it would 
> add.
>
> I'd love to hear your feedback on this.

Thank you for the criticisms, I went ahead and spent an hour or so 
changing DAMPE to use .System/expansion commands. I think I prefer this 
method.

I could have left priv-enter/priv-exit as using .System/debug, since 
they don't have arguments, but I think it's best to maintain 
consistency. Thus, they all use expansion commands.

The expansion command is as follows:

NAME        ID    FIELDS
backtrace    0x40    n/a
protect        0x41    ptr* len*
unprotect    0x42    ptr*
priv-enter    0x43    n/a
priv-exit    0x44    n/a
assert-prot    0x45    ptr*

I'm still not sure whether to use the UUID extension or not. On one 
hand, this is a development extension, so not super likely to come into 
catastrophic collision with something -- plus I'm not a fan of the added 
complexity. On the other, best to future proof.

After a bit of thought, I might also remove the backtrace command with a 
get-debug-info command that's more generic. Then again, wouldn't hurt to 
have both, especially if I use UUIDs.

Thoughts?

P.S. Forgot to specify what "dampe" means -- it's just "Debug and Memory 
Protection Extension." More of a placeholder name than anything else.