Hello all,

The ARM port, initially targeting the Raspberry Pi model 1, reached a
significant milestone today: we have prompt!

As documented[1], the rpi port only supports the UART exposed through GPIO
ports. Also, we don't have mass storage yet, only sys/rdln prompt. But it works!

There's still a lot of work to do to make it usable, notably mass storage,
video and USB drivers.

The ARM architecture presents interesting challenges for Dusk and its HAL. The
greatest challenge is its inability to execute, read or write unaligned data[2].
This will mean that a new "alignment discipline" will need to be imposed on Dusk
code. To help enforce that discipline, I think I'll add the same alignment
constraints as ARM in the POSIX VM.

The RISC architecture also reveals some HAL misdesigns, in the sense that some
HAL instructions are really inefficient under ARM. I'll have to make some
adjustments there too.

I think all those challenges can be met elegantly.

Regards,
Virgil

[1]: https://git.sr.ht/~vdupras/duskos/tree/master/item/fs/doc/hw/arm/rpi/rpi.txt
[2]: That is, a 4b word can only be accessed from a 4b-aligned address, a 2b one
from a 2b-aligned address. A byte can be accessed from anywhere.

Significant progress on the ARM port: the EMMC driver on the RPi is now good
enough to mount Dusk's FAT partition and boot from it. This also works on real
hardware.

This shortens the TODO list for this port:

* Add write support to the EMMC driver.
* Implement High HAL for ARM.
* Run the test suite on a RPi (this probably means a lot of
  "alignment discipline" across the code).
* Implement a Grid for VideoCore.
* Add driver for USB keyboard.

Once that's done, we can pretty much say that Dusk OS runs on the RPi.

I've reached another important milestone today: the ARM port passes all tests
in the tests/hal unit.

In the context of that work, I've made an important change to HAL design: there
is no longer a Low HAL and a High HAL, the whole HAL is implemented in kernels.

The drawback of this design is that it makes ports to new architectures harder.
Writing those kernels is some serious meta-compilation sport. Whew, I'm dizzy!

However, it simplifies the architecture as a whole and doesn't add *that* many
lines of code to the kernels. The i386 kernel is still under 1000 lines and the
ARM kernel is around 1200 lines.

An important pain point this organization relieves is to be able the full HAL in
low level code such as fs/fatlo and lib/endian. Dusk really needs the speed
there and the only ideas I had in mind to give it some was to re-implement them
in High HAL later in the boot sequence and realias them. Convulsed, didn't like
it.

Now, theoretically, the only part missing for a functional ARM DuskCC is
alignment discipline throughout Dusk code. There is also the high probability
of ARM HAL bugs lurking due to missing coverage in tests/hal, but in the
transition to the "single HAL" I've just made in i386, I've greatly improved
that coverage.

So, after a few rounds of code consolidation (some parts of the kernels are a
bit messy), that's where I'm going.

Aaaand.. all tests pass on the ARM port! Alignment discipline for the win!

On Wed, Jul 5, 2023, at 10:25 AM, Virgil Dupras wrote:
> Aaaand.. all tests pass on the ARM port! Alignment discipline for the win!

I spoke a bit too fast. All tests pass on QEMU ARM, but not on the real RPi. The
first failure I stumbled upon was alignment-related, but passed under the radar
with QEMU. I have a few other failures which look trickier to debug, but it
still goes pretty well: most DuskCC tests pass.

Shouldn't be too tricky to debug...

On Wed, Jul 5, 2023, at 8:27 PM, Virgil Dupras wrote:
> On Wed, Jul 5, 2023, at 10:25 AM, Virgil Dupras wrote:
>> Aaaand.. all tests pass on the ARM port! Alignment discipline for the win!
>
> I spoke a bit too fast. All tests pass on QEMU ARM, but not on the real RPi. The
> first failure I stumbled upon was alignment-related, but passed under the radar
> with QEMU. I have a few other failures which look trickier to debug, but it
> still goes pretty well: most DuskCC tests pass.
>
> Shouldn't be too tricky to debug...

Aaaand... done for real this time. All tests pass on a real RPi!

On Thu, Jul 6, 2023, at 7:50 PM, Virgil Dupras wrote:
> On Wed, Jul 5, 2023, at 8:27 PM, Virgil Dupras wrote:
>> On Wed, Jul 5, 2023, at 10:25 AM, Virgil Dupras wrote:
>>> Aaaand.. all tests pass on the ARM port! Alignment discipline for the win!
>>
>> I spoke a bit too fast. All tests pass on QEMU ARM, but not on the real RPi. The
>> first failure I stumbled upon was alignment-related, but passed under the radar
>> with QEMU. I have a few other failures which look trickier to debug, but it
>> still goes pretty well: most DuskCC tests pass.
>>
>> Shouldn't be too tricky to debug...
>
> Aaaand... done for real this time. All tests pass on a real RPi!

Raspberry Pi models 2 and 3 are now supported! I don't have a model 2, so I can't
be 100% sure that it works on it, but it works on my model3. If someone with a
model 2 could confirm, it would be nice.

"Virgil Dupras" <hsoft@hardcoded.net> writes:

> On Thu, Jul 6, 2023, at 7:50 PM, Virgil Dupras wrote:
>> On Wed, Jul 5, 2023, at 8:27 PM, Virgil Dupras wrote:
>>> On Wed, Jul 5, 2023, at 10:25 AM, Virgil Dupras wrote:
>>>> Aaaand.. all tests pass on the ARM port! Alignment discipline for the win!
>>>
>>> I spoke a bit too fast. All tests pass on QEMU ARM, but not on the real RPi. The
>>> first failure I stumbled upon was alignment-related, but passed under the radar
>>> with QEMU. I have a few other failures which look trickier to debug, but it
>>> still goes pretty well: most DuskCC tests pass.
>>>
>>> Shouldn't be too tricky to debug...
>>
>> Aaaand... done for real this time. All tests pass on a real RPi!
>
> Raspberry Pi models 2 and 3 are now supported! I don't have a model 2, so I can't
> be 100% sure that it works on it, but it works on my model3. If someone with a
> model 2 could confirm, it would be nice.

Re: Pi and other devices using microSD, is there a simple
flash-friendlier file system that CollapseOS could be using?  I know
OpenWRT uses JFFS2 on a lot of devices.
Or just use FAT and pray that the microSD has a translation layer that
handles FAT?

On Sun, Jul 16, 2023, at 6:50 PM, Csepp wrote:
> "Virgil Dupras" <hsoft@hardcoded.net> writes:
>
>> On Thu, Jul 6, 2023, at 7:50 PM, Virgil Dupras wrote:
>>> On Wed, Jul 5, 2023, at 8:27 PM, Virgil Dupras wrote:
>>>> On Wed, Jul 5, 2023, at 10:25 AM, Virgil Dupras wrote:
>>>>> Aaaand.. all tests pass on the ARM port! Alignment discipline for the win!
>>>>
>>>> I spoke a bit too fast. All tests pass on QEMU ARM, but not on the real RPi. The
>>>> first failure I stumbled upon was alignment-related, but passed under the radar
>>>> with QEMU. I have a few other failures which look trickier to debug, but it
>>>> still goes pretty well: most DuskCC tests pass.
>>>>
>>>> Shouldn't be too tricky to debug...
>>>
>>> Aaaand... done for real this time. All tests pass on a real RPi!
>>
>> Raspberry Pi models 2 and 3 are now supported! I don't have a model 2, so I can't
>> be 100% sure that it works on it, but it works on my model3. If someone with a
>> model 2 could confirm, it would be nice.
>
> Re: Pi and other devices using microSD, is there a simple
> flash-friendlier file system that CollapseOS could be using?  I know
> OpenWRT uses JFFS2 on a lot of devices.
> Or just use FAT and pray that the microSD has a translation layer that
> handles FAT?

Flash wear is of course a problem with FAT because of how that allocation tables
are organized and updated, but I think this problem is greatly reduced in
importance by the fact that Dusk (and Forths in general) is memory oriented:
there's very little disk write going on. Only files directly written by the
operator result in writes. Usually, that will be text files containing source
code and those text files will seldom grow a cluster, so the FAT will not be
used much.

Collapse OS uses traditional Forth blocks, so it's less susceptible to flash
wearing.

I think I'll let the operator manage this problem herself rather than building
complex systems. If that's really a problem with the operator, she can
periodically move her partitions around: Space needed for Dusk OS and Collapse
OS is vastly inferior than the capacity of most cards.