~qaul/community

2 2

Weekly status update (2020-21)

Details
Message ID
<871rn8bufo.fsf@kookie.space>
DKIM signature
missing
Download raw message
Hey there,

I don't know why but this week went by extra quickly.  Most of the work
that was done has been around integrating the Rust codebase with various
platforms.  First I'm gonna go through changes that I made this week,
and then also highlight some contributions that made the project better
as a whole.


Android client
--------------

The biggy for this week is the Android client.  It basically didn't
exist last week, and today it exists.  Hurray!  Well, sort of.

The first hurdle was making all the build systems integrate with each
other.  Luckly Mozilla has done most of the work here for us, by writing
a gradle plugin that we can use to cross-compile the Rust code and
hooking it into the Android build step for linking.  This is something
that worked before, sort of.  I renamed librobot to "android-support" to
be more descriptive and also no longer clash with my shell
auto-completion for libqaul :)

Cross-compiling can be a huge hassle, and extra so on my operating
system (nixos).  So I opted to refine our docker builds.  There's now a
"build.sh" script in the clients/android folder that will build the rust
code in a docker container that we provide via dockerhub.  It's a pretty
huge image, I'm still trying to figure out how to trim down some of the
toolchains that we really don't need.

If you want to build the Rust code without throwing away the container
each time (to cache build outputs, etc) you can run the docker command
yourself, but be sure to use clients/android/.build_nested.sh because it
will properly restore file ownership again (the container builds as
root).

Next up was log integration.  Jessica figured out how to make tracing
log in a format that is "log" crate compatible and there's an android
shim for the log ecosystem, so now we have all our tracing logging in
logcat on android.  Sweet!

I stole some more code from firefox to register a panic handler in Rust
that will log panics to show up in logcat too because previously they
just aborted the app.

A bit of an annoying step in building the Android app is the UI.  It's a
bunch of html and js assets and the webserver demands a path on disk to
point to.  The Android ecosystem is pretty weird, but luckily I found a
bit of a hacky work-around.  The assets are being included into the .apk
at compile time and are then extracted into the app-specific internal
storage region at runtime.  The path to them is then given to the
web-server to store things at.  This internal store is also gonna be
useful to put alexandria specific files, but since that's currently
non-persisting anyway, it doesn't matter as much.

A lot of the work on the Android app has been around trying to make the
build process well documented and straight forward.  The webgui doesn't
seem to work properly in all regards yet so I started writing some
native UI elements to do basic things like creating a user ID and
logging in.  I've also started working on integrating with WiFi Direct
although no native bridge code was written for that yet.

That's one of the weird issues I will have to get working next week:
integrating the netmod-wd interface with the kotlin glue code to send
and receive frames.  Oh...and all the code is Kotlin now, because it was
pretty easy to change and seems more future proof.


Other changes and contributions
-------------------------------

A few things I want to mention!  Nora hase been hunting down a bug in
the netmod-tcp that I wrote about last week.  And she managed to find
the problem and fix it, so thank you for that!  It means that Linux
networks are now fully working!  Woo.

Also we've started doing some planning work for the filesharing
service.  Aman has sent in a patch to shuffle some files around to
remove the files capability from the libqaul API and moving it into a
file service instead.

This brings up the problem of sending rpc messages between services, but
I feel like this is not the best time to tackle that problem (and also
there just needs to be an api to probe for the existenc of a service and
then have a semi-generic way to get a valid rpc instance for libqaul so
that services can send each other messages and commands.  Easy stuff :P)


The next week
--------------

So I think the main thing that I'll be busy with the next week is
android integration.  The webgui is bundled into the app now but there's
no internal web view.  And the WiFi Direct driver doesn't work yet.  I
actually hope I can find a second phone to test this with x)

If I run into any problems with WiFi Direct I think I'll have a look at
bluetooth to make connections instead, it might be easier to discovery
potential peers that way.

The native UI components on android will remain pretty limited for now.
But currently the webgui also doesn't seem to work very well, so there's
probably a lot of bugs we have to fix for it to work.

Anyway, that's enough from me.  We're gonna have our regular development
chat again tomorrow at 16:00 UTC on https://meet.jit.si/qaul.net.  Come
by if you wanna help out or just learn about how things are going
otherwise!


~k
Details
Message ID
<6d9a5243-f20d-4c26-a361-9540bc8e981d@www.fastmail.com>
In-Reply-To
<871rn8bufo.fsf@kookie.space> (view parent)
DKIM signature
fail
Download raw message
DKIM signature: fail
The amount of work that goes into Android client is staggering!
I cannot imagine the work iOS might need. The "smart" phone
ecosystem is pretty bleak, isn't it?

> ...  Nora hase been hunting down a bug in
> the netmod-tcp that I wrote about last week.  And she
> managed to find the problem and fix it, so thank you
> for that!  It means that Linux networks are now fully
> working!  Woo.

This is so cool! I wonder if there'd be a blog post for mortals
like me? :)

> Also we've started doing some planning work for the filesharing
> service.  Aman has sent in a patch to shuffle some files around to
> remove the files capability from the libqaul API and moving it into a
> file service instead.

Sorry, I have been horribly slow at this and have bugged Katharina
more than what she could have done by herself. On one hand I
want to continue but on the other hand I feel the deadline is near.

But it is exciting!

-aj

On Sun, May 24, 2020, at 10:49 PM, Katharina Fey wrote:
> Hey there,
> 
> I don't know why but this week went by extra quickly.  Most of the work
> that was done has been around integrating the Rust codebase with various
> platforms.  First I'm gonna go through changes that I made this week,
> and then also highlight some contributions that made the project better
> as a whole.
> 
> 
> Android client
> --------------
> 
> The biggy for this week is the Android client.  It basically didn't
> exist last week, and today it exists.  Hurray!  Well, sort of.
> 
> The first hurdle was making all the build systems integrate with each
> other.  Luckly Mozilla has done most of the work here for us, by writing
> a gradle plugin that we can use to cross-compile the Rust code and
> hooking it into the Android build step for linking.  This is something
> that worked before, sort of.  I renamed librobot to "android-support" to
> be more descriptive and also no longer clash with my shell
> auto-completion for libqaul :)
> 
> Cross-compiling can be a huge hassle, and extra so on my operating
> system (nixos).  So I opted to refine our docker builds.  There's now a
> "build.sh" script in the clients/android folder that will build the rust
> code in a docker container that we provide via dockerhub.  It's a pretty
> huge image, I'm still trying to figure out how to trim down some of the
> toolchains that we really don't need.
> 
> If you want to build the Rust code without throwing away the container
> each time (to cache build outputs, etc) you can run the docker command
> yourself, but be sure to use clients/android/.build_nested.sh because it
> will properly restore file ownership again (the container builds as
> root).
> 
> Next up was log integration.  Jessica figured out how to make tracing
> log in a format that is "log" crate compatible and there's an android
> shim for the log ecosystem, so now we have all our tracing logging in
> logcat on android.  Sweet!
> 
> I stole some more code from firefox to register a panic handler in Rust
> that will log panics to show up in logcat too because previously they
> just aborted the app.
> 
> A bit of an annoying step in building the Android app is the UI.  It's a
> bunch of html and js assets and the webserver demands a path on disk to
> point to.  The Android ecosystem is pretty weird, but luckily I found a
> bit of a hacky work-around.  The assets are being included into the .apk
> at compile time and are then extracted into the app-specific internal
> storage region at runtime.  The path to them is then given to the
> web-server to store things at.  This internal store is also gonna be
> useful to put alexandria specific files, but since that's currently
> non-persisting anyway, it doesn't matter as much.
> 
> A lot of the work on the Android app has been around trying to make the
> build process well documented and straight forward.  The webgui doesn't
> seem to work properly in all regards yet so I started writing some
> native UI elements to do basic things like creating a user ID and
> logging in.  I've also started working on integrating with WiFi Direct
> although no native bridge code was written for that yet.
> 
> That's one of the weird issues I will have to get working next week:
> integrating the netmod-wd interface with the kotlin glue code to send
> and receive frames.  Oh...and all the code is Kotlin now, because it was
> pretty easy to change and seems more future proof.
> 
> 
> Other changes and contributions
> -------------------------------
> 
> A few things I want to mention!  Nora hase been hunting down a bug in
> the netmod-tcp that I wrote about last week.  And she managed to find
> the problem and fix it, so thank you for that!  It means that Linux
> networks are now fully working!  Woo.
> 
> Also we've started doing some planning work for the filesharing
> service.  Aman has sent in a patch to shuffle some files around to
> remove the files capability from the libqaul API and moving it into a
> file service instead.
> 
> This brings up the problem of sending rpc messages between services, but
> I feel like this is not the best time to tackle that problem (and also
> there just needs to be an api to probe for the existenc of a service and
> then have a semi-generic way to get a valid rpc instance for libqaul so
> that services can send each other messages and commands.  Easy stuff :P)
> 
> 
> The next week
> --------------
> 
> So I think the main thing that I'll be busy with the next week is
> android integration.  The webgui is bundled into the app now but there's
> no internal web view.  And the WiFi Direct driver doesn't work yet.  I
> actually hope I can find a second phone to test this with x)
> 
> If I run into any problems with WiFi Direct I think I'll have a look at
> bluetooth to make connections instead, it might be easier to discovery
> potential peers that way.
> 
> The native UI components on android will remain pretty limited for now.
> But currently the webgui also doesn't seem to work very well, so there's
> probably a lot of bugs we have to fix for it to work.
> 
> Anyway, that's enough from me.  We're gonna have our regular development
> chat again tomorrow at 16:00 UTC on https://meet.jit.si/qaul.net.  Come
> by if you wanna help out or just learn about how things are going
> otherwise!
> 
> 
> ~k
> 
> Attachments:
> * signature.asc
Details
Message ID
<87pnar9jor.fsf@kookie.space>
In-Reply-To
<6d9a5243-f20d-4c26-a361-9540bc8e981d@www.fastmail.com> (view parent)
DKIM signature
missing
Download raw message
> The amount of work that goes into Android client is staggering!
> I cannot imagine the work iOS might need. The "smart" phone
> ecosystem is pretty bleak, isn't it?

To an extent, yes.  I think a lot of the problems are generally just
bundling an application, i.e. going from "development" to "production"
phase.  But we should definitely be on the lookout for someone who can
do iOS development for the project in the future because I'm not feeling
like touching that x)


~k
Export thread (mbox)