Weekly status update (2020-20)

Message ID
DKIM signature
Download raw message

Moderation tools

Hey there,

welcome to the weekly rambly e-mail :)  The last week I wasn't nearly as
productive as the week before, but there's still some news to share, and
also things that I might need help with.  If you're interested in
helping out (but don't want to read all of this up-front), scroll to the
bottom of this e-mail to see a list of smaller-ish tasks that can be
helped with.

Let's dive in!

libqaul chat service

I announced last week that the chat service was mostly working, and well
now it's mostly mostly working ;)  There were a few bugs in the service
code, and one bug in Ratman that prevented the chat service from working
properly.  But a lot of this seems to be fixed now.  We have a lot more
working tests, both in Rust code and using the RPC json/http interface.
The bug in ratman might have caused other transitive problems down the
line so I'm glad I was able to find that.

(in short, it would sometimes hang when a new user had been discovered
if libqaul wasn't actively polling for new users because it was busy
with another request.  Basically a 5-dimentional deadlock.  Fun stuff)

qaul.net over netmod-tcp

I also managed to hit a nice milestone this week: getting the first
connection over the tcp overlay working!  Two qaul.net instances (the
hubd server), running on two separate computers on my home network,
talking to each other!  Unfortunately there's a bug that prevents it
from being super useful at the moment, and it only happens when running
on two different computers; not in our test suite.  Weird!

Basically: the netmod-tcp driver sends a "hello" to each peer that it's
supposed to be peering with.  If this peer has the received IP in it's
peer list, it sends a hello back, making a handshake, and initialising a
back and forth of keep-alive packets.  On my local machine running both
endpoints this back and forth of tiny "pings" will go on forever (well,
I ran it for ~10 minutes).  When running between my desktop and my file
server, it will crash with an "unexpected EOF" after about 10 seconds.
I'm not really sure why this is happening at the moment, and also not
sure about the architecture of the tcp driver in general.

The currenty architecture has some limitations and unimplemented

The first one is dynamic peering, meaning being able to introduce new
peers to existing peers, in case they are relevant: this might require
some deep-frame inspection.  I'll quickly explain: say we have the
topology A - B - C.  A sends a lot of messages to B, over tcp, which
inspects the destination address and id of each frame and keeps track of
whether this frame is being re-transmit over the tcp driver, to another
peer in it's peer list.  If this is the case, after some threathold, it
should ask peer C and A if they support dynamic peering (if B supports
dynamic peering), and then initiate a handshake between A and C because
it will be able to cut out B from all transations between them.

The second feature is re-connections.  The keep alive is meant to make
it easy to spot when a connection has terminated, but I think it would
be much better to remove them altogether and instead replacing the send
mechanism with a re-connect hook that will, on send failure, reconnect
to the peer in question.  If a listener block receives a dropped
connection, it should terminate it, and wait for a re-connect.  This
might only happen one-way, or two way.  If we remove the keep-alive
packets for overhead reasons we might want to have a service that checks
the active sender connections against the registered peers (meaning the
peers that _should_ be connected) and re-connects to any that it has
lost a sending connection to.

Outlook for the next week

Our alpha deadline is coming closer and closer, but I think we're in a
pretty decent shape.  The next week I mainly want to focus on three

1. Getting netmod-tcp into a working shape, even if that means removing
any connection-drop detection it has at the moment.  As long as we can
run demos on a good network connection it'll be fine!

2. Start bootstrapping the Android UI and service integrations  (more
on that in a bit)!

3. Implementing a basic filesharing service.  Basically sending an
announcement into the network advertising a file and then responding to
any file requests that are sent it's way.  No access modifiers or
private stuff yet.  Simple stuff.

I want to help. How?

I'm so glad you asked.  There's many many ways you can, and most of them
are related to Android!  I don't really know very much about Android,
and there's many things to do here.  In no particular order:

- Initialising the web backend (calling some setup methods in Rust and
  starting a server) and connecting a webview to it (making sure the
  webgui is built and packaged into the app, serving it correctly, etc)

- Hooking up a phone state listener to the call service to make the app
  tell the phone that a user is having a phone call

- Getting a stream from the microphone and sending it's data in packaged
  chunks into the "voices" service.  Help build a simple API abstraction
  for Java to be able to call these functions.

- Help me build the WiFi Direct driver that will automatically connect
  to nearby devices, attempt a handshake to see if they're another
  qaul.net device, and then maintain a connection between them.  How
  many devices can be actually be connected to?  Is there a way to build
  connection groups or do they always have to be 1-to-1 (might be
  simpler anyway).  Many many open questions!  I built a small WiFi
  Direct prototype in 2018 [1] but this will probably have to be adapted
  quite a lot!

Note: for the Android stuff I don't particularly care about whether it's
Java or Kotlin.  If you want to help and have a particular language
preference, we'll just use that!

As for the rest of the project:

- Help test the chat service, maybe writing some Rust benchmarks that
  creates a network and sends chat messages through the entire stack to
  measure latency and throughput, at least on a local device (we have no
  idea how performant this is at the moment)

- Help me figure out the connection bug in netmod-tcp please?  I have no
  idea what's going on and a new perspective on things might be very

I'm pretty confident that we'll make our deadline.  Things are coming
together quite nicely.  There's still a few open questions, and
problematic bugs to fix, but hopefully they won't stop us in our

If you are interested in helping on any of these points, please get in
touch with us via this mailing list, our IRC channel (#qaul.net on
freenode - generally a cool place to be, wink wink), or just DM me on
whatever platform you know me from x)

We'll have our weekly dev call tomorrow, at a changed time (!), at 15:00
UTC in https://meet.jit.si/qaul.net

Thank you for your attention!

Reply to thread Export thread (mbox)