User-Agent: Cyrus-JMAP/3.5.0-alpha0-1369-gd055fb5e7c-fm-20211018.002-gd055fb5e
Mime-Version: 1.0
Message-Id: <986582af-5b0c-4be7-8b7d-204af693ff3f@www.fastmail.com>
Date: Sun, 31 Oct 2021 17:26:02 -0400
From: "Miles Alan" <m@milesalan.com>
To: ~mil/mepo-devel@lists.sr.ht
Subject: Mepo - November update: 0.1 Released
Content-Type: text/plain

Welcome to the first in a series of monthly updates focused on mepo's
development. The plan is to send an email to the mailing list at the
end of each month to keep everyone following the project updated.

---------------------------------------------------------------------------
Contents:

Status update - high-level updates from October:
- General stability improvements (memory leaks resolved, errorchecking logic)
- LIFO datastructure
- Tile-retrying logic implemented
- Text rendering logic improved
- Contributions
- Mepo 0.1 tagged

Planning - high-level plans for November:
- Support for non-interactive downloading & CLI (video-free context)
- Rework concurrency model
- Convenience features & planning for pin groups
- Plan to tag 0.2

---------------------------------------------------------------------------
October - status update:

1. DONE - General stability improvements:
A large part of October's work focused around improving the core
stability of the application. Previously, mepo could fail in a number
of ways usually resulting in directly crashing the application! These
failure cases ranged from race-conditions, to bad tile data not being
consistently error checked, to memory leaks, and more. These errors have
on the whole been resolved.  A good deal of robustness and consistency
has been added to the main application logic to handle error & edge
cases. Numerous memory leaks have been resolved and SDL & curl downloading
errors are now consistently checked. Crashes now should be rare in normal
operation and should be reported on the ticket system. Further stability
in mepolang and the parser will come later in the roadmap as well!

2. DONE - LIFO datastructure:
Another large push in October focused on updating the core-datastructure
used for tile downloading to be a LIFO (last-in-first-out queue). The idea
behind this change is that whatever you're viewing on the screen is the
last coordinates requested, as such that coordinate should take priority
in downloading. Implementing a LIFO means that rendering within the
UI is substantially faster since simply by panning the screen quickly,
the application logic will always set whatever you're seeing on the
screen to download immediately.  Beyond improving UI responsiveness, this
change also sets things up nicely for offline downloading since now the
LIFO can be used to push long-queued tile coordinates in the background
(since they'll be automatically de-prioritized in normal UI usage,
wherein on-screen tiles will download first due to the LIFO policy).

3. DONE - Tile-retrying logic implemented:
When tile data fails to download - since early in the project, the
application's downloading thread would halt. This bug has been resolved
and additionally logic has been implemented such that upon failure the
requested coordinate is pushed back into the queue. As such, if you're
offline now and using mepo (or even if just a request times out or
similar), the tile will automatically retry until online and tile data
comes through successfully. This behavior takes place ad-infinitum,
so things will retry until you're back online or the server responds
accordingly. In the future a max-retry-limit may be implemented, but
for now to keep implementation simple there is no retry limit.

4. DONE - Text rendering logic improved:
A smaller but important change that took place has to do with improvements
in text rendering. Text rendering now supports arbitrary alignment
(horizontally and vertically); centering / top / bottom / left / right
alignments. This change was done to support centering the help-text,
improve rendering for the pin overlay, and debug messages. But in
addition, this alignment feature will also pave the way for centering text
information for pins eventually too.  Another component of text-rendering
that was improved additionally is that transparency support was added for
backgrounds and also multiple font sizes are supported in the underlying
logic as well now.

5. CONTRIBUTIONS - From ~amk, ~anjan, and ~jpsamaroo
I did also just want to make a small note here to thank the contributors
who sent patches which have been merged. From git shortlog up to 0.1:

Alex McGrath (1):
      Use a MEPO_MENU env variable for menu scripts

Anjandev Momi (2):
      docs: instructions to install on alpine
      mepo_helper_menu.sh: use bemenu on wayland

Julian P Samaroo (6):
      mepo: shellpipe: Print stderr on non-0 exitcode
      mepo: mepo_bbox_search: Escape single quotes
      mepo_bbox_search: Fix long multi-word pin names
      mepo_bbox_search: Ignore empty entry
      nominatim: die gracefully on empty COORDS
      nominatim_searchprompt: Use busybox grep

6. DONE - Mepo 0.1 tagged:
Finally, the most important news of all is that mepo 0.1 has finally been
tagged and a tarball is available! Also thanks to Frank J Cameron (~fjc)
for pushing forward an initial stab at the alpine package which I expect
to be merged in the coming days, at which point a first testing version
of mepo should be available in the alpine testing repository then!

See:
https://gitlab.alpinelinux.org/alpine/aports/-/merge_requests/22055

---------------------------------------------------------------------------
November - agenda:

So what's on the agenda for November? Mainly I'll be focused on improving
offline downloading (Milestone 2), the concurrency model, and a few other
minor aspects & convenience features.

The primary goal of November will be to build out robust support for
offline downloading (Milestone 2). The idea overall is to rework mepo's
core application logic to be usable in such a way that the application
can be used for downloading without SDL video context. In this way there
will be a nice CLI (e.g. mepolang), for downloading tiles for a whole
city, based on a radius from a certain point, etc. Offline operation is
described as being a first-class in the README and I want to sync things
up so this is actually the case. This feature should put mepo ahead of
many other linux OSM map viewers in regards to this usecase and along
with good documentation should aide in adoption.

Along with the work planned for offline downloading, a secondary goal
for November is to rework the concurrency model overall as this work
dovetails nicely together with the offline downloading work. Currently,
the downloading of tiles is based on a separate (SDL) thread which works
via a SDL delay. This has many downsides such as: limiting downloading
speed based on that delay, lockups in the UI related to mutex locking
/ the async model, and making the logic overall conceptually more
complex then things need to be. The plan to rework this logic is to use
curl_multi_wait / wait on FDs rather then the SDL delay and possibly also
attempt to move back to a single threaded model with select / FD polling
(since curl_multi abstracts the concurrency work needed in anycase).

Other then that, within November, I plan to add some other convenience
features to the codebase such as: a new mepolang command that'll
selectively clear the download cache, logic to convert from lat/lon to
km/miles, and will likely start planning or may even start implementing
the refactoring for pin groups. I will plan to complete Milestone 2 by
the end of November but I think more may be doable then what is listed; 
so stay tuned. At the end of November I'll plan to tag 0.2.

---------------------------------------------------------------------------

So that's about it for this update - thanks for reading. Please put in
tickets, drop in IRC, or send a note to the mailing list if you have
any feedback or ideas for improvements.

Cheers,
Miles