Hi, everyone!

So, the first month of development is already behind, and it started out
well!
 
--- Progress Meter ---

- [WIP] Knowledge-Representation Improvements
  - [DONE] Better syntax, cleaner API & refactoring
  - Meta info in slots
  - Advice
  - Multi-Methods (value-based)
- Constraint Management Improvements
  - Better Syntax
  - Parameterized pointer variables
  - Advice (for constraints)
  - Store constraints as KR objects
  - Async execution
  - Change predicates
- 2D Graphics
  - Linear Algebra and Geometry additions
  - 2D API for geometry + Backend (via SDL)
  - Event objects + Backend (via SDL)
  - Text API + Backend (via Pango)
- Cells
- Configurations
- Some demo (after everything else)
 
--- What has been done ---

First and foremost, I have set up a Wiki which acts both as interim
documentation and as a learning resource, see [1].

Next, I have refactored the core Knowledge-Representation code. It
wasn't pretty and had many questionable decisions. So many, in fact,
that after throwing most of them away, I was left with a meager ~800
lines of core functionality. This also turned out to be not only buggy,
but a bit weird in that it was using both a push and a pull model for
the sake of memory preservation. This is not necessary now, and I doubt
it was necessary back when it was written. I have rewritten all this,
yielding only ~400 lines of code due to a slightly simplified model and
a more straightforward approach. All the tests run fine, so that's good.

The API got a much-needed simplification as well. Now it boils down to
just these few operations:

  Ξ ← ↓ ↑ → ↓↓

The breakdown is as follows:

  Ξ creates a schema: (Ξ name (first "John") (last "Doe"))
  ← gets a value: (← name 'first)
  ↓ destroys a slot
  → sets a value: (→ name 'last "McCarthy")
  ↑ calls a function in the slot, passing the object as the first arg
  ↓↓ destroys the schema

There's a bit more to it than that for getting or setting a value,
though. For instance, one might want to just get the local value (as
opposed to a possibly inherited one); or write without automatically
updating the inherited values (for performance). This requires some
further syntax. For getting a local value, this is done like this:

  (← name (local 'first))

So, `local' here is just a macro which spits out an expanded slot
declaration:

  (← name (:key 'first :local t))

These declarations may get combined and they seem to provide a pretty
straightforward interface without any need to create an abundance of
specialized get/set functions.

Besides these simple functions and macros, there will likely be just
some mapping constructs exported for walking the inheritence
hierarchy. So, overall, it's looking pretty minimal so far (of course,
this is just for the KR itself, there's also the constraints-related
stuff (which is also pretty minimal)).

As for the use of non-ASCII symbols: typing these isn't really a
problem, and they really make the code look better and much more
compact.

--- The immediate plans and further thoughts ---

- Within a few days, I will reintegrate the new code with the
  constraints engine (which was thankfully written by different people
  and is of much higher quality than the KR core it was built for).

- The core KR API is minimal, but there are no aggregate operations
  available for these objects. But they are just maps! Not good. The
  solution I am set for right now is to expose the container directly to
  the user: just via a ~:container~ slot. This one has to be functional,
  and, so, fset will be used. So, I am pretty much offloading the work
  of providing the bulk operations on maps to fset (a well-tested and
  well-known functional library). So, if you want to walk all the slots,
  you just take ~:container~ and walk through it however you prefer. Or
  if you want to take an intersection, you call fset:map-intersection on
  the containers of the schemas in question, and you are done.

  Writing to ~:container~ can also have semantics, and this is yet to be
  thought out.

- The interface for type information, read-only slots, etc will be
  universal: these will simply be schema slots in the schema itself. So,
  to set the type of a slot, you would just do:

    (→ name :slot-type 'first 'cl:string)

  where :slot-type is just another schema.

  This is very neat because you can just reuse the KR interface for
  doing all kinds of things. Advice will be done the same way.

- Declaring a type for a slot and then getting or setting a value can
  emit type information dynamically into the compiled code when the
  object is available. So, the type info can actually be used for
  optimization.

- I have seperated the goal of /Configurations/ from /KR/. I figured
  that configurations don't require any special kind of inheritience if
  you structure your program the right way, which shouldn't be a
  problem.

- I decided not to proceed with generalized selectors (which would allow
  custom containers instead of just hash tables). In principle, this is
  cool to think about, but so far everything this could give can be done
  by the standard means anyway. So, unless I encounter a really good use
  case for this, I will deem it an unnecessary feature.

- There's a new item on the list: "Store constraints as KR objects". The
  problem with the constraints right now is that they are local to the
  object and inheritence doesn't work on them. If they were represented
  as KR objects, then these objects, when created, could inherit from
  the parent slot. It's the same kind of inheritence which will be used
  for contexts. Moreover, this paves a natural way to add advice to the
  constraint methods.

- I have made a decision to disallow keywords as slot names for the user
  code. This just wouldn't be good. Keywords will only be used for core
  KR needs (and, potentially, for its extensions).

- The standard generic methods won't do. And even though named schemas
  could declare namesake types, and type-based dispatch is also possible
  (via something like [2]), it won't do either. If we learn from Clojure
  [3], there's a more practical notion of a dispatch function which
  examines the actual values to help determine what methods to run. The
  overall rationale behind that is pretty solid: value-based dispatch is
  just more meaningful than types (Rich Hickey gives an interesting talk
  on this here [4]).

  However, I won't just copy the multi-method interface like from
  Clojure. I think it could be more transparent, especially for
  inspection, and the prefer-method deal is kind of
  strange-looking. There's also no advice. I have a hunch of how to do
  this: by representing the multi-method as a decision tree (built with
  KR). It can then be exposed to the user for direct inspection and
  modification. This would easily yield an advice interface and expose
  the ordering (via a ~slot-order~ slot). So, this is just exposing the
  building block, not simply the API.

  Further on, such dispatch could later be extended to the methods used
  in constraints (and I would like to think without much fuss).

  All this, in fact, could actually yield decent performance, especially
  with declarations. In certain cases, probably better than CLOS, and
  within declarations that allow inlining, certainly better than
  standard CLOS. We shall see, of course.

  So, I will be rolling my own for this. The whole MetaObject Protocol
  arcana is just not very elegant. (And it would yield 0 benefits
  anyway.)
 
- Added a goal: build a demo, when all the features are ready. It won't
  have much to do with structural text editing, will just be some small
  application showcasing all the facets of Fern. Will be done when
  everything else is, of course.

---

I estimate it will take about a month to sort through the rest of the KR
stuff, to add the necessary features. Then, probably, another month or
so for the constraint management improvements. (Just eyeballing this
now, and assuming the constraint engine won't give me trouble.) So, it
wouldn't be bad to start working on some actual graphics some time in
May, if everything goes well.

If anyone has comments or questions, just start a thread on the
dicussions or dev list here [5] or contact me directly.

PS Barring unforeseen circumstances, I will be publishing progress
reports on the 23rd of each month.

Thank you!

Best,

  -- Dmitrii Korobeinikov

[1] https://project-mage.org/wiki
[2] https://github.com/digikar99/polymorphic-functions
[3] https://clojure.org/reference/multimethods
[4] https://www.youtube.com/watch?v=YR5WdGrpoug
[5] https://lists.sr.ht/