Hi all,

I've now got a working prototype of the full datalog featureset, and 
I've transitioned all the queries in my forest to datalog. The good news 
is it works; the bad news is, at the end of the day, my forest renders 
in about twice the time that it built with the old queries. Here is what 
it looks like to query for my refereed publications:

   \query\datalog{
     ?X -:
      {\rel/authors ?X @{jonmsterling}}
      {\rel/taxa ?X '{reference}}
      {\rel/tags ?X '{refereed}}
   }

The syntax is a work in progress. A few explanations are in order. 
Vertices in forester's database are either addresses (i.e. which get 
normalised to IRIs against the current host) or they are pure content. I 
am writing @{...} for the latter and '{...} for the former. This syntax 
appears just in the embedded datalog expressions.

You can add new facts (with arbitrary relation symbols) to the database 
as in the following example:

	\let\rel/accepted-or-refereed{jonmsterling.query.bibliography.accepted-or-refereed}

	\execute\datalog{
	 \rel/accepted-or-refereed ?X -: {\rel/tags ?X '{refereed}}
	}
	\execute\datalog{
	 \rel/accepted-or-refereed ?X -: {\rel/tags ?X '{accepted}}
	}

It is necessary to do stuff like this in order to implement queries that 
do unions. You can also use it to compute transitive closures.


------

I think that this performance hit is potentially acceptable if we can 
*also* transition to a standard mode-of-use in which not the entire 
forest is getting rendered all the time, but instead pages are being 
displayed on-demand through a local server that the Forester binary can 
throw up. (And nobody cares if it takes 10s to render once in a while 
prior to uploading to their web site, etc.). But what do you think?

(It may also be that by trying a different datalog engine, we might get 
better performance. I have not really studied the design space yet.)

Anyway, my proposal is that Forester 5.0 include the new engine *and* 
the legacy one, and that the latter be immediately deprecated and then 
removed for Forester 5.1. This gives people time to incrementally 
rewrite their queries without getting screwed over on day one.

Best,
Jon

> I think that this performance hit is potentially acceptable if we can 
> *also* transition to a standard mode-of-use in which not the entire 
> forest is getting rendered all the time, but instead pages are being 
> displayed on-demand through a local server that the Forester binary can 
> throw up. (And nobody cares if it takes 10s to render once in a while 
> prior to uploading to their web site, etc.). But what do you think?

Yes!! I think it is only natural to move in this direction, not only for
performance reasons. I have a lot of pieces that are required for this
scattered throughout some branches. Once you are ready to move on this,
I will make sure you get the code. This will also allow us to send
diagnostics to the client, and other cool stuff.

On 18 Oct 2024, at 19:51, Kento Okura wrote:

>> I think that this performance hit is potentially acceptable if we can
>> *also* transition to a standard mode-of-use in which not the entire
>> forest is getting rendered all the time, but instead pages are being
>> displayed on-demand through a local server that the Forester binary can
>> throw up. (And nobody cares if it takes 10s to render once in a while
>> prior to uploading to their web site, etc.). But what do you think?
>
> Yes!! I think it is only natural to move in this direction, not only for
> performance reasons. I have a lot of pieces that are required for this
> scattered throughout some branches. Once you are ready to move on this,
> I will make sure you get the code. This will also allow us to send
> diagnostics to the client, and other cool stuff.

Great!! Realistically, I want to do this within the coming two weeks.

Best,
Jon