I'm not very knowledgeable is far as GUI toolkits go and I'm not sure
exactly where I can post this question--but I thought this list might be
one place to start.

I've only used electron as an end user, not as a programmer--it seems to
have gained traction as a popular web based GUI toolkit, but many
complain of the bloat generated by all the HTML/CSS/JavaScript overhead.

I was wondering about two things:

1. Has anyone created a realistic alternative to HTML/CSS--maybe a
binary DOM that has a reasonable API for manipulation? I supposed it'd
also need some sort of rendering engine

2. Has anyone created a Wayland-like protocol for #1 so that you may
create an app that communicates through a socket to manipulate the DOM
and receive events? Basically you could use any programming language you
like, and you would speak the binary protocol to create windows,
manipulate the DOM, handle events, etc. A "binary web view" could be a
separate executable that just connects back to a socket specified as an
argument to the program. This binary web view just renders the DOM and
handles events and would be reusable across applications. Then your
widget toolkit SDK speaks the binary format over the wire and is written
in your programming language of choice.


Does this even make sense? Thoughts?

> 1. Has anyone created a realistic alternative to HTML/CSS--maybe a
> binary DOM that has a reasonable API for manipulation? I supposed it'd
> also need some sort of rendering engine
No, and there's no real need for one. Existing GUI frameworks (none of which are
small enough, but they work well enough) already are very mutable from APIs;
such a DOM would be no improvement, in fact, the only reason DOM manipulation
is so needed in the Electron world is because HTML is *not* a UI language, and
it shows; it's a markup language unfit for program UIs.
In that regard, Qt is not terrible, it produces good GUIs, even though it acts
like a bit of a virus in any codebase, and sadly exposes a C++ API instead of a
C one.

> 2. Has anyone created a Wayland-like protocol for #1 so that you may
> create an app that communicates through a socket to manipulate the DOM
> and receive events? Basically you could use any programming language you
> like, and you would speak the binary protocol to create windows,
> manipulate the DOM, handle events, etc. A "binary web view" could be a
> separate executable that just connects back to a socket specified as an
> argument to the program. This binary web view just renders the DOM and
> handles events and would be reusable across applications. Then your
> widget toolkit SDK speaks the binary format over the wire and is written
> in your programming language of choice.
This sounds massively over engineered, just link a library, let it render your
bitmaps and blit them on the screen for you, and that's it, which is how it
worked for the last three or so decades.

-- 
Arsen Arsenović

Arsen Arsenović <arsen@aarsen.me> writes:

>> 1. Has anyone created a realistic alternative to HTML/CSS--maybe a
>> binary DOM that has a reasonable API for manipulation? I supposed it'd
>> also need some sort of rendering engine
> No, and there's no real need for one. Existing GUI frameworks (none of which are
> small enough, but they work well enough) already are very mutable from APIs;
> such a DOM would be no improvement, in fact, the only reason DOM manipulation
> is so needed in the Electron world is because HTML is *not* a UI language, and
> it shows; it's a markup language unfit for program UIs.
> In that regard, Qt is not terrible, it produces good GUIs, even though it acts
> like a bit of a virus in any codebase, and sadly exposes a C++ API instead of a
> C one.

That's one of the problems having a DOM and protocol for manipulating it
would aim to solve--allowing a polyglot friendly widget toolkit. Seems
like it could also be beneficial for licensing as well.


>> 2. Has anyone created a Wayland-like protocol for #1 so that you may
>> create an app that communicates through a socket to manipulate the DOM
>> and receive events? Basically you could use any programming language you
>> like, and you would speak the binary protocol to create windows,
>> manipulate the DOM, handle events, etc. A "binary web view" could be a
>> separate executable that just connects back to a socket specified as an
>> argument to the program. This binary web view just renders the DOM and
>> handles events and would be reusable across applications. Then your
>> widget toolkit SDK speaks the binary format over the wire and is written
>> in your programming language of choice.
> This sounds massively over engineered, just link a library, let it render your
> bitmaps and blit them on the screen for you, and that's it, which is how it
> worked for the last three or so decades.

ok

On 2021-02-23 06:14, Timmy Douglas wrote:
> I'm not very knowledgeable is far as GUI toolkits go and I'm not sure
> exactly where I can post this question--but I thought this list might 
> be
> one place to start.
> 
> I've only used electron as an end user, not as a programmer--it seems 
> to
> have gained traction as a popular web based GUI toolkit, but many
> complain of the bloat generated by all the HTML/CSS/JavaScript 
> overhead.
> 
> I was wondering about two things:
> 
> 1. Has anyone created a realistic alternative to HTML/CSS--maybe a
> binary DOM that has a reasonable API for manipulation? I supposed it'd
> also need some sort of rendering engine
> 

By DOM, you mean the HTML serialization or the tree-like structure that 
is in memory?

I think HTML/CSS even DOM API are nice, but it is a long way to get 
there.
And not a lot of people have the time or energy to dive into 
alternatives approaches.

GUI without HTML/CSS is more powerful than what the browser can do. I 
figured that while trying to build a text editor.

Amirouche <amirouche@hyper.dev> writes:

> On 2021-02-23 06:14, Timmy Douglas wrote:
>> 1. Has anyone created a realistic alternative to HTML/CSS--maybe a
>> binary DOM that has a reasonable API for manipulation? I supposed it'd
>> also need some sort of rendering engine
>> 
>
> By DOM, you mean the HTML serialization or the tree-like structure that 
> is in memory?

well it'd probably be both--you'd need some sort of serialization format
(HTML today) to materialize the one you actually operate on. But if you
were going to create a GUI toolkit around it, you probably wouldn't want
to spend time parsing HTML unless you made that choice to reuse other code.

> I think HTML/CSS even DOM API are nice, but it is a long way to get 
> there.
> And not a lot of people have the time or energy to dive into 
> alternatives approaches.
>
> GUI without HTML/CSS is more powerful than what the browser can do. I 
> figured that while trying to build a text editor.

If I was going to build a simple text editor, I think rendering the view
with HTML/a couple of absolute-positioned preformatted divs would be
easier than writing my own renderer on a canvas-like object, if that
makes sense.

On 2021-02-24 04:56, Timmy Douglas wrote:
> Amirouche <amirouche@hyper.dev> writes:
> 
>> On 2021-02-23 06:14, Timmy Douglas wrote:
>>> 1. Has anyone created a realistic alternative to HTML/CSS--maybe a
>>> binary DOM that has a reasonable API for manipulation? I supposed 
>>> it'd
>>> also need some sort of rendering engine
>>> 
>> 
>> By DOM, you mean the HTML serialization or the tree-like structure 
>> that
>> is in memory?
> 
> well it'd probably be both--you'd need some sort of serialization 
> format
> (HTML today) to materialize the one you actually operate on.

HTML in the case of webdev is useful to exchange the information about 
the DOM between the frontend
and backend. Where the DOM represent the information required to do the 
rendering.
In fact, that HTML serialzation is not necessary in Single Page 
Application that
exchange JSON with the backend, and build the DOM fully browser side. 
Even more so
HTML is useless for rendering purpose in a desktop application (except 
for debugging).

> But if you were going to create a GUI toolkit around it, you probably 
> wouldn't want
> to spend time parsing HTML unless you made that choice to reuse other 
> code.

Yes. Qt or GTK can re-use existing components / widgets without a text 
or binary serialization.

> 
>> I think HTML/CSS even DOM API are nice, but it is a long way to get
>> there.
>> And not a lot of people have the time or energy to dive into
>> alternatives approaches.
>> 
>> GUI without HTML/CSS is more powerful than what the browser can do. I
>> figured that while trying to build a text editor.
> 
> If I was going to build a simple text editor, I think rendering the 
> view
> with HTML/a couple of absolute-positioned preformatted divs would be
> easier than writing my own renderer on a canvas-like object, if that
> makes sense.


I was not sure it was the case on your side, but basically me and
most web developers have a skewed vision of what GUI dev really is.

At the bottom of the GUI stack there is a bytes array, such as 'x'
times 'y' times 24 bits for 8 bits per RGB components.

At the top level there is some kind of representation such as the DOM.
In between there is a mess... What the representation is and how to 
translate
that into bytes or bits (what I call the 'mess') is up for engineering.

Maybe the case of a text editor is very peculiar, but I think similar
problems arise with diagramming tools, visual programming, and prolly 
others.

In the case of the text editor, there is a very specific datastructure
to represent the text that is edited (immutable array of array, rope,
balanced binary tree, fingertree..). When one thinks in term of browser 
GUI
and putting aside the canvas, one necessarily need to serialize that 
datastructure
not into HTML text, but into DOM text nodes, and that is very costly,
because of back and forth conversion when you consider key events, 
selections etc...

DOM does not accept arbitrary data nodes or even custom text node 
following a specific
interface or protocol. Similarly, in the case of diagramming tool, one 
need
to convert back and forth (hence copy much data) between the best 
diagram data structure
and the "render nodes" that are svg. Whereas in a desktop GUI, you can 
use
the same data structure for doing updates on events and rendering.

I did not check, but last time I read about it Atom editor use HTML 
canvas.

tl;dr: browsers are a framework that need to do trade-offs,
        and force you to think in a very specific narrow way.

-- 
Amirouche ~ https://hyper.dev

> 1. Has anyone created a realistic alternative to HTML/CSS--maybe a
> binary DOM that has a reasonable API for manipulation? I supposed it'd
> also need some sort of rendering engine

As additional data point I'll mention an effort to make a standalone
executable wrapper around a web application by using Webkit instead of Electron:
https://tauri.studio/

I have tested it on a VueJS web application and it works (ish). It's
still the same HTML+CSS+JS mess you mention but squeezed in a more
compact executable.

IIUC, Electron should be available for Wayland from March, 2021:
https://github.com/microsoft/vscode/issues/109176#issuecomment-752134096