~adnano/gemini

33 11

Documents with mixed languages

Details
Message ID
<20211211120013.u26rzavy7o5bfkys@mail.wolfsden.cz>
DKIM signature
missing
Download raw message
Hello,

I'm considering implementing a gemini client over the Christmas, so I've
started to read the spec. And I think I've notice a missing thing in
text/gemini format. In particular regarding documents with content
composed from multiple languages. The specification provides this
example:

"text/gemini; lang=en,fr"
  Denotes a text/gemini document written in a mixture of English and
  French

However, that seems to be a document-wide settings. If I wanted a
document to have a mix of Japanese and Chinese, I believe it would be:

"text/gemini; lang=ja,zh"

However, that in itself is not very useful. What I think is missing is
an ability to set language for a specific part of the document itself.
I'm not sure if the spec is permanently frozen or can still be extended,
but I would like to propose adding a language toggle command to
text/gemini.

Not sure about specific syntax, basically anything would work, ideally
something that would be not very disrupting if just render by clients
that do not know about it. So for example:

    Here are some examples in Japanese and Chinese:

    !lang=ja
    ??????

    !lang=zh
    ???

    !lang=en
    And back to the original language.

This is just proposal, I don't really care if it is a switch (like mine
example) or some kind of block.

Now, why is this necessary. For Japanese and Chinese in particular,
Unicode is simply not enough. Same code point can be rendered very
differently depending on the language. So for proper rendering of the
text, you need to know the language. If the document has just one, there
is currently available way to do so (document language), but for
mixture, it is simply not possible. And I think that is a shame.



Thanks for considering.

W.

PS: I'm not subscribed, so please CC.

-- 
There are only two hard things in Computer Science:
cache invalidation, naming things and off-by-one errors.
Details
Message ID
<CAFTy05YftRMJB9RV2Q2W33gboA9vw0soCV9==na7=L6rQDLuJA@mail.gmail.com>
In-Reply-To
<20211211120013.u26rzavy7o5bfkys@mail.wolfsden.cz> (view parent)
DKIM signature
missing
Download raw message
On Sat, Dec 11, 2021 at 8:31 AM Wolf <wolf at wolfsden.cz> wrote:
>
> Hello,

Greetings!

> I'm considering implementing a gemini client over the Christmas, so I've
> started to read the spec. And I think I've notice a missing thing in
> text/gemini format.

Many things are missing from the text/gemini format, as it was
intended to be a simple markup format. Instead of "is this possible?",
think "does this extension meet the power-to-weight ratio?".

> What I think is missing is
> an ability to set language for a specific part of the document itself.

Do you have any examples of existing gemini pages in the wild that
suffer from this problem? Or examples of pages that you intend to
publish on gemini? I'm interested in the real world use-cases (in
gemini) and not contrived scenarios.

> I'm not sure if the spec is permanently frozen or can still be extended,
> but I would like to propose adding a language toggle command to
> text/gemini.

I can tell you there's approximately zero chance of this being added
to the official spec. Your best bet, if you're serious about this, is
to go ahead and implement your proposal in your client. If others find
it useful they will start using it too (see: how titan is being
adopted).

> Not sure about specific syntax, basically anything would work, ideally
> something that would be not very disrupting if just render by clients
> that do not know about it. So for example:
>
>     Here are some examples in Japanese and Chinese:
>
>     !lang=ja
>     ??????
>
>     !lang=zh
>     ???
>
>     !lang=en
>     And back to the original language.
>
> This is just proposal, I don't really care if it is a switch (like mine
> example) or some kind of block.

This doesn't allow for mixed languages inside of a single
line/paragraph though.

> Now, why is this necessary. For Japanese and Chinese in particular,
> Unicode is simply not enough. Same code point can be rendered very
> differently depending on the language. So for proper rendering of the
> text, you need to know the language. If the document has just one, there
> is currently available way to do so (document language), but for
> mixture, it is simply not possible. And I think that is a shame.

I'm curious how this actually works with text rendering engines. Do
they expose an API that allows you to specify the language? Which one
are you planning on using, and could you point me to the relevant
docs?

>
> Thanks for considering.
>
> W.
>
> PS: I'm not subscribed, so please CC.
>
> --
> There are only two hard things in Computer Science:
> cache invalidation, naming things and off-by-one errors.
Stephane Bortzmeyer <stephane@sources.org>
Details
Message ID
<YbXShFOdt3HL69Yk@sources.org>
In-Reply-To
<CAFTy05YftRMJB9RV2Q2W33gboA9vw0soCV9==na7=L6rQDLuJA@mail.gmail.com> (view parent)
DKIM signature
missing
Download raw message
On Sat, Dec 11, 2021 at 01:06:25PM -0500,
 Michael Lazar <lazar.michael22 at gmail.com> wrote 
 a message of 75 lines which said:

> Your best bet, if you're serious about this, is to go ahead and
> implement your proposal in your client. If others find it useful
> they will start using it too

I'm not sure it is something to recommend since it can leads to "de
facto" standards and to "best viewed with client XYZ > 7" which were
one of the reasons we ran away from the Web.

> This doesn't allow for mixed languages inside of a single
> line/paragraph though.

Unicode has a solution, but its use is discouraged
<http://unicode.org/faq/languagetagging.html>. "Most other users who
need to tag text with the language identity should be using standard
markup mechanisms, such as those provided by HTML, XML, or other rich
text mechanisms."

Gemtext cannot do everything (and I tend to regard this as a feature,
not a bug).
Details
Message ID
<CAFTy05ZDsCiDaXHY5gc8m4+w0ch7QW-e11K3879qnQWDY6TnCA@mail.gmail.com>
In-Reply-To
<YbXShFOdt3HL69Yk@sources.org> (view parent)
DKIM signature
missing
Download raw message
On Sun, Dec 12, 2021 at 5:48 AM Stephane Bortzmeyer
<stephane at sources.org> wrote:
>
> On Sat, Dec 11, 2021 at 01:06:25PM -0500,
>  Michael Lazar <lazar.michael22 at gmail.com> wrote
>  a message of 75 lines which said:
>
> > Your best bet, if you're serious about this, is to go ahead and
> > implement your proposal in your client. If others find it useful
> > they will start using it too
>
> I'm not sure it is something to recommend since it can leads to "de
> facto" standards and to "best viewed with client XYZ > 7" which were
> one of the reasons we ran away from the Web.

Speak for yourself. De-facto standards are social proof that a subset
of the community actually wants and will use a feature. Which is much
more convincing to me than a loud minority arguing for (or against)
something based on principle alone.

> > This doesn't allow for mixed languages inside of a single
> > line/paragraph though.
>
> Unicode has a solution, but its use is discouraged
> <http://unicode.org/faq/languagetagging.html>. "Most other users who
> need to tag text with the language identity should be using standard
> markup mechanisms, such as those provided by HTML, XML, or other rich
> text mechanisms."

This is super interesting! I wonder what "deprecated" means for
unicode, surely they wouldn't release a backwards incompatible
version.
Their implementation guidelines systematically refute the argument for
language tags.

Requirements for Language Tagging

The requirement for language information embedded in plain text data
is often overstated. Many commonplace operations such as collation
seldom require this extra information. In collation, for example,
foreign language text is generally collated as if it were not in a
foreign language. (See Unicode Technical Standard #10, ?Unicode
Collation Algorithm,? for more information.) For example, an index in
an English book would not sort the Slovak word ?chlieb? after ?czar,?
where it would be collated in Slovak, nor would an English atlas put
the Swedish city of ?rebro after Zanzibar, where it would appear in
Swedish.

Text to speech is also an area where the case for embedded language
information is overstated. Although language information may be useful
in performing text-to-speech operations, modern software for doing
acceptable text-to-speech must be so sophisticated in performing
grammatical analysis of text that the extra work in determining the
language is not significant in practice.

Language information can be useful in certain operations, such as
spell-checking or hyphenating a mixed-language document. It is also
useful in choosing the default font for a run of unstyled text; for
example, the ellipsis character may have a very different appearance
in Japanese fonts than in European fonts. Modern font and layout
technologies produce different results based on language information.
For example, the angle of the acute accent may be different for French
and Polish.

Language Tags and Han Unification

A common misunderstanding about Unicode Han unification is the
mistaken belief that Han characters cannot be rendered properly
without language information. This idea might lead an implementer to
conclude that language information must always be added to plain text
using the tags. However, this implication is incorrect. The goal and
methods of Han unification were to ensure that the text remained
legible. Although font, size, width, and other format specifications
need to be added to produce precisely the same appearance on the
source and target machines, plain text remains legible in the absence
of these specifications. There should never be any confusion in
Unicode, because the distinctions between the unified characters are
all within the range of stylistic variations that exist in each
country. No unification in Unicode should make it impossible for a
reader to identify a character if it appears in a different font.
Where precise font information is important, it is best conveyed in a
rich text format.
Details
Message ID
<e5d7e8f3-aea3-4a86-b4ff-21847424fc1f@torresjrjr.com>
In-Reply-To
<CAFTy05ZDsCiDaXHY5gc8m4+w0ch7QW-e11K3879qnQWDY6TnCA@mail.gmail.com> (view parent)
DKIM signature
missing
Download raw message
12 Dec 2021 14:47:22 Michael Lazar <lazar.michael22 at gmail.com>:

> On Sun, Dec 12, 2021 at 5:48 AM Stephane Bortzmeyer
> <stephane at sources.org> wrote:
>>
>> On Sat, Dec 11, 2021 at 01:06:25PM -0500,
>> Michael Lazar <lazar.michael22 at gmail.com> wrote
>> a message of 75 lines which said:
>>
>>> Your best bet, if you're serious about this, is to go ahead and
>>> implement your proposal in your client. If others find it useful
>>> they will start using it too
>>
>> I'm not sure it is something to recommend since it can leads to "de
>> facto" standards and to "best viewed with client XYZ > 7" which were
>> one of the reasons we ran away from the Web.
>
> Speak for yourself. De-facto standards are social proof that a subset
> of the community actually wants and will use a feature. Which is much
> more convincing to me than a loud minority arguing for (or against)
> something based on principle alone.

Gemini is a principled project.
"social proof" is not proof, unqualified.

Gemini users vs everyone else is a minority.
Numbers games are not effective without substance.
Details
Message ID
<20211213020808.a1aea53ca128500cf2ff3c5f@gmail.com>
In-Reply-To
<CAFTy05ZDsCiDaXHY5gc8m4+w0ch7QW-e11K3879qnQWDY6TnCA@mail.gmail.com> (view parent)
DKIM signature
missing
Download raw message
On Sun, 12 Dec 2021 09:46:15 -0500
Michael Lazar <lazar.michael22 at gmail.com> wrote:

> Speak for yourself. De-facto standards are social proof that a subset
> of the community actually wants and will use a feature. Which is much
> more convincing to me than a loud minority arguing for (or against)
> something based on principle alone.

A de facto standard, as far as it's actually implemented, is at best
evidence that current implementors are willing to to implement it. This
in itself represents a minority.

The point of a well-defined standard is that while creating that
definition might be a social process, learning and implementing it
doesn't have to be. The ideal outcome of a formal, canonical standard is
that an implementor should not have to go spelunking and survey existing
implementation (or indeed this mailing list) to learn what feature set
a client or a server is currently expected to implement not to be
effectively broken.

For example, I can go to your capsule and discover your streaming chat.
This might have worried me based on just a basic description of a
living standard, but because there is a formal standard according to
which a client handles a response only after the server closes the
connection of the transaction, I can instead treat it as a weird
mutation of the protocol, and generally not worry about the whims of
a few implementors that ignore the spec in various ways. In this way, I
avoid the tyranny of a multitude of "loud minorities" that would have
been much more disruptive without a canonical specification of the
protocol.

-- 
Philip
Omar Polo <op@omarpolo.com>
Details
Message ID
<877dc87rmt.fsf@omarpolo.com>
In-Reply-To
<20211213020808.a1aea53ca128500cf2ff3c5f@gmail.com> (view parent)
DKIM signature
missing
Download raw message
Philip Linde <linde.philip at gmail.com> writes:

> [...]

I know I'm going out of topic, but I've seen this argument other times
and I honestly don't understand it.

> For example, I can go to your capsule and discover your streaming chat.
> This might have worried me based on just a basic description of a
> living standard, but because there is a formal standard according to
> which a client handles a response only after the server closes the
> connection of the transaction [...]

I'd be glad if you could point me to where in the specification is
written that a client *MUST* handle the response only after the server
has close the connection.

I can't find any evidence of that.

(No, the outline of the transaction in 1.1 is not a formal description
of what each party should do, it's just an outline.)

On the contrary, there's a post by solderpunk himself where he describes
exactly that:

gemini://gemini.circumlunar.space/users/solderpunk/gemlog/a-vision-for-gemini-applications.gmi
Details
Message ID
<QROqC4EalCU4FrsWcT47KVSs8ObpdY8cDeFszGlPg6E2tm93bkzSoR9NF56EmxzZ83VMrQOSX7_FcrqNtboRjQ5oTwVD3Ulx-Q3qNDX9eyY=@protonmail.com>
In-Reply-To
<877dc87rmt.fsf@omarpolo.com> (view parent)
DKIM signature
missing
Download raw message
Yeah, some people don't know Gemini History apparently, lol.

People need to stop imposing their views on the standard as if it was part of the original design of the protocol when it wasn't. Solderpunk could change things at any time, of course, but that doesn't change the original designs. So... No, as far as I'm aware, Gemini was never about read-only. It was about balancing the power-to-weight ratio. No, Gemini's text input was never exclusively for search - there's an article written by Solderpunk that details why he included input labels. No, Gemini was never anti-applications (at least since the idea of Client Certificates). Solderpunk wrote an article showing that applications were at least considered as a potential for the protocol. Also, no, having (text, and others) streaming and browsers that interact with streams in Gemini is not against the intended spec. In fact, allowing for Text Streaming was a decision talked about between Solderpunk, Tomasino (iirc), Mozz, and a few other people. Solderpunk references this feature in one of his recent emails here on the mailing list (regarding how close_notify doesn't affect streaming via Gemini).

Now, about what the spec states. If you read the spec too literally and linearly, then I can kinda see how people might think the spec says requests must be handled after close_notify, from:
> C: Opens connection
> S: Accepts connection
> C/S: Complete TLS handshake (see section 4)
> C: Validates server certificate (see 4.2)
> C: Sends request (one CRLF terminated line) (see section 2)
> S: Sends response header (one CRLF terminated line), closes connection
 under non-success conditions (see 3.1 and 3.2)
> S: Sends response body (text or binary data) (see 3.3)
> S: Closes connection (including TLS close_notify, see section 4)
> C: Handles response (see 3.4)

I firmly believe that this is an undesired overly-linear interpretation, and that perhaps Solderpunk might want to consider changing this if necessary, and if his views on streaming haven't changed. Remember, the spec isn't actually final yet.

As for the links:
On Inputs and Client Certificates: gopher://zaibatsu.circumlunar.space:70/0/~solderpunk/gemini/inputs-and-client-certs.txt
On Streaming (Tomasino): gemini://rawtext.club/~sloum/geminilist/001696.gmi
On Applications and Streaming (Solderpunk): gemini://gemini.circumlunar.space/users/solderpunk/gemlog/a-vision-for-gemini-applications.gmi
Mozz's Response to Solderpunk on Streaming: gemini://mozz.us/journal/2020-06-21.gmi

Links via Gemini:
On Inputs and Client Certificates: gemini://zaibatsu.circumlunar.space/~solderpunk/gemini/inputs-and-client-certs.txt
On Streaming (Tomasino): [Only Available Via Gopher, afaik]
On Applications and Streaming (Solderpunk): gemini://gemini.circumlunar.space/users/solderpunk/gemlog/a-vision-for-gemini-applications.gmi
Mozz's Response to Solderpunk on Streaming: [Only Available Via Gopher, afaik]

Christian Seibold

Is streaming a normative protocol expectation? (was Re: Documents with mixed languages)

Details
Message ID
<da33a30c-b0e3-dbf0-e684-282ab9b64a28@marmaladefoo.com>
In-Reply-To
<QROqC4EalCU4FrsWcT47KVSs8ObpdY8cDeFszGlPg6E2tm93bkzSoR9NF56EmxzZ83VMrQOSX7_FcrqNtboRjQ5oTwVD3Ulx-Q3qNDX9eyY=@protonmail.com> (view parent)
DKIM signature
missing
Download raw message
Hello

(we are straying off the thread a bit so I've tweaked the subject)

On 13-Dec-2021 11:21, Krixano wrote:

> Now, about what the spec states. If you read the spec too literally and linearly, then I can kinda see how people might think the spec says requests must be handled after close_notify, from:
>> C: Opens connection
>> S: Accepts connection
>> C/S: Complete TLS handshake (see section 4)
>> C: Validates server certificate (see 4.2)
>> C: Sends request (one CRLF terminated line) (see section 2)
>> S: Sends response header (one CRLF terminated line), closes connection
>   under non-success conditions (see 3.1 and 3.2)
>> S: Sends response body (text or binary data) (see 3.3)
>> S: Closes connection (including TLS close_notify, see section 4)
>> C: Handles response (see 3.4)
> 
> I firmly believe that this is an undesired overly-linear interpretation, and that perhaps Solderpunk might want to consider changing this if necessary, and if his views on streaming haven't changed. Remember, the spec isn't actually final yet.

Yes there would be an ambiguity in the spec if streaming clients are 
expected or normative. Personally I think it is intentional that the 
spec aims to keep the complexity of implementation low, as that is in 
keeping with the rest of Gemini's design approach. Implementing a 
streaming client is more complex, and there are more cases that would 
need to be specified to resolve protocol ambiguity. For example how does 
a client tell the difference between a poorly behaved server that is 
just dripping a few bytes out second by second and one that is a 
streaming service? I can't see how it can, particularly given that there 
are no content-length headers in Gemini.

The Gemini FAQ also hints towards this simplicity and says Gemini is not 
suited to large downloads

   "2.6 Does Gemini have any shortcomings of it's own?
  Naturally!
  Gemini has no support for caching, compression, or resumption of
  interrupted downloads. As such, it's not very well suited to distributing
  large files, for values of "large" which depend upon the speed and 
reliability
   of your network connection."

At present the spec says the client waits for the content then handles 
the response.

So this is what the spec says at present, it doesn't say that clients 
may expect that server streams are indefinite in length.

We have previously discussed this and there was some useful thoughts 
written up by James Tomasino

https://lists.orbitalfox.eu/archives/gemini/2020/002404.html

My view is that if you want to stream content, there are better 
protocols available (e.g. HTTP and others).

Best wishes

  - Luke
Details
Message ID
<b56693ab-77e1-f268-7915-f5ccf6756355@marmaladefoo.com>
In-Reply-To
<20211211120013.u26rzavy7o5bfkys@mail.wolfsden.cz> (view parent)
DKIM signature
missing
Download raw message
Hello

On 11-Dec-2021 12:00, Wolf wrote:

> And I think I've notice a missing thing in
> text/gemini format. In particular regarding documents with content
> composed from multiple languages. The specification provides this
> example:
> 
> "text/gemini; lang=en,fr"
>    Denotes a text/gemini document written in a mixture of English and
>    French
> 
> However, that seems to be a document-wide settings. If I wanted a
> document to have a mix of Japanese and Chinese, I believe it would be:
> 
> "text/gemini; lang=ja,zh"
> 
> However, that in itself is not very useful. What I think is missing is
> an ability to set language for a specific part of the document itself.
> I'm not sure if the spec is permanently frozen or can still be extended,
> but I would like to propose adding a language toggle command to
> text/gemini.

Yes there are lots of modes of communication that Gemtext is not perfect 
for. Tabular data, inline formatting, etc.

You are not limited to only serving gemtext over gemini, you can serve 
other media types such as HTML, PDF, ODF etc, which may be better suited 
to this particular desire than gemtext which is - as plain to see and by 
design - quite basic. Those might be better for your use case. For 
non-gemtext pages, clients will usually provide an option to save 
locally or to open other media types in the associated application 
allowing the content to be viewed.

Regards

  - Luke
Details
Message ID
<A3CA6789-6FE8-40B4-87EE-7224ED736E09@ngalt.com>
In-Reply-To
<b56693ab-77e1-f268-7915-f5ccf6756355@marmaladefoo.com> (view parent)
DKIM signature
missing
Download raw message

> On Dec 13, 2021, at 9:28 AM, Luke Emmet <luke at marmaladefoo.com> wrote:
> 
> Hello
> 
> On 11-Dec-2021 12:00, Wolf wrote:
> 
>> And I think I've notice a missing thing in
>> text/gemini format. In particular regarding documents with content
>> composed from multiple languages. The specification provides this
>> example:
>> "text/gemini; lang=en,fr"
>>   Denotes a text/gemini document written in a mixture of English and
>>   French
>> However, that seems to be a document-wide settings. If I wanted a
>> document to have a mix of Japanese and Chinese, I believe it would be:
>> "text/gemini; lang=ja,zh"
>> However, that in itself is not very useful. What I think is missing is
>> an ability to set language for a specific part of the document itself.
>> I'm not sure if the spec is permanently frozen or can still be extended,
>> but I would like to propose adding a language toggle command to
>> text/gemini.
> 
> Yes there are lots of modes of communication that Gemtext is not perfect for. Tabular data, inline formatting, etc.
> 
> You are not limited to only serving gemtext over gemini, you can serve other media types such as HTML, PDF, ODF etc, which may be better suited to this particular desire than gemtext which is - as plain to see and by design - quite basic. Those might be better for your use case. For non-gemtext pages, clients will usually provide an option to save locally or to open other media types in the associated application allowing the content to be viewed.
> 
> Regards
> 
> - Luke

On the other hand, if you foresee yourself frequently adding tables or complicated formatting or mixed-language pages, you and your readers will probably be much better served if you set up a website with some sensible, lightweight CSS style rules and almost certainly no JavaScript.

I have a couple things I?ve written that?re languishing in the hopper, but I?ll probably never publish them on Gemini because they have too many semi-complicated tables in them.

I?m a big fan of Gemini?s current feature set and its lack of extensibility, but my hunch is that if you?ve never been bitten by the lack of any features that HTTP+HTML has, you haven?t used Gemini much.

Streaming responses

Details
Message ID
<20211213200037.a6b1f288f83ebb03099f8e6b@gmail.com>
In-Reply-To
<877dc87rmt.fsf@omarpolo.com> (view parent)
DKIM signature
missing
Download raw message
On Mon, 13 Dec 2021 10:48:33 +0100
Omar Polo <op at omarpolo.com> wrote:

> (No, the outline of the transaction in 1.1 is not a formal description
> of what each party should do, it's just an outline.)

Says who? It's part of the specification. There is nothing in the
specification itself that qualifies it as an informal hint. Given that
it's the only part of the specification that describes a complete
transaction, I take it to be clearly specified that the client handles
a response after the server closes the connection. Meanwhile, there's
nothing in the specification that remotely indicates that a client
should render content while the server is streaming it.

> On the contrary, there's a post by solderpunk himself where he describes
> exactly that:
> 
> gemini://gemini.circumlunar.space/users/solderpunk/gemlog/a-vision-for-gemini-applications.gmi

This perfectly exemplifies my point that it's easier to read a
specification than to go spelunking through gemlog posts to learn what
set of features I am currently expected to implement as a Gemini
implementor. What's extra funny is that you dismiss my references to
the specification as an informal part of it (with absolutely no basis)
and at the same time you refer to a gemlog where that pretty much only
documents that Solderpunk thought the idea was cool at some point.

-- 
Philip

Is streaming a normative protocol expectation? (was Re: Documents with mixed languages)

Sean Conner <sean@conman.org>
Details
Message ID
<20211213191039.GD22651@brevard.conman.org>
In-Reply-To
<da33a30c-b0e3-dbf0-e684-282ab9b64a28@marmaladefoo.com> (view parent)
DKIM signature
missing
Download raw message
It was thus said that the Great Luke Emmet once stated:
> 
> Yes there would be an ambiguity in the spec if streaming clients are 
> expected or normative. Personally I think it is intentional that the 
> spec aims to keep the complexity of implementation low, as that is in 
> keeping with the rest of Gemini's design approach. Implementing a 
> streaming client is more complex, and there are more cases that would 
> need to be specified to resolve protocol ambiguity. For example how does 
> a client tell the difference between a poorly behaved server that is 
> just dripping a few bytes out second by second and one that is a 
> streaming service? I can't see how it can, particularly given that there 
> are no content-length headers in Gemini.

  If you receive a MIME type of "audio/*", "video/*" or "text/event-stream"
[1], then you probably would want to stream the data or expect to spend
some time downloading the entire content (or abort entirely if you get
"text/event-stream"). 

  And there's nothing in the spec that precludes a client from dealing with
data in a streaming way (even *if* it may be more complicated).

  -spc

[1]	Yup, it's a thing.

	https://html.spec.whatwg.org/multipage/server-sent-events.html

Streaming responses

Details
Message ID
<RYWkCVganC_glZ38u2-LVfe_PhclKhk-aChoE23jj2u-Hsgy1DSG0EX6P5XKr__29VR1wwtiHpflc6_WItICbHy4fvYsegi_yIHwZ4PQ_8w=@protonmail.com>
In-Reply-To
<20211213200037.a6b1f288f83ebb03099f8e6b@gmail.com> (view parent)
DKIM signature
missing
Download raw message
Philip Linde, It turns out you were already involved in this very discussion on the mailing list where Solderpunk had brought up the idea of Stream requests on July 14th, 2020 (subject title: "Re: Reopening: Stream status code"). He was against the idea of a request, but was for the idea of allowing streams. You're trying to make this seem more complicated than it really is when we have very simple handling of "streams" happening for a few clients out in the wild. The solution is that you can start displaying the content/data as it comes in (which is actually simple for gemtext, since it's line-based). In fact, the close_notify tells the client when a connection has *actually* finished. And then if you are not satisfied with that, then you can add a timeout, which servers **should be doing anyways even if they weren't doing "streaming"**.

Solderpunk has made it sufficiently clear that clients can handle responses before the close_notify (and if I'm wrong, it's important that he corrects me), and this is for several reasons, one being that it's a better user experience with a very low tradeoff. The primary example of "streaming" (which is literally just handling data *as it comes in*) was Text Streaming. Things like mbays' games at gemini://gemini.thegonz.net/sggs/)

Next, about the specification. Nowhere does it state in the specification that steps cannot overlap. Again, you're using an overly-linear interpretation of that outline of the requests. Remember, *the spec is not finished.* It is important, however, I'm certainly not against things being more explicit in the spec. In fact, I think they should be.

Christian Seibold

Sent with ProtonMail Secure Email.

??????? Original Message ???????

On Monday, December 13th, 2021 at 1:00 PM, Philip Linde <linde.philip at gmail.com> wrote:

> On Mon, 13 Dec 2021 10:48:33 +0100
>
> Omar Polo op at omarpolo.com wrote:
>
> > (No, the outline of the transaction in 1.1 is not a formal description
> >
> > of what each party should do, it's just an outline.)
>
> Says who? It's part of the specification. There is nothing in the
>
> specification itself that qualifies it as an informal hint. Given that
>
> it's the only part of the specification that describes a complete
>
> transaction, I take it to be clearly specified that the client handles
>
> a response after the server closes the connection. Meanwhile, there's
>
> nothing in the specification that remotely indicates that a client
>
> should render content while the server is streaming it.
>
> > On the contrary, there's a post by solderpunk himself where he describes
> >
> > exactly that:
> >
> > gemini://gemini.circumlunar.space/users/solderpunk/gemlog/a-vision-for-gemini-applications.gmi
>
> This perfectly exemplifies my point that it's easier to read a
>
> specification than to go spelunking through gemlog posts to learn what
>
> set of features I am currently expected to implement as a Gemini
>
> implementor. What's extra funny is that you dismiss my references to
>
> the specification as an informal part of it (with absolutely no basis)
>
> and at the same time you refer to a gemlog where that pretty much only
>
> documents that Solderpunk thought the idea was cool at some point.
>
> -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Philip

Streaming responses

Details
Message ID
<Uc8cltnf6-hBv8zqBqEfrRn5m7vd_wzXdzpPgn1DtC09f7Q5VBVbbf0er0GdoUBxzrFzzcf2H51rHrnYCLvyYhYL9aU1wqAc4M7O3Bk_tR0=@protonmail.com>
In-Reply-To
<20211213200037.a6b1f288f83ebb03099f8e6b@gmail.com> (view parent)
DKIM signature
missing
Download raw message
In any case, Solderpunk is the one who ultimately decides what happens with this. I feel that he has already decided to allow this, but I guess some people want additional confirmation. And in any case, he could always change his mind.

Christian Seibold
Details
Message ID
<20211213212225.d6b473389fee1ad8ba426506@gmail.com>
In-Reply-To
<QROqC4EalCU4FrsWcT47KVSs8ObpdY8cDeFszGlPg6E2tm93bkzSoR9NF56EmxzZ83VMrQOSX7_FcrqNtboRjQ5oTwVD3Ulx-Q3qNDX9eyY=@protonmail.com> (view parent)
DKIM signature
missing
Download raw message
On Mon, 13 Dec 2021 11:21:44 +0000
Krixano <krixano at protonmail.com> wrote:

> Yeah, some people don't know Gemini History apparently, lol.

I do (being an avid phlog reader since long before Gemini) and I'm well
aware of the apparent controversy around streaming responses, but so
what if I didn't? Should implementors necessarily familiarize
themselves with the entire history of the protocol in order to be able
to implement it? Or fail and get laughed at because they "don't know
Gemini History apparently, lol"?

I imagine that if I actually was clueless about the history of Gemini
and came to this mailing list only to get laughed at because I didn't
read a bunch of phlogs where people vaguely promote an idea that isn't
allowed by the specification, I would think it patronizing, rude and
condescending, and would quickly leave what I'd assume was a bunch of
antisocial nerds to guard their lawn in peace.

> People need to stop imposing their views on the standard as if it 
> part of the original design of the protocol when it wasn't.

Just to be clear, I'm reading the standard and interpreting it to my
best ability. This, in my view, exactly entails not reading things
into it, by reading it as literally as possible. This isn't just out
of an obstinate desire to cause a ruckus, but because that's how people
will read the specification when they don't have our contextual
baggage. Apparently in this case we've learned that the specification
doesn't reflect behavior which you seem to think is standard.
Regardless of whether that's a mistake in the spec or a
misinterpretation on your end, we can learn from that literal reading
what about the spec can be improved.

If from a description of a transaction as something that
ends with the server closing the connection followed by the client
handling the response I interpret it to mean exactly that, I don't
think that I am particularly imposing my views on the standard.

> Solderpunk could change things at any time, of course, but that doesn't
> change the original designs. So... No, as far as I'm aware, Gemini was
> never about read-only. It was about balancing the power-to-weight
> ratio. No, Gemini's text input was never exclusively for search -
> there's an article written by Solderpunk that details why he included
> input labels. No, Gemini was never anti-applications (at least since
> the idea of Client Certificates). Solderpunk wrote an article showing
> that applications were at least considered as a potential for the
> protocol.

Completely beside the point. My criticism here has nothing to do with
the underlying principles of the design of the protocol, or the
intended use of certain features. My criticism is based on a reading of
the specification, and server behavior which apparently depends on
clients not following the specification according to this reading.

> Also, no, having (text, and others) streaming and browsers
> that interact with streams in Gemini is not against the intended spec.

So you say, but perhaps you have just imposed your own views on the
standard.

> In fact, allowing for Text Streaming was a decision talked about
> between Solderpunk, Tomasino (iirc), Mozz, and a few other people.
> Solderpunk references this feature in one of his recent emails here on
> the mailing list (regarding how close_notify doesn't affect streaming
> via Gemini).

It was talked about between some people, and Solderpunk refers to it in
some recent email, yada yada. What use is this to someone reading the
specification? Is there a canonical amendment to the specification only
existing in the form of various phlog entries, gemlogs and emails?

> Now, about what the spec states. If you read the spec too literally
> and linearly, then I can kinda see how people might think the spec says
> requests must be handled after close_notify, from:

What is the right degree of literality and linearity I should read the
specification with? Is it OK for example for the server to send the
response header after sending the response body? May the server
respond before CR+LF in the request line? What should I replace the
literal part of my reading with? My own views? The content of some
even relatively obscure phlog entry?

The point strikes me as rather absurd in the context of what you've
just said. On one hand you're asking people not to read things into the
standard. On the other hand, apparently my reading of the spec is too
literal.

> I firmly believe that this is an undesired overly-linear
> interpretation, and that perhaps Solderpunk might want to consider
> changing this if necessary, and if his views on streaming haven't
> changed. Remember, the spec isn't actually final yet.

That may very well be true, in which case I agree that the
specification should be updated. But it clearly isn't the behavior the
specification currently specifies.

> 
> As for the links:
> On Inputs and Client Certificates: gopher://zaibatsu.circumlunar.space:70/0/~solderpunk/gemini/inputs-and-client-certs.txt
> On Streaming (Tomasino): gemini://rawtext.club/~sloum/geminilist/001696.gmi
> On Applications and Streaming (Solderpunk): gemini://gemini.circumlunar.space/users/solderpunk/gemlog/a-vision-for-gemini-applications.gmi
> Mozz's Response to Solderpunk on Streaming: gemini://mozz.us/journal/2020-06-21.gmi
> 
> Links via Gemini:
> On Inputs and Client Certificates: gemini://zaibatsu.circumlunar.space/~solderpunk/gemini/inputs-and-client-certs.txt
> On Streaming (Tomasino): [Only Available Via Gopher, afaik]
> On Applications and Streaming (Solderpunk): gemini://gemini.circumlunar.space/users/solderpunk/gemlog/a-vision-for-gemini-applications.gmi
> Mozz's Response to Solderpunk on Streaming: [Only Available Via Gopher, afaik]

Hey, don't forget that email you vaguely referred to that Solderpunk
sent to the list recently! Jokes aside, this is another great example
of why I think de facto standards are a bad idea. Clearly, it would be
advantageous for an implementor to be able to read from the
specification that it was indeed intended for clients to handle
streaming responses. Then you don't need what increasingly will
resemble case law to determine whether your implementation is correct.

-- 
Philip

Streaming responses

Details
Message ID
<mnAlmid85YDm2zvAcFbBVYjNT3Kb9AJrh8oeAxD86X7aQ7keZ4TgY8WOVt_T_zsN8mQjfpVV1Mp-5RkjSqoEouDCAb0G1AX1fwXU6_TY22s=@protonmail.com>
In-Reply-To
<RYWkCVganC_glZ38u2-LVfe_PhclKhk-aChoE23jj2u-Hsgy1DSG0EX6P5XKr__29VR1wwtiHpflc6_WItICbHy4fvYsegi_yIHwZ4PQ_8w=@protonmail.com> (view parent)
DKIM signature
missing
Download raw message
> He was against the idea of a request, but was for the idea of allowing streams

To make sure I'm clear, I meant to type "He was against the idea of a request status code, but was for the idea of allowing streams"

Christian Seibold

??????? Original Message ???????

On Monday, December 13th, 2021 at 2:12 PM, Krixano <krixano at protonmail.com> wrote:

> Philip Linde, It turns out you were already involved in this very discussion on the mailing list where Solderpunk had brought up the idea of Stream requests on July 14th, 2020 (subject title: "Re: Reopening: Stream status code"). He was against the idea of a request, but was for the idea of allowing streams. You're trying to make this seem more complicated than it really is when we have very simple handling of "streams" happening for a few clients out in the wild. The solution is that you can start displaying the content/data as it comes in (which is actually simple for gemtext, since it's line-based). In fact, the close_notify tells the client when a connection has actually finished. And then if you are not satisfied with that, then you can add a timeout, which servers should be doing anyways even if they weren't doing "streaming".
>
> Solderpunk has made it sufficiently clear that clients can handle responses before the close_notify (and if I'm wrong, it's important that he corrects me), and this is for several reasons, one being that it's a better user experience with a very low tradeoff. The primary example of "streaming" (which is literally just handling data as it comes in) was Text Streaming. Things like mbays' games at gemini://gemini.thegonz.net/sggs/)
>
> Next, about the specification. Nowhere does it state in the specification that steps cannot overlap. Again, you're using an overly-linear interpretation of that outline of the requests. Remember, the spec is not finished. It is important, however, I'm certainly not against things being more explicit in the spec. In fact, I think they should be.
>
> Christian Seibold
Details
Message ID
<1lXHv8IMv1ssYl-z26qAuwYto5dfWQW_wX9Gyq64F5Z3mdfT9XH8biXFyY05EP5a0FA9SVsnvTP2DNQ0g3hYZ05mgY0AibHYzbpr09eftWk=@protonmail.com>
In-Reply-To
<20211213212225.d6b473389fee1ad8ba426506@gmail.com> (view parent)
DKIM signature
missing
Download raw message
The INTENDED SPEC is not the same as the ACTUAL SPEC! The intended spec matters because THE SPEC IS NOT FINAL.

Christian Seibold

Sent with ProtonMail Secure Email.

??????? Original Message ???????

On Monday, December 13th, 2021 at 2:22 PM, Philip Linde <linde.philip at gmail.com> wrote:

> On Mon, 13 Dec 2021 11:21:44 +0000
>
> Krixano krixano at protonmail.com wrote:
>
> > Yeah, some people don't know Gemini History apparently, lol.
>
> I do (being an avid phlog reader since long before Gemini) and I'm well
>
> aware of the apparent controversy around streaming responses, but so
>
> what if I didn't? Should implementors necessarily familiarize
>
> themselves with the entire history of the protocol in order to be able
>
> to implement it? Or fail and get laughed at because they "don't know
>
> Gemini History apparently, lol"?
>
> I imagine that if I actually was clueless about the history of Gemini
>
> and came to this mailing list only to get laughed at because I didn't
>
> read a bunch of phlogs where people vaguely promote an idea that isn't
>
> allowed by the specification, I would think it patronizing, rude and
>
> condescending, and would quickly leave what I'd assume was a bunch of
>
> antisocial nerds to guard their lawn in peace.
>
> > People need to stop imposing their views on the standard as if it
> >
> > part of the original design of the protocol when it wasn't.
>
> Just to be clear, I'm reading the standard and interpreting it to my
>
> best ability. This, in my view, exactly entails not reading things
>
> into it, by reading it as literally as possible. This isn't just out
>
> of an obstinate desire to cause a ruckus, but because that's how people
>
> will read the specification when they don't have our contextual
>
> baggage. Apparently in this case we've learned that the specification
>
> doesn't reflect behavior which you seem to think is standard.
>
> Regardless of whether that's a mistake in the spec or a
>
> misinterpretation on your end, we can learn from that literal reading
>
> what about the spec can be improved.
>
> If from a description of a transaction as something that
>
> ends with the server closing the connection followed by the client
>
> handling the response I interpret it to mean exactly that, I don't
>
> think that I am particularly imposing my views on the standard.
>
> > Solderpunk could change things at any time, of course, but that doesn't
> >
> > change the original designs. So... No, as far as I'm aware, Gemini was
> >
> > never about read-only. It was about balancing the power-to-weight
> >
> > ratio. No, Gemini's text input was never exclusively for search -
> >
> > there's an article written by Solderpunk that details why he included
> >
> > input labels. No, Gemini was never anti-applications (at least since
> >
> > the idea of Client Certificates). Solderpunk wrote an article showing
> >
> > that applications were at least considered as a potential for the
> >
> > protocol.
>
> Completely beside the point. My criticism here has nothing to do with
>
> the underlying principles of the design of the protocol, or the
>
> intended use of certain features. My criticism is based on a reading of
>
> the specification, and server behavior which apparently depends on
>
> clients not following the specification according to this reading.
>
> > Also, no, having (text, and others) streaming and browsers
> >
> > that interact with streams in Gemini is not against the intended spec.
>
> So you say, but perhaps you have just imposed your own views on the
>
> standard.
>
> > In fact, allowing for Text Streaming was a decision talked about
> >
> > between Solderpunk, Tomasino (iirc), Mozz, and a few other people.
> >
> > Solderpunk references this feature in one of his recent emails here on
> >
> > the mailing list (regarding how close_notify doesn't affect streaming
> >
> > via Gemini).
>
> It was talked about between some people, and Solderpunk refers to it in
>
> some recent email, yada yada. What use is this to someone reading the
>
> specification? Is there a canonical amendment to the specification only
>
> existing in the form of various phlog entries, gemlogs and emails?
>
> > Now, about what the spec states. If you read the spec too literally
> >
> > and linearly, then I can kinda see how people might think the spec says
> >
> > requests must be handled after close_notify, from:
>
> What is the right degree of literality and linearity I should read the
>
> specification with? Is it OK for example for the server to send the
>
> response header after sending the response body? May the server
>
> respond before CR+LF in the request line? What should I replace the
>
> literal part of my reading with? My own views? The content of some
>
> even relatively obscure phlog entry?
>
> The point strikes me as rather absurd in the context of what you've
>
> just said. On one hand you're asking people not to read things into the
>
> standard. On the other hand, apparently my reading of the spec is too
>
> literal.
>
> > I firmly believe that this is an undesired overly-linear
> >
> > interpretation, and that perhaps Solderpunk might want to consider
> >
> > changing this if necessary, and if his views on streaming haven't
> >
> > changed. Remember, the spec isn't actually final yet.
>
> That may very well be true, in which case I agree that the
>
> specification should be updated. But it clearly isn't the behavior the
>
> specification currently specifies.
>
> > As for the links:
> >
> > On Inputs and Client Certificates: gopher://zaibatsu.circumlunar.space:70/0/~solderpunk/gemini/inputs-and-client-certs.txt
> >
> > On Streaming (Tomasino): gemini://rawtext.club/~sloum/geminilist/001696.gmi
> >
> > On Applications and Streaming (Solderpunk): gemini://gemini.circumlunar.space/users/solderpunk/gemlog/a-vision-for-gemini-applications.gmi
> >
> > Mozz's Response to Solderpunk on Streaming: gemini://mozz.us/journal/2020-06-21.gmi
> >
> > Links via Gemini:
> >
> > On Inputs and Client Certificates: gemini://zaibatsu.circumlunar.space/~solderpunk/gemini/inputs-and-client-certs.txt
> >
> > On Streaming (Tomasino): [Only Available Via Gopher, afaik]
> >
> > On Applications and Streaming (Solderpunk): gemini://gemini.circumlunar.space/users/solderpunk/gemlog/a-vision-for-gemini-applications.gmi
> >
> > Mozz's Response to Solderpunk on Streaming: [Only Available Via Gopher, afaik]
>
> Hey, don't forget that email you vaguely referred to that Solderpunk
>
> sent to the list recently! Jokes aside, this is another great example
>
> of why I think de facto standards are a bad idea. Clearly, it would be
>
> advantageous for an implementor to be able to read from the
>
> specification that it was indeed intended for clients to handle
>
> streaming responses. Then you don't need what increasingly will
>
> resemble case law to determine whether your implementation is correct.
>
> ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Philip

Is streaming a normative protocol expectation? (was Re: Documents with mixed languages)

Details
Message ID
<7cccb279-875a-0e8f-9c36-24cc20b54884@marmaladefoo.com>
In-Reply-To
<20211213191039.GD22651@brevard.conman.org> (view parent)
DKIM signature
missing
Download raw message
Hello

On 13-Dec-2021 19:10, Sean Conner wrote:
> 
>    If you receive a MIME type of "audio/*", "video/*" or "text/event-stream"
> [1], then you probably would want to stream the data or expect to spend
> some time downloading the entire content (or abort entirely if you get
> "text/event-stream").

Although strictly speaking according to the workflow described in the 
spec, you might validly not know the media type until *after* you have 
received the response, the connection is closed, then you have processed 
the response :)

I still think the expectation that you may be dealing with indefinite 
streams either needs to be specified or considered an unspecified 
anomaly which clients may handle however they like. Currently, clients 
may quite rightfully abandon server responses that don't terminate 
within a certain amount of time.

 > 	https://html.spec.whatwg.org/multipage/server-sent-events.html

text/event-stream seems to be non-normative and also comes with a set of 
assumptions about stream resumption and other HTTP headers. So it is 
likely too complex for Gemini I presume.

regards

  - Luke
Details
Message ID
<wg2PgF80Ih9cLn3TDK7I0VBOYQrpMD-Ir7OQuLVUOl12gDlHKeFO4KzUin2UnCkfYl5Y1-0mmptdeYYDj5gJx9LqAUpvCF7_FAohhsxie6s=@protonmail.com>
In-Reply-To
<20211213212225.d6b473389fee1ad8ba426506@gmail.com> (view parent)
DKIM signature
missing
Download raw message
Do you just think constitutions are read literally without context? What about the Bible? They are NOT. Why? Because intention behind words matter. Solderpunk has given us a ton of posts that detail his intention, so that in a case where there is ambiguity or some disparity between the spec and what was intended, we can actually know this by looking at what he wrote and what was desired. This is important because it tells us what in the spec needs to be made more explicit.

Christian Seibold
Details
Message ID
<nVyONc98W6q0erQONwkVkGb8pFBnHBmlJU9fVRmw9zjfmD4hufWfm-bz2wr_8SFTiWrlUpseONUrzUYKvQBB_8uj9YYux-9phBRI7djXszE=@protonmail.com>
In-Reply-To
<20211213212225.d6b473389fee1ad8ba426506@gmail.com> (view parent)
DKIM signature
missing
Download raw message
And the argument that this cannot be desired in the spec because the spec would explicitly state so is a non-argument. People can easily make mistakes or not think about how their words might be against what they intended to say, which is why intention behind words is more important then the actual words - and hence why overly-literal interpretations are in fact *interpretations* and not just reading what was written.

Again, I was never against making the spec more explicit and up to par with what was intended. What I am for is not judging implementations for using an intended design of the protocol (even if it's not in the spec!) when a spec isn't even final yet.

Christian Seibold

Is streaming a normative protocol expectation? (was Re: Documents with mixed languages)

Details
Message ID
<-oaBeFX8m8OvmRRoDSw-gAiVAzupf91Ion37ndKs8BjykirBurfaC0tPtlg_SE6FUfdlqOduAzyblrn8Vwk4kdgZs1fAXZ3OzrbbrwxEwwo=@protonmail.com>
In-Reply-To
<da33a30c-b0e3-dbf0-e684-282ab9b64a28@marmaladefoo.com> (view parent)
DKIM signature
missing
Download raw message
You know what, this whole argument was actually useless, because it's based on a portion of an outdated version of the spec, that portion of text no longer even being present in the latest version of the spec on gitlab: https://gitlab.com/gemini-specification/protocol/-/blob/master/specification.gmi

If we were to read this latest version of the spec, there is absolutely nothing in there that would precluding handling data before the close_notify. Period. Therefore, all clients that do this *are* compliant with the latest WIP version of the spec.

Additionally, it is important to quote this portion of the latest spec:
> While this document just covers the protocol with some mandates for what
> clients and servers have to do, there are other aspects of Gemini that
> aren't covered here in the specification which fall outside the core
> protocol.  Implementors of both clients and servers are RECOMMENDED to
> follow the best practice guide for the Gemini protocol.

and

> Upon sending the complete response (which may include
> content), the server closes the connection and MUST use the TLS close_notify
> mechanism to inform the client that no more data will be sent.

Christian Seibold

Is streaming a normative protocol expectation? (was Re: Documents with mixed languages)

Details
Message ID
<bc9012c2-f7bd-7b83-b6d3-c127119e6f0e@marmaladefoo.com>
In-Reply-To
<-oaBeFX8m8OvmRRoDSw-gAiVAzupf91Ion37ndKs8BjykirBurfaC0tPtlg_SE6FUfdlqOduAzyblrn8Vwk4kdgZs1fAXZ3OzrbbrwxEwwo=@protonmail.com> (view parent)
DKIM signature
missing
Download raw message
On 13-Dec-2021 20:55, Krixano wrote:
> that portion of text no longer even being present in the latest version of the spec on gitlab: https://gitlab.com/gemini-specification/protocol/-/blob/master/specification.gmi

Thanks for the link to the updated protocol, I was just going on the 
version on the gemini home capsule as the official version.

Clearly whatever solderpunk puts in is what the spec is. And there is 
less said now in the current draft of the spec about the expected 
interleaving of client and server events. This leaves more room for 
servers to expect clients to process the data before it is completed.

Still it seems unclear to me how a client can now tell between a faulty 
or excessively slow server and one that is valid but is streaming 
content very slowly. In the previous draft the expectation was the 
client waits for content, which will conclude, then it is processed. Now 
clients don't get to have a general expectation that the content will 
conclude or not as it comes down the pipe.

Probably the best option is to try to guess whether the content might be 
a stream from the media type as spc suggests, although that seems like 
guesswork to me.

Anyway, clients and servers probably need to have a connection timeout 
to prevent tar pits.

regards

 ?- Luke

Re: Is streaming a normative protocol expectation? (was Re: Documents with mixed languages)

romina xj ix k <gemini@xj-ix.luxe>
Details
Message ID
<E4E785FE-35A9-49A7-9BF8-F714F0F99B6A@xj-ix.luxe>
In-Reply-To
<bc9012c2-f7bd-7b83-b6d3-c127119e6f0e@marmaladefoo.com> (view parent)
DKIM signature
missing
Download raw message
On December 13, 2021 9:38:04 PM UTC, Luke Emmet <luke at marmaladefoo.com> wrote:
>
>On 13-Dec-2021 20:55, Krixano wrote:
>> that portion of text no longer even being present in the latest version of the spec on gitlab: https://gitlab.com/gemini-specification/protocol/-/blob/master/specification.gmi
>
>Thanks for the link to the updated protocol, I was just going on the 
>version on the gemini home capsule as the official version.
>
>Clearly whatever solderpunk puts in is what the spec is. And there is 
>less said now in the current draft of the spec about the expected 
>interleaving of client and server events. This leaves more room for 
>servers to expect clients to process the data before it is completed.
>
>Still it seems unclear to me how a client can now tell between a faulty 
>or excessively slow server and one that is valid but is streaming 
>content very slowly. In the previous draft the expectation was the 
>client waits for content, which will conclude, then it is processed. Now 
>clients don't get to have a general expectation that the content will 
>conclude or not as it comes down the pipe.
>
>Probably the best option is to try to guess whether the content might be 
>a stream from the media type as spc suggests, although that seems like 
>guesswork to me.
>
>Anyway, clients and servers probably need to have a connection timeout 
>to prevent tar pits.
>
>regards
>
> ?- Luke

afaik the canonical spec lives on gemini.circumlunar.space not gitlab

Is streaming a normative protocol expectation? (was Re: Documents with mixed languages)

Details
Message ID
<CY5MiKqtMLWkEvpgp91xapLggPIvsHQyOccGEePsXuV_rCUwMEpX3eUFddyESU0H2uNNZkcN9Cs8yk_FUaNkHutCgNbTRb8GKeCqXgkhxX0=@protonmail.com>
In-Reply-To
<E4E785FE-35A9-49A7-9BF8-F714F0F99B6A@xj-ix.luxe> (view parent)
DKIM signature
missing
Download raw message
Yes, you are right, actually, the Speculative Specification at the gemini homepage is the canonical one for the moment. The gitlab spec is for tracking changes to future versions of the spec.

What I'm frustrated with is that we have a bunch of evidence that shows that the intention has always been that clients could handle data before the close_notify, and yet people insist that this is not the case and that this is all just made-up crap being read into the spec. Intention behind what the spec says matters, because it's how we know whether the spec needs to be further clarified.
This is why I think we should absolutely refrain from judging implementations that are not "spec-compliant" based on a non-finalized *speculative* specification. Experimentation early on in Gemini was appreciated because it allowed people to test out the protocol to see what could be done with it - this included a simple change to some clients that allowed for handling of data before close_notify. There is *no doubt* in my mind that allowing this was intended. Whether the specification allows for this is another matter that needs to then be clarified. Which is why I have created a GitLab Issue here: https://gitlab.com/gemini-specification/protocol/-/issues/44

Christian Seibold

Sent with ProtonMail Secure Email.

??????? Original Message ???????

On Monday, December 13th, 2021 at 3:44 PM, romina xj ix k <gemini at xj-ix.luxe> wrote:

> On December 13, 2021 9:38:04 PM UTC, Luke Emmet luke at marmaladefoo.com wrote:
>
> > On 13-Dec-2021 20:55, Krixano wrote:
> >
> > > that portion of text no longer even being present in the latest version of the spec on gitlab: https://gitlab.com/gemini-specification/protocol/-/blob/master/specification.gmi
> >
> > Thanks for the link to the updated protocol, I was just going on the
> >
> > version on the gemini home capsule as the official version.
> >
> > Clearly whatever solderpunk puts in is what the spec is. And there is
> >
> > less said now in the current draft of the spec about the expected
> >
> > interleaving of client and server events. This leaves more room for
> >
> > servers to expect clients to process the data before it is completed.
> >
> > Still it seems unclear to me how a client can now tell between a faulty
> >
> > or excessively slow server and one that is valid but is streaming
> >
> > content very slowly. In the previous draft the expectation was the
> >
> > client waits for content, which will conclude, then it is processed. Now
> >
> > clients don't get to have a general expectation that the content will
> >
> > conclude or not as it comes down the pipe.
> >
> > Probably the best option is to try to guess whether the content might be
> >
> > a stream from the media type as spc suggests, although that seems like
> >
> > guesswork to me.
> >
> > Anyway, clients and servers probably need to have a connection timeout
> >
> > to prevent tar pits.
> >
> > regards
> >
> > -   Luke
>
> afaik the canonical spec lives on gemini.circumlunar.space not gitlab

Is streaming a normative protocol expectation? (was Re: Documents with mixed languages)

Details
Message ID
<20211213234103.45453106c7b79c9cbb3641e5@gmail.com>
In-Reply-To
<-oaBeFX8m8OvmRRoDSw-gAiVAzupf91Ion37ndKs8BjykirBurfaC0tPtlg_SE6FUfdlqOduAzyblrn8Vwk4kdgZs1fAXZ3OzrbbrwxEwwo=@protonmail.com> (view parent)
DKIM signature
missing
Download raw message
On Mon, 13 Dec 2021 20:55:11 +0000
Krixano <krixano at protonmail.com> wrote:

> You know what, this whole argument was actually useless, because it's
> based on a portion of an outdated version of the spec, that portion of
> text no longer even being present in the latest version of the spec on
> gitlab:
> https://gitlab.com/gemini-specification/protocol/-/blob/master/specification.gmi

"NOTE: This is a work in progress.  Until it's finalized, this is NOT
the official specification."

FYI, the current spec is here:
gemini://gemini.circumlunar.space/docs/specification.gmi

-- 
Philip
Details
Message ID
<20211213234853.9c42b92182dd3494836291d3@gmail.com>
In-Reply-To
<nVyONc98W6q0erQONwkVkGb8pFBnHBmlJU9fVRmw9zjfmD4hufWfm-bz2wr_8SFTiWrlUpseONUrzUYKvQBB_8uj9YYux-9phBRI7djXszE=@protonmail.com> (view parent)
DKIM signature
missing
Download raw message
On Mon, 13 Dec 2021 20:38:12 +0000
Krixano <krixano at protonmail.com> wrote:

> And the argument that this cannot be desired in the spec because the
> spec would explicitly state so is a non-argument.

I am not aware that anyone has made that argument. I certainly haven't.
Perhaps you aren't reading my messages linearly and literally enough.

> People can easily make mistakes or not think about how their words
> might be against what they intended to say, which is why intention
> behind words is more important then the actual words - and hence why
> overly-literal interpretations are in fact *interpretations* and not
> just reading what was written.

Yes, people can make mistakes. If such a mistake has happened here, it
has resulted in the specification (yes, not finalized) doesn't
practically permit indefinitely streamed responses.

> Again, I was never against making the spec more explicit and up to par
> with what was intended. What I am for is not judging implementations
> for using an intended design of the protocol (even if it's not in the
> spec!) when a spec isn't even final yet.

It's the document we have as a basis for writing clients. If it's
wrong, it's wrong. That can be fixed. But telling me that I should dig
through old gopher holes and mailing list archives just to get a sense
of Solderpunk's general opinion on streaming request responses, and
interpret that opinion as a canonical part of the standard is not a
viable basis for a standard.

-- 
Philip
Details
Message ID
<20211214001712.2e146d812c62f009c8fe3de7@gmail.com>
In-Reply-To
<wg2PgF80Ih9cLn3TDK7I0VBOYQrpMD-Ir7OQuLVUOl12gDlHKeFO4KzUin2UnCkfYl5Y1-0mmptdeYYDj5gJx9LqAUpvCF7_FAohhsxie6s=@protonmail.com> (view parent)
DKIM signature
missing
Download raw message
On Mon, 13 Dec 2021 20:31:23 +0000
Krixano <krixano at protonmail.com> wrote:

> Do you just think constitutions are read literally without context?
> What about the Bible? They are NOT.

Thankfully I have little reason to implement software that interprets
the Bible or any legal constitution in such a way that it shares its
interpretation with other software. Metaphors like this are only
confusing. It's simply not a valid comparison. A computer communication
protocol gains from conformity and lack of ambiguity. A machine
eventually needs to execute code according to the protocol, and a
machine can't make a liberal but reasonable interpretation about
what exactly inherently ambiguous concepts like freedom of speech or
reverence entail. Laws and religious texts on the other hand gain from
ambiguity and wealth of interpretation; they're to be both implemented
and executed by humans who enjoy a luxury machines probably never will
have.

If I could feed into a machine the fact that Solderpunk thinks
streaming response experiments are cool and interesting, and get an
unambiguously correct implementation out of it, I would, but I can't.

> Why? Because intention behind words matter. Solderpunk has given us a
> ton of posts that detail his intention, so that in a case where there
> is ambiguity or some disparity between the spec and what was intended,
> we can actually know this by looking at what he wrote and what was
> desired.

As far as I'm concerned, until they're in the specification, they're
just ponderings. Why should I have to care, as an implementor, about
small experiments and loosely defined ideas?

It's my opinion that if someone who has never visited the mailing
list and who doesn't keep tabs on Solderpunk's other writing takes the
specification and implements it exactly as written, they're correct. If
their software then doesn't interoperate with other software
implementing the protocol, either the other software or the
specification is incorrect.

Whether I, personally, can read it and consider its content with the
context of having followed the mailing list or having followed some
phlogs early on is irrelevant. My point is that no one who wants to
implement the protocol should have to waste that time to achieve
interoperability.

> This is important because it tells us what in the spec needs
> to be made more explicit.

100% agreed, because the specification is the canonical representation
of the protocol. I wouldn't expect anyone to look further than the
specification and the standards it refers to in order to learn how to
implement the protocol.

-- 
Philip
Details
Message ID
<PZP4iC0R8Zorh9a7vDgUgkD1Z26fejYblyJNPZt_jxWiWd-In8ke4J5XQCqsdygzkBhrhWv_PWYngwze-MHEexMC8fY699M7h4Ki2P2Amfg=@protonmail.com>
In-Reply-To
<20211214001712.2e146d812c62f009c8fe3de7@gmail.com> (view parent)
DKIM signature
missing
Download raw message
> If I could feed into a machine the fact that Solderpunk thinks
> streaming response experiments are cool and interesting, and get an
> unambiguously correct implementation out of it, I would, but I can't.

You can, it's called programming. The "unabiguously correct implementation" is to start handling the data the moment is gets to the client. Again, you're trying to overcomplicate this to make your point look better, when it's literally not. The most obvious thing to do would be to handle the data as it comes in. That's the natural thing to do, it's what most things do, and it's what most users expect, and it doesn't involve any ambiguity or breaking changes to capsules that don't use streaming (because the disconnect is happening in the opposite direction, when a client doesn't handle data as it comes in when the server might want it to).

> It's my opinion that if someone who has never visited the mailing
> list and who doesn't keep tabs on Solderpunk's other writing takes the
> specification and implements it exactly as written, they're correct. If
> their software then doesn't interoperate with other software
> implementing the protocol, either the other software or the
> specification is incorrect.

This is not what you originally said, fyi.

> Whether I, personally, can read it and consider its content with the
> context of having followed the mailing list or having followed some
> phlogs early on is irrelevant. My point is that no one who wants to
> implement the protocol should have to waste that time to achieve
> interoperability.

Nobody said otherwise. What you shouldn't do is act like clients that allow for streaming are somehow ruining Gemini by adhering to the intended designs of the protocol, or even by adhering to the latest non-official specification of the protocol even if it conflicts with the official speculative specification. This isn't a 'tyranny of a multitude of "loud minorities"'.

> The ideal outcome of a formal, canonical standard is
> that an implementor should not have to go spelunking and survey existing
> implementation (or indeed this mailing list) to learn what feature set
> a client or a server is currently expected to implement not to be
> effectively broken.

And in my opinion, this should only apply to finalized specifications. Why? Because a non-finalized specifications might desire things that haven't been added or fixed in the formal specification, and it's useful to not have the ecosystem be all over the place when the protocol is still growing or finalizing. Do you think there was a fully written out spec before the first Gemini servers were created? Of course not, because doing so would have been disastrous - there would have been no testing done whatsoever, so the spec would be just a bunch of theoretical crap with no real-world experience behind it.

Christian Seibold
Details
Message ID
<Of0jU7KuSvImIQCbjObdnbMa_KmuryH4KfbfbWKm1QzriCpDMnw1Cr08pjvpSuCQH0vNnSKUiclEV3X1CILYe1d4lUBB68HJjB1UG7qiCOc=@protonmail.com>
In-Reply-To
<PZP4iC0R8Zorh9a7vDgUgkD1Z26fejYblyJNPZt_jxWiWd-In8ke4J5XQCqsdygzkBhrhWv_PWYngwze-MHEexMC8fY699M7h4Ki2P2Amfg=@protonmail.com> (view parent)
DKIM signature
missing
Download raw message
Let me just be clear here, in response to this:
> I can instead treat it as a weird
> mutation of the protocol, and generally not worry about the whims of
> a few implementors that ignore the spec in various ways. In this way, I
> avoid the tyranny of a multitude of "loud minorities" that would have
> been much more disruptive without a canonical specification of the
> protocol.

1.) I agree with what is your most important argument, Clients and Servers should only be *expected* to adhere to the literalness of the spec
2.) The intended protocol design is necessary for pointing out needed clarifications to the specification, and for people who want to adhere to the intended design of the spec.
3.) Clients and Servers should therefore not be derided for adhering to the intended design even if it goes against the literalness of the spec, as the spec is possibly flawed or unclarified while still n development.
4.) Clients and Servers that adhere to the literalness of the official spec are just as valid as clients that adhere to the intentions behind the spec when it comes to an unfinalized spec. *Both* are valid.

Christian Seibold

Is streaming a normative protocol expectation? (was Re: Documents with mixed languages)

Sean Conner <sean@conman.org>
Details
Message ID
<20211214010916.GG22651@brevard.conman.org>
In-Reply-To
<7cccb279-875a-0e8f-9c36-24cc20b54884@marmaladefoo.com> (view parent)
DKIM signature
missing
Download raw message
It was thus said that the Great Luke Emmet once stated:
> Hello
> 
> On 13-Dec-2021 19:10, Sean Conner wrote:
> >
> >   If you receive a MIME type of "audio/*", "video/*" or 
> >   "text/event-stream"
> >[1], then you probably would want to stream the data or expect to spend
> >some time downloading the entire content (or abort entirely if you get
> >"text/event-stream").
> 
> Although strictly speaking according to the workflow described in the 
> spec, you might validly not know the media type until *after* you have 
> received the response, the connection is closed, then you have processed 
> the response :)

  I think you are talking about this from the specification: [1]

C:   Opens connection
S:   Accepts connection
C/S: Complete TLS handshake (see section 4)
C:   Validates server certificate (see 4.2)
C:   Sends request (one CRLF terminated line) (see section 2)
S:   Sends response header (one CRLF terminated line), closes connection
     under non-success conditions (see 3.1 and 3.2)
S:   Sends response body (text or binary data) (see 3.3)
S:   Closes connection (including TLS close_notify, see section 4)
C:   Handles response (see 3.4)

  Yes, it's easier for a client to implement this as shown above---read all
the input, then handle it.  But things can go wrong for a variety of reasons
(bugs, malicious intent) so clients should be able to deal with a server
that doesn't close the connection (or the connection is so slow that it's
preferrable for the client to abort the transaction) and the server should
be able to deal with a client that closes the connection before it does. 

  Now strictly speaking, the client knows the MIME type as soon as it gets
the response header.  How the client deals with the response body (see 3.3
and 3.4) is up to the client.

  Yes, I get the complaint about having to read anciliary sources to
"properly" implement the protocol (my biggest beef---not many bot writers
have bothered reading the "best practices" document sitting along side the
specification [2]), but as I mentioned above, there are a ton of corner
cases that need to be considered when implementing the protocol that aren't
mentioned in either document (like text with no whitespace---how do you deal
with formatting?).  

  -spc

[1]	gemini://gemini.circumlunar.space/docs/specification.gmi

[2]	gemini://gemini.circumlunar.space/docs/best-practices.gmi
Details
Message ID
<20211214022426.48a39b6f35f141f6961e9756@gmail.com>
In-Reply-To
<Of0jU7KuSvImIQCbjObdnbMa_KmuryH4KfbfbWKm1QzriCpDMnw1Cr08pjvpSuCQH0vNnSKUiclEV3X1CILYe1d4lUBB68HJjB1UG7qiCOc=@protonmail.com> (view parent)
DKIM signature
missing
Download raw message
On Tue, 14 Dec 2021 00:21:52 +0000
Krixano <krixano at protonmail.com> wrote:

> Let me just be clear here, in response to this:
> > I can instead treat it as a weird
> > mutation of the protocol, and generally not worry about the whims of
> > a few implementors that ignore the spec in various ways. In this way, I
> > avoid the tyranny of a multitude of "loud minorities" that would have
> > been much more disruptive without a canonical specification of the
> > protocol.
> 
> 1.) I agree with what is your most important argument, Clients and Servers should only be *expected* to adhere to the literalness of the spec

If by that you mean that they should be expected to adhere to the
specification, yes, we agree. I don't know exactly what you mean by the
"literalness of the spec".

> 2.) The intended protocol design is necessary for pointing out needed clarifications to the specification, and for people who want to adhere to the intended design of the spec.

Yes, changes to the protocol need to be considered in terms of the
objectives of the project, and we can use the objectives to discuss what
should be specified. But the objectives insofar that they haven't been
specified aren't a specification.

> 3.) Clients and Servers should therefore not be derided for adhering to the intended design even if it goes against the literalness of the spec, as the spec is possibly flawed or unclarified while still n development.

Agreed, but I think derided is a little too strong a word if you are
referring to my example that you quoted. If that's how it came across, I
apologize to who ever felt it unfair, and I recognize that it's blunt
to the point that it might seem rude without the tone only I can hear.
My point was that diverging implementations that stray from the
specification create exactly the kind of governance by "loud minority"
that Michael Lazar sees in centralized authority over the protocol,
just multiplied by the number of different divergent behaviors.

> 4.) Clients and Servers that adhere to the literalness of the official spec are just as valid as clients that adhere to the intentions behind the spec when it comes to an unfinalized spec. *Both* are valid.

So we have two protocols. Possibly many more, depending on our value
judgement of what the "intended spec" is. As far as I'm concerned,
there either is a specification, or there isn't. There can be working
ideas and discussions for changes to or replacement of a specification,
but that isn't a specification unto itself until it actually specifies
something.

As far as validity go, I wouldn't say in any absolute sense that a
particular piece of software is valid or invalid. But I do consider
software that fails to fulfill a specification to be incorrect in terms
of the specification. I try not to add much value to that; sometimes
that's a good thing and sometimes it's not. But it seems clear that we
both agree that it's better for everyone if the specification and
implementations reflect each other closely.

-- 
Philip
Details
Message ID
<c43670604058902bdec430e0b911a84f8e1c012c.camel@posteo.mx>
In-Reply-To
<20211213234853.9c42b92182dd3494836291d3@gmail.com> (view parent)
DKIM signature
missing
Download raw message
On Mon, 2021-12-13 at 23:48 +0100, Philip Linde wrote:
> On Mon, 13 Dec 2021 20:38:12 +0000, Krixano <krixano at protonmail.com> wrote:
> 
> > Again, I was never against making the spec more explicit and up to par
> > with what was intended. What I am for is not judging implementations
> > for using an intended design of the protocol (even if it's not in the
> > spec!) when a spec isn't even final yet.
> 
> It's the document we have as a basis for writing clients. If it's
> wrong, it's wrong. That can be fixed. But telling me that I should dig
> through old gopher holes and mailing list archives just to get a sense
> of Solderpunk's general opinion on streaming request responses, and
> interpret that opinion as a canonical part of the standard is not a
> viable basis for a standard.

This is a fundamental misunderstanding of Gemini's current state. Right
now, there is no standard; we have a specification, but it is not yet
standardized. Yes, email archives and old phlogs is not a viable basis
for a standard. That is why the spec is being revised - so that it can
become a standard without the previously-mentioned issues.

Cheers,
-- 
DJ Chase
They, Them, Theirs

Is streaming a normative protocol expectation? (was Re: Documents with mixed languages)

Details
Message ID
<27c8ae3f3f9313295b2a0e3a93b4a31a46389fa4.camel@posteo.mx>
In-Reply-To
<20211214010916.GG22651@brevard.conman.org> (view parent)
DKIM signature
missing
Download raw message
On Mon, 2021-12-13 at 20:09 -0500, Sean Conner wrote:
>   Yes, I get the complaint about having to read anciliary sources to
> "properly" implement the protocol (my biggest beef---not many bot writers
> have bothered reading the "best practices" document sitting along side the
> specification [2]), but as I mentioned above, there are a ton of corner
> cases that need to be considered when implementing the protocol that aren't
> mentioned in either document (like text with no whitespace---how do you deal
> with formatting?).
> 
>   -spc
> 
> [2]	gemini://gemini.circumlunar.space/docs/best-practices.gmi

Not strictly on-topic, but I don't think a Gemtext document without
whitespace is a corner-case because it doesn't need to be treated
differently at all. It's just a document with only one line;
parse the first three characters and format it accordingly.

Cheers,
-- 
DJ Chase
They, Them, Theirs
Reply to thread Export thread (mbox)