Hi,
I tried to understand the how the IME works with Gio text editing
recently. There is a little problem
I found hard to handle. For each pre-composition, Gio sent a
key.EditEvent and a key.SelectionEvent.
The pre-composed text is inserted into the text buffer which is
important for visual cues. But there's a problem.
Undo operation would handle the pre-composed text as well, which makes
the undo look weird.
And in my case of using a Piece table buffer, the pre-composed texts
are appended to the add buffer
in an incremental ways which makes the buffer grow unnecessarily large.
So I would like to know if there's any way to know when the IME
composition is committed, so that
we can buffer the pre-composed text and not insert into the buffer.
Thank you!
Best Regards,
Zhijian Zhang
On Fri, 17 Jan 2025 at 09:05, zhang zj <zhangzj33@gmail.com> wrote:
>> Hi,>> I tried to understand the how the IME works with Gio text editing> recently. There is a little problem> I found hard to handle. For each pre-composition, Gio sent a> key.EditEvent and a key.SelectionEvent.> The pre-composed text is inserted into the text buffer which is> important for visual cues. But there's a problem.> Undo operation would handle the pre-composed text as well, which makes> the undo look weird.>> And in my case of using a Piece table buffer, the pre-composed texts> are appended to the add buffer> in an incremental ways which makes the buffer grow unnecessarily large.>> So I would like to know if there's any way to know when the IME> composition is committed, so that> we can buffer the pre-composed text and not insert into the buffer.>
Not that I know of. Note that I'm very much not an expert in IME and my
efforts in that area has so far been to make the minimally viable
implementation capable of working across platforms.
I'd love your input as to how you think would be a better way, or
extensions such as composition commit. One thing to know is whether
such information is generally available from ~all platforms.
Thanks,
Elias
> Not that I know of. Note that I'm very much not an expert in IME and my> efforts in that area has so far been to make the minimally viable> implementation capable of working across platforms.> I'd love your input as to how you think would be a better way, or> extensions such as composition commit. One thing to know is whether> such information is generally available from ~all platforms.>
Thanks for your great work! I agree that Gio only provides minimally viable
implementation for what can also be done in the application layer. But I would
be nice to have more support for general tasks from the framework layer.
The IME behavior I reported above is ok for latin scripts, but I think
this is vital
for languages such as Chinese, as I think general users might expect revert the
input in one undo instead of multiple undos.
As you suggested, I will try to look into the this and improve it. BTW, I tested
the latest version of Gio(v0.8.0), it also does not work well
regarding CJK character
input.The composition seems inconsistent that sometimes returns
composition result
without the first one or more input characters.
Best Regards,
Zhijian Zhang
On Sat, 18 Jan 2025 at 04:45, zhang zj <zhangzj33@gmail.com> wrote:
>> > Not that I know of. Note that I'm very much not an expert in IME and my> > efforts in that area has so far been to make the minimally viable> > implementation capable of working across platforms.> > I'd love your input as to how you think would be a better way, or> > extensions such as composition commit. One thing to know is whether> > such information is generally available from ~all platforms.> >>> Thanks for your great work! I agree that Gio only provides minimally viable> implementation for what can also be done in the application layer. But I would> be nice to have more support for general tasks from the framework layer.> The IME behavior I reported above is ok for latin scripts, but I think> this is vital> for languages such as Chinese, as I think general users might expect revert the> input in one undo instead of multiple undos.>
Absolutely. Another nice feature I'd like to see is the visible marking of the
current composition (typically a blue underline).
> As you suggested, I will try to look into the this and improve it. BTW, I tested> the latest version of Gio(v0.8.0), it also does not work well> regarding CJK character> input.The composition seems inconsistent that sometimes returns> composition result> without the first one or more input characters.>
Do you mean a regression compared to v0.7? If so, that's probably caused by
https://gioui.org/commit/8107ec22066f3e4d308d1591057fb1f6e45be3f8
Thanks,
Elias
<p dir="ltr">Gm dears I am grateful for sharing some possible solutions<br>
1. Distinguishing Pre-composed and Committed Text<br>
Approach: Track the IME state (pre-composition, committed, or canceled).<br>
API: Use key.EditEvent to differentiate between pre-composed text and committed text. Check if the text contains IME-specific markers or empty strings indicating pre-composition.<br>
2. Managing Undo Behavior<br>
Solution: Separate pre-composed text from the undo history. Only commit finalized text to the buffer for undo tracking.<br>
Implementation:<br>
Store pre-composed text in a temporary buffer.<br>
Commit text to the main buffer and undo history only on IME commit events.<br>
3. Improving CJK Input Consistency<br>
Solution: Investigate potential regressions between Gio v0.7 and v0.8. Ensure correct handling of EditEvent for multi-character compositions.</p>
<br><div class="gmail_quote gmail_quote_container"><div dir="ltr" class="gmail_attr">في الثلاثاء، 21 يناير 2025، 12:36 م Elias Naur <<a href="mailto:mail@eliasnaur.com">mail@eliasnaur.com</a>> كتب:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">On Sat, 18 Jan 2025 at 04:45, zhang zj <<a href="mailto:zhangzj33@gmail.com" target="_blank" rel="noreferrer">zhangzj33@gmail.com</a>> wrote:<br>
><br>
> > Not that I know of. Note that I'm very much not an expert in IME and my<br>
> > efforts in that area has so far been to make the minimally viable<br>
> > implementation capable of working across platforms.<br>
> > I'd love your input as to how you think would be a better way, or<br>
> > extensions such as composition commit. One thing to know is whether<br>
> > such information is generally available from ~all platforms.<br>
> ><br>
><br>
> Thanks for your great work! I agree that Gio only provides minimally viable<br>
> implementation for what can also be done in the application layer. But I would<br>
> be nice to have more support for general tasks from the framework layer.<br>
> The IME behavior I reported above is ok for latin scripts, but I think<br>
> this is vital<br>
> for languages such as Chinese, as I think general users might expect revert the<br>
> input in one undo instead of multiple undos.<br>
><br>
<br>
Absolutely. Another nice feature I'd like to see is the visible marking of the<br>
current composition (typically a blue underline).<br>
<br>
> As you suggested, I will try to look into the this and improve it. BTW, I tested<br>
> the latest version of Gio(v0.8.0), it also does not work well<br>
> regarding CJK character<br>
> input.The composition seems inconsistent that sometimes returns<br>
> composition result<br>
> without the first one or more input characters.<br>
><br>
<br>
Do you mean a regression compared to v0.7? If so, that's probably caused by<br>
<a href="https://gioui.org/commit/8107ec22066f3e4d308d1591057fb1f6e45be3f8" rel="noreferrer noreferrer" target="_blank">https://gioui.org/commit/8107ec22066f3e4d308d1591057fb1f6e45be3f8</a><br>
<br>
Thanks,<br>
Elias<br>
</blockquote></div>
<div dir="auto"><p dir="ltr">Hi guys</p>
<p dir="ltr">Thank you for clarifying the points regarding the IME issue in the Gio library, especially the impact of pre-composed text on user experience. I fully understand the importance of improving this aspect to provide better support for complex languages like Chinese. Based on the discussion, I would like to share a few thoughts:</p><p dir="ltr">> >Improving Undo Behavior: A dedicated buffer for pre-composed text could be implemented to store the text until a commit event is received.</p><p dir="ltr">Undo operations could then clear the pre-composed text in a single step instead of multiple incremental steps.</p><p dir="ltr">> >IME Commit Confirmation: If Gio does not currently support notifications for composition commit events, it might be worth exploring interactions with native input layers to retrieve commit information.</p>
<p dir="ltr">> >Handling the Current Buffer Model:For the Piece Table Buffer, an optimized garbage collection mechanism could be introduced to clean up unused or pre-composed text to avoid unnecessary memory growth.</p>
<p dir="ltr">> >Support for Non-Latin Scripts:It might be helpful to include tests for languages like Chinese or Japanese to ensure the next release of Gio works seamlessly with IME.</p><p dir="ltr">> >If you need further support or experimental samples to improve this behavior, I would be happy to help or conduct additional testing.</p>
<p dir="ltr">Thank you for your continuous efforts to enhance the Gio library!</p>
<p dir="ltr">Best regards,<br>Ahmed</p><br>
<br><div class="gmail_quote" dir="auto"><div dir="ltr" class="gmail_attr">---------- Forwarded message ---------<br>من: <strong class="gmail_sendername" dir="auto">1i1</strong> <span dir="auto"><<a href="mailto:albarraq415@gmail.com" target="_blank" rel="noreferrer">albarraq415@gmail.com</a>></span><br>Date: الثلاثاء، 21 يناير 2025، 12:59 م<br>Subject: Re: Finalized mark for IME Composition<br>To: Elias Naur <<a href="mailto:mail@eliasnaur.com" target="_blank" rel="noreferrer">mail@eliasnaur.com</a>><br>Cc: zhang zj <<a href="mailto:zhangzj33@gmail.com" target="_blank" rel="noreferrer">zhangzj33@gmail.com</a>>, <~eliasnaur/<a href="mailto:gio@lists.sr.ht" target="_blank" rel="noreferrer">gio@lists.sr.ht</a>><br></div><br><br><p dir="ltr">Gm dears I am grateful for sharing some possible solutions<br>
1. Distinguishing Pre-composed and Committed Text<br>
Approach: Track the IME state (pre-composition, committed, or canceled).<br>
API: Use key.EditEvent to differentiate between pre-composed text and committed text. Check if the text contains IME-specific markers or empty strings indicating pre-composition.<br>
2. Managing Undo Behavior<br>
Solution: Separate pre-composed text from the undo history. Only commit finalized text to the buffer for undo tracking.<br>
Implementation:<br>
Store pre-composed text in a temporary buffer.<br>
Commit text to the main buffer and undo history only on IME commit events.<br>
3. Improving CJK Input Consistency<br>
Solution: Investigate potential regressions between Gio v0.7 and v0.8. Ensure correct handling of EditEvent for multi-character compositions.</p>
<br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">في الثلاثاء، 21 يناير 2025، 12:36 م Elias Naur <<a href="mailto:mail@eliasnaur.com" rel="noreferrer noreferrer" target="_blank">mail@eliasnaur.com</a>> كتب:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">On Sat, 18 Jan 2025 at 04:45, zhang zj <<a href="mailto:zhangzj33@gmail.com" rel="noreferrer noreferrer noreferrer" target="_blank">zhangzj33@gmail.com</a>> wrote:<br>
><br> Not that I know of. Note that I'm very much not an expert in IME and my<br>
> > efforts in that area has so far been to make the minimally viable<br>
> > implementation capable of working across platforms.<br>
> > I'd love your input as to how you think would be a better way, or<br>
> > extensions such as composition commit. One thing to know is whether<br>
> > such information is generally available from ~all platforms.<br>
> ><br>
><br>
> Thanks for your great work! I agree that Gio only provides minimally viable<br>
> implementation for what can also be done in the application layer. But I would<br>
> be nice to have more support for general tasks from the framework layer.<br>
> The IME behavior I reported above is ok for latin scripts, but I think<br>
> this is vital<br>
> for languages such as Chinese, as I think general users might expect revert the<br>
> input in one undo instead of multiple undos.<br>
><br>
<br>
Absolutely. Another nice feature I'd like to see is the visible marking of the<br>
current composition (typically a blue underline).<br>
<br>
> As you suggested, I will try to look into the this and improve it. BTW, I tested<br>
> the latest version of Gio(v0.8.0), it also does not work well<br>
> regarding CJK character<br>
> input.The composition seems inconsistent that sometimes returns<br>
> composition result<br>
> without the first one or more input characters.<br>
><br>
<br>
Do you mean a regression compared to v0.7? If so, that's probably caused by<br>
<a href="https://gioui.org/commit/8107ec22066f3e4d308d1591057fb1f6e45be3f8" rel="noreferrer noreferrer noreferrer noreferrer" target="_blank">https://gioui.org/commit/8107ec22066f3e4d308d1591057fb1f6e45be3f8</a><br>
<br>
Thanks,<br>
Elias<br><br>
</blockquote></div>
</div>
</div>
> Absolutely. Another nice feature I'd like to see is the visible marking of the> current composition (typically a blue underline).>
Yes, that would be a nice feature to have if we can detect the
pre-composing text.
> Do you mean a regression compared to v0.7? If so, that's probably caused by> https://gioui.org/commit/8107ec22066f3e4d308d1591057fb1f6e45be3f8
Possibly https://git.sr.ht/~eliasnaur/gio/commit/d7a1ec7 as I kept a
local version
of that. The issue can easily be reproduced, but the same key strokes may have
different results. But the overall behaviour remains the same. Usually
the first one
or two keys do not anticipate the composition, only the remaining
keystrokes, which
is unexpected.
Best Regards,
Zhijian Zhang
On Wed, 22 Jan 2025 at 17:29, zhang zj <zhangzj33@gmail.com> wrote:
> > Do you mean a regression compared to v0.7? If so, that's probably caused by> > https://gioui.org/commit/8107ec22066f3e4d308d1591057fb1f6e45be3f8>> Possibly https://git.sr.ht/~eliasnaur/gio/commit/d7a1ec7 as I kept a> local version> of that. The issue can easily be reproduced, but the same key strokes may have> different results. But the overall behaviour remains the same. Usually> the first one> or two keys do not anticipate the composition, only the remaining> keystrokes, which> is unexpected.>
Can I ask you to give me precise reproduction steps? I need ELI5 instructions
because of my complete ignorance of IME :)
I tried the "Chinese, Simplified", "Pinyin - Simplified" IME in my
macOS keyboard
settings, and I couldn't immediately detect any behaviour diference between
Gio v0.8.0 and my Firefox address bar.
Elias
Sorry, I missed the mail. Here is the precise instructions to
reproduce the issue:
1. Switch to "Pinyin - Simplified" IME in macOS.
2. Press the following keys: "women". The IME should display some
candidate words. Possibly "我们" is in the first place.
3. Press Space key to confirm the input. Then we'll get the words "我们“
shown in the editor.
4. Then try to type the chinese word "的" by pressing the following keys: "de".
5. After that the IME would give you candidates having pinyin "e", or
similar ones. And the first candidate should be "额".
6. Press Space key to select the first candidate. This time we will
get the final result: "我们d额".
What I expected is "我们的". The above result is wrong as the character
"d" seems to be missed out during the composition
of "的" which has a pinyin of "de".
I hope this makes it a little clearer for you.
Best Regards,
Zhijian Zhang
On Thu, Jan 30, 2025 at 7:48 PM Elias Naur <mail@eliasnaur.com> wrote:
>> On Wed, 22 Jan 2025 at 17:29, zhang zj <zhangzj33@gmail.com> wrote:> > > Do you mean a regression compared to v0.7? If so, that's probably caused by> > > https://gioui.org/commit/8107ec22066f3e4d308d1591057fb1f6e45be3f8> >> > Possibly https://git.sr.ht/~eliasnaur/gio/commit/d7a1ec7 as I kept a> > local version> > of that. The issue can easily be reproduced, but the same key strokes may have> > different results. But the overall behaviour remains the same. Usually> > the first one> > or two keys do not anticipate the composition, only the remaining> > keystrokes, which> > is unexpected.> >>> Can I ask you to give me precise reproduction steps? I need ELI5 instructions> because of my complete ignorance of IME :)>> I tried the "Chinese, Simplified", "Pinyin - Simplified" IME in my> macOS keyboard> settings, and I couldn't immediately detect any behaviour diference between> Gio v0.8.0 and my Firefox address bar.>> Elias