~emersion/soju-dev

Make sure that WebSocket messages are valid UTF-8 v1 SUPERSEDED

Hubert Hirtz: 1
 Make sure that WebSocket messages are valid UTF-8

 1 files changed, 2 insertions(+), 1 deletions(-)
Le 02/09/2020 à 16:57, Simon Ser a écrit :
Next
Hubert Hirtz <hubert@hirtzfr.eu> wrote:
Next
Export patchset (mbox)
How do I use this?

Copy & paste the following snippet into your terminal to import this patchset into git:

curl -s https://lists.sr.ht/~emersion/soju-dev/patches/13069/mbox | git am -3
Learn more about email & git
View this thread in the archives

[PATCH] Make sure that WebSocket messages are valid UTF-8 Export this patch

... by replacing invalid bytes with the REPLACEMENT CHARACTER U+FFFD

This is better than:
- discarding the whole message, since the user would not see it...
- removing invalid bytes, since the user would not see their presence,
- converting the encoding (this is actually not possible).

Contrary to its documentation, strings.ToValidUTF8 doesn't copy the
string if it's valid UTF-8:
<https://golang.org/src/strings/strings.go?s=15815:15861#L623>
---
 conn.go | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/conn.go b/conn.go
index 6914179..701e8f5 100644
--- a/conn.go
+++ b/conn.go
@@ -5,6 +5,7 @@ import (
	"fmt"
	"io"
	"net"
	"strings"
	"sync"
	"time"

@@ -62,7 +63,7 @@ func (wic websocketIRCConn) ReadMessage() (*irc.Message, error) {
}

func (wic websocketIRCConn) WriteMessage(msg *irc.Message) error {
	b := []byte(msg.String())
	b := []byte(strings.ToValidUTF8(msg.String(), "\ufffd"))
	ctx := context.Background()
	if !wic.writeDeadline.IsZero() {
		var cancel context.CancelFunc
-- 
2.28.0
Nice!

Can we use the unicode.ReplacementChar [1] constant instead of
hardcoding the value?

There's also bytes.ToValidUTF8, but not sure it's better than
strings.ToValidUTF8.

[1]: https://golang.org/pkg/unicode/#pkg-constants