~emersion/soju-dev

chathistory: Fix truncated backlog due to timezones v2 APPLIED

Alex Karle: 1
 chathistory: Fix truncated backlog due to timezones

 1 files changed, 6 insertions(+), 0 deletions(-)
Export patchset (mbox)
How do I use this?

Copy & paste the following snippet into your terminal to import this patchset into git:

curl -s https://lists.sr.ht/~emersion/soju-dev/patches/25581/mbox | git am -3
Learn more about email & git
View this thread in the archives

[PATCH v2] chathistory: Fix truncated backlog due to timezones Export this patch

Because msgstore_fs writes logs in localtime, the CHATHISTORY timestamps
(UTC) must be converted to localtime prior to filtering ranges ensure
the right range is sent back to the client.

Prior to this patch, the iteration back from the BEFORE time failed to
load the hours between midnight UTC and midnight localtime in each day's
logged messages. This is because the final time to be considered in a
day's log file (the "start" time) reuses the previous start time's
locale:

    start = time.Date(year, month, day, 0, 0, 0, 0, start.Location()).Add(-1)

By converting the original start and end from the CHATHISTORY commands
to localtime in Load*Time and ListTargets, we ensure we read through
midnight each day.

---

I tested this patch on my server and confirmed the backlog is being sent
completely (BEFORE). I haven't tested AFTER/BETWEEN/TARGETS, but they
should also be converted to localtime since they have the same
comparison issue with the localtime logs.

I think the alternative solution would be to convert the logs to UTC on
read (and for ListTargets convert the ModTime to UTC), so that all range
filtering is done in a known locale. It feels a bit cleaner this way,
but would be N timezone conversions instead of a constant of 2 (I'm not
sure how timezones are implemented in Go / whether the conversion has
any overhead).
If you have a preference for one over the other, I'm happy to send a v3!

Thanks,
Alex

 msgstore_fs.go | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/msgstore_fs.go b/msgstore_fs.go
index 53b6b6e..20653c8 100644
--- a/msgstore_fs.go
+++ b/msgstore_fs.go
@@ -383,6 +383,8 @@ func (ms *fsMessageStore) parseMessagesAfter(network *network, entity string, re
}

func (ms *fsMessageStore) LoadBeforeTime(network *network, entity string, start time.Time, end time.Time, limit int) ([]*irc.Message, error) {
	start = start.In(time.Local)
	end = end.In(time.Local)
	history := make([]*irc.Message, limit)
	remaining := limit
	tries := 0
@@ -406,6 +408,8 @@ func (ms *fsMessageStore) LoadBeforeTime(network *network, entity string, start
}

func (ms *fsMessageStore) LoadAfterTime(network *network, entity string, start time.Time, end time.Time, limit int) ([]*irc.Message, error) {
	start = start.In(time.Local)
	end = end.In(time.Local)
	var history []*irc.Message
	remaining := limit
	tries := 0
@@ -472,6 +476,8 @@ func (ms *fsMessageStore) LoadLatestID(network *network, entity, id string, limi
}

func (ms *fsMessageStore) ListTargets(network *network, start, end time.Time, limit int) ([]chatHistoryTarget, error) {
	start = start.In(time.Local)
	end = end.In(time.Local)
	rootPath := filepath.Join(ms.root, escapeFilename(network.GetName()))
	root, err := os.Open(rootPath)
	if err != nil {
-- 
2.32.0
Pushed, thanks!

On Thursday, October 7th, 2021 at 05:39, Alex Karle <alex@alexkarle.com> wrote: