~bitfehler/m2dir

4 2

[Proposal] JMAP metadata

Details
Message ID
<D0LWP3NLXLIU.2CJ2UN40K0AHY@timculverhouse.com>
DKIM signature
pass
Download raw message
Hello -

First of all, this is a great spec. Maildir has considerable problems and I
couldn't agree more with your September post about it. Static filenames are the
way to go, and I'm glad you went that route with this. I also see how easily
this spec can be extended to support JMAP, and would propose adding two
additional `.meta` filetypes to fully support JMAP. These filetypes would need
to 1. store mailbox metadata and 2. store email-id to mailbox-id maps. The
second filetype would be exactly as the `.flags` filetype, but associating to a
mailbox ID as opposed to a static flag.

The first metadata file would need to contain the following information, and
could be a "mailbox per line" syntax, or even a folder containing the metadata:

- name
- id
- [parent-id]

These three pieces of data would be the minimal to allow storage of a complete
JMAP mailstore. There could be some data loss regarding mailbox access control,
but that is true on the IMAP side as well, and probably not worth considering
for a simple file storage spec such as this. At that point, a database makes
more sense.

I'm happy to bikeshed the design of these two files if this would be accepted!

-- 
Tim
Details
Message ID
<D0LWURXRH531.270DPPLTEQFUJ@timculverhouse.com>
In-Reply-To
<D0LWP3NLXLIU.2CJ2UN40K0AHY@timculverhouse.com> (view parent)
DKIM signature
pass
Download raw message
As an aside, I would also propose using the IANA naming for flags, to avoid any
weird escaping issues that come with using '\'

  $MDNSent
  $Forwarded
  $SubmitPending
  $Submitted
  $Important
  $draft
  $seen
  $flagged
  $answered
  $recent

  # mutually exclusive flags
  [$Junk]
  [$NotJunk]
  [$Phishing]

-- 
Tim
Details
Message ID
<e01781ba-0436-4118-af06-883628bf1ffe@bitfehler.net>
In-Reply-To
<D0LWP3NLXLIU.2CJ2UN40K0AHY@timculverhouse.com> (view parent)
DKIM signature
pass
Download raw message
Hi,
On 4/17/24 12:39 AM, Tim Culverhouse wrote:
> First of all, this is a great spec. Maildir has considerable problems and I
> couldn't agree more with your September post about it. Static filenames are the
> way to go, and I'm glad you went that route with this. I also see how easily
> this spec can be extended to support JMAP, and would propose adding two
> additional `.meta` filetypes to fully support JMAP. These filetypes would need
> to 1. store mailbox metadata and 2. store email-id to mailbox-id maps. The
> second filetype would be exactly as the `.flags` filetype, but associating to a
> mailbox ID as opposed to a static flag.
> 
> The first metadata file would need to contain the following information, and
> could be a "mailbox per line" syntax, or even a folder containing the metadata:
> 
> - name
> - id
> - [parent-id]
> 
> These three pieces of data would be the minimal to allow storage of a complete
> JMAP mailstore. There could be some data loss regarding mailbox access control,
> but that is true on the IMAP side as well, and probably not worth considering
> for a simple file storage spec such as this. At that point, a database makes
> more sense.

Thanks for bringing this up. The intention of M2dir was of course to be 
remote-store-agnostic, and as such I certainly want to support JMAP as 
well. I admittedly do not have a lot of experience with JMAP, so I will 
need to read up on a few things you mention here.

I will point out, though: the idea is to only store the messages and 
their message-specific metadata, _not_ remote-store specific metadata. I 
can only provide an IMAP example: anything you'd need to map a message 
in an m2dir to a message on an IMAP server (such as a UID/UIDVALIDITY 
pair) is to be handled by the application doing the synchronization. 
 From a quick look, I think at least some of the data you are suggesting 
falls into this category (e.g. your item 2.)?

Not sure right now about "mailbox metadata". I'll do some reading up on 
JMAP to at least better understand the terminology and get back to you 
about this.

Cheers,
Conrad
Details
Message ID
<1c450bf2-6b6e-46b8-ab64-a37653b84cd5@bitfehler.net>
In-Reply-To
<D0LWURXRH531.270DPPLTEQFUJ@timculverhouse.com> (view parent)
DKIM signature
pass
Download raw message
Hey,

On 4/17/24 12:46 AM, Tim Culverhouse wrote:
> As an aside, I would also propose using the IANA naming for flags, to avoid any
> weird escaping issues that come with using '\'

I agree, even if not for the same reason ($ will also need to be escaped 
in various places, but ¯\_(ツ)_/¯ ). But the current version is clearly 
IMAP-biased there, and using the IANA naming makes it more neutral, 
which I like. I will update the spec accordingly.

Cheers,
Conrad
Details
Message ID
<D0MFPGCRRHR4.1YFKNGAFSDB89@timculverhouse.com>
In-Reply-To
<e01781ba-0436-4118-af06-883628bf1ffe@bitfehler.net> (view parent)
DKIM signature
pass
Download raw message
On Wed Apr 17, 2024 at 1:45 AM AKDT, Conrad Hoffmann wrote:
> Thanks for bringing this up. The intention of M2dir was of course to be 
> remote-store-agnostic, and as such I certainly want to support JMAP as 
> well. I admittedly do not have a lot of experience with JMAP, so I will 
> need to read up on a few things you mention here.
>
> I will point out, though: the idea is to only store the messages and 
> their message-specific metadata, _not_ remote-store specific metadata. I 
> can only provide an IMAP example: anything you'd need to map a message 
> in an m2dir to a message on an IMAP server (such as a UID/UIDVALIDITY 
> pair) is to be handled by the application doing the synchronization. 
>  From a quick look, I think at least some of the data you are suggesting 
> falls into this category (e.g. your item 2.)?

The reason for both of these files (mapping email to mailbox) is because JMAP
allows for an email to exist in multiple mailboxes, eg "labels" or "tags". This
could be a design choice to limit the model to only allow an email to exist in
one mailbox and then these files wouldn't be needed - at that point JMAP is the
same data model as IMAP.

My thought was that a JMAP backend would store all the emails in a single folder
and clients would reference the .meta mailbox map to determine the mailbox(es)
for each message.

-- 
Tim
Reply to thread Export thread (mbox)