I found a message in my junk email folder which was relayed by sr.ht from a protonmail user. Here are some relevant headers that my email provider added:
X-Spam-Score: 9.9
Sender: extsmtp
Authentication-Results: mx2.messagingengine.com; arc=none (no signatures found); dkim=none (no signatures found); dmarc=fail (p=quarantine,sp=none,d=quarantine,arc_aware_result=fail) header.from=protonmail.com; iprev=pass policy.iprev=173.195.146.137 (mail.sr.ht); spf=pass smtp.mailfrom=extsmtp@sr.ht smtp.helo=mail.sr.ht; x-aligned-from=fail; x-cm=none score=0; x-ptr=pass smtp.helo=mail.sr.ht policy.ptr=mail.sr.ht; x-return-mx=pass header.domain=protonmail.com policy.is_org=yes (MX Record found); x-return-mx=pass smtp.domain=sr.ht policy.is_org=yes (MX Record found); x-tls=pass smtp.version=TLSv1.2 smtp.cipher=ECDHE-RSA-AES128-GCM-SHA256 smtp.bits=128/128; x-vs=clean score=49 state=0
I think the problem is that sr.ht sets the From field as the original sender rather than munging it, and protonmail says to quarantine such impersonation with "v=DMARC1; p=quarantine; fo=1;"
Hi,
On Friday, September 14, 2018 8:30 AM, Joe Nelson <joe@begriffs.com> wrote:
> I found a message in my junk email folder which was relayed by sr.ht from a> protonmail user.
I've had similar issues for a while. Here are headers from one of Drew's
messages:
Authentication-Results: mail15i.protonmail.ch; dmarc=fail (p=none dis=none)
header.from=cmpwn.com
Authentication-Results: mail15i.protonmail.ch; spf=pass smtp.mailfrom=extsmtp@sr.ht
Authentication-Results: mail15i.protonmail.ch; dkim=permerror (0-bit key)
header.d=cmpwn.com header.i=@cmpwn.com header.b="bkInET4T"
Authentication-Results: mail.sr.ht; dkim=permerror (0-bit key) header.d=cmpwn.com
header.i=@cmpwn.com header.b=bkInET4T
Dkim-Signature: v=1; a=rsa-sha256; c=simple/simple; d=cmpwn.com; s=cmpwn; t=1536668082;
bh=uVF3RRE2NtZEru3TlueAfgpW5sTogn14yyTLhmsHslw=;
h=Date:From:To:Cc:Subject:References:In-Reply-To;
b=bkInET4TdpT3lNPLasPvGWbkYfDF0Y92lmfwMD65icRs3+0myd1f6h1WsJRp3LVj8
I9fi5Pxqz+ohiTJ7dg6y7XeKXMMTYgvLcddUFkMWx9Wld/oI/1TAGyLvmWpD7DLj4U
W+n01aZMcq8z5/axclVl+4/rDaKKKkdNv0wnxW7E=
Message-Id: <20180911120726.GA1649@homura.localdomain>
The interesting bit is that mail.sr.ht and ProtonMail both agree there's a DKIM
issue.
However your issues seem different. I don't have DMARC errors with
ProtonMail emails.
On 2018-09-14 1:30 AM, Joe Nelson wrote:
> I think the problem is that sr.ht sets the From field as the original> sender rather than munging it, and protonmail says to quarantine such> impersonation with "v=DMARC1; p=quarantine; fo=1;"
This is an important feature of email, I'd suggest your mail server is
in the wrong here. sr.ht populates the Sender header and uses the From
header for the user sending the email or initiating the action from the
web.
W dniu 14.09.2018 o 13:51, Drew DeVault pisze:
> On 2018-09-14 1:30 AM, Joe Nelson wrote:>> I think the problem is that sr.ht sets the From field as the original>> sender rather than munging it, and protonmail says to quarantine such>> impersonation with "v=DMARC1; p=quarantine; fo=1;"> > This is an important feature of email, I'd suggest your mail server is> in the wrong here. sr.ht populates the Sender header and uses the From> header for the user sending the email or initiating the action from the> web.>
It's more like DMARC the standard [0] is wrong, rather than a particular
server. They effectively force you to be non-compliant with RFC 5322 [1]
Wikipedia has a nice summary [2].
[0]: https://tools.ietf.org/html/rfc7489#section-3.1
[1]: https://tools.ietf.org/html/rfc5322#section-3.6.2
[2]: https://en.wikipedia.org/wiki/DMARC#Sender_field
> I'd suggest your mail server is in the wrong here. sr.ht populates the Sender header and uses the From header for the user sending the email or initiating the action from the> web.
I think my mail server (Fastmail) is acting correctly. It's honoring the SPF+DMARC preferences published by Protonmail, and your relay is sending messages that are out of alignment. Either way, I suspect that mailservers other than Fastmail will junk the messages too.
You might take inspiration from other mailing list software such as Mailman. It gives the option to set the From field as the server itself, and put the original author in the Reply-To field. This satisfies DMARC while allowing replies to still work. https://wiki.list.org/DEV/DMARC#Mailman_3
On 2018-09-14 12:32 PM, Joe Nelson wrote:
> You might take inspiration from other mailing list software such as> Mailman. It gives the option to set the From field as the server> itself, and put the original author in the Reply-To field. This> satisfies DMARC while allowing replies to still work.> https://wiki.list.org/DEV/DMARC#Mailman_3
It gives the option because there are broken mail servers out there. I
don't tolerate broken mail severs, report the issue to Fastmail.
> It gives the option because there are broken mail servers out there. I> don't tolerate broken mail severs, report the issue to Fastmail.
Let's review the facts together to determine whether my understanding is correct.
First, which servers does Protonmail mandate may send email marked in the From header as originating in their domain?
$ dig protonmail.com TXT +short | grep spf
"v=spf1 include:_spf.protonmail.ch ~all"
Well, sr.ht is not listed as one of those servers, and the "~all" indicates the failure will be considered a soft rather than hard failure.
How then should DMARC compatible mail servers react to this failure?
$ dig _dmarc.protonmail.com TXT +short
"v=DMARC1; p=quarantine; fo=1;"
p=quarantine
Advises the receiving MTA to treat any email that fails any DKIM and/or SPF checks as suspicious and perform additional checks or mark the mail as suspected SPAM or whatever local policy is in operation.
fo=1
Generate report to the sending MTA if any underlying check failed.
Thus it appears that Fastmail is
a) encountering a message whose From header generates a DMARC failure
b) following the instructions to quarantine the message, moving it to the Spam folder
Am I missing something in my interpretation of the standard? Perhaps Fastmail is being too restrictive because of the "~all" (rather than "-all") specification in the DMARC? I don't see an obvious place where Fastmail is "broken" here.
My interpretation is that DMARC is a bunk standard that no one should
implement to the letter. Implementing DMARC correctly requires
implementing other standards incorrectly.
> My interpretation is that DMARC is a bunk standard that no one should> implement to the letter. Implementing DMARC correctly requires> implementing other standards incorrectly.
Can you recommend the appropriate way for mail servers to handle the current SPF/DMARC problem? Should they treat your mail as an SPF success, despite your server not appearing in the list of allowed senders, or should they disregard the directive to quarantine the message?
Perhaps a better way to prevent email impersonation would be to sign messages with PGP rather than relying on SPF+DMARC. (In fact the latter does nothing to prevent employees of a shared mail host from spoofing an email from one of their subscribers...) But this is a digression.
Anyway, I guess the point of this email thread is to alert you that some of your messages are going to the junk folder on my server, and possibly other people's servers too. I don't particularly feel like arguing too much about it, just wanted you to know so you can take the action you feel is appropriate.
On 2018-09-14 1:00 PM, Joe Nelson wrote:
> Can you recommend the appropriate way for mail servers to handle the> current SPF/DMARC problem? Should they treat your mail as an SPF> success, despite your server not appearing in the list of allowed> senders, or should they disregard the directive to quarantine the> message?
I think that DMARC should be fixed or abandoned entirely. DKIM and SPF
should be sufficient.
> Anyway, I guess the point of this email thread is to alert you that> some of your messages are going to the junk folder on my server, and> possibly other people's servers too. I don't particularly feel like> arguing too much about it, just wanted you to know so you can take the> action you feel is appropriate.
Thanks for the tip. I think you should write to your mail provider about
this.
W dniu 14.09.2018 o 19:46, Joe Nelson pisze:
> First, which servers does Protonmail mandate may send email marked in the From header as originating in their domain?> > $ dig protonmail.com TXT +short | grep spf> "v=spf1 include:_spf.protonmail.ch ~all"
AFAIK SPF doesn't say anything about the From header, it only talks about
MAIL FROM (the address provided during the SMTP transaction, as part of an envelope)
and HELO (the name the sending server calls itself).[0]
[0]: https://tools.ietf.org/html/rfc7208#section-4.1
On Fri, Sep 14, 2018, at 13:01, Drew DeVault wrote:
> I think that DMARC should be fixed or abandoned entirely. DKIM and SPF> should be sufficient.
This is true, and it's a lovely idea, but in the real world DMARC and friends are here to stay and your list is broken for a number of your users and operators (including me). Standing on principle won't change that.
Violating RFC 5322 appears to be what a lot of mailing list software is doing these days, and isn't the end of the world (clients et al. generally still behave correctly if you munge the from field). It's an unfortunate reality that we have to live in. If it helps, think of it as superseding some of the rules in 5322 instead of violating it.
—Sam
On 2018-09-14 1:53 PM, Sam Whited wrote:
> This is true, and it's a lovely idea, but in the real world DMARC and> friends are here to stay and your list is broken for a number of your> users and operators (including me). Standing on principle won't change> that.
I don't want to budge on this. Your mail server is broken, not mine.
I've never been someone to bend my software to accomodate for other
people's bad software. I've been the one to email them and help them fix
their software. This isn't some immutable property of email, mail
servers are run by humans and they can fix their configurations.
On Fri, Sep 14, 2018, at 13:55, Drew DeVault wrote:
> I don't want to budge on this. Your mail server is broken, not mine.> I've never been someone to bend my software to accomodate for other> people's bad software. I've been the one to email them and help them fix> their software. This isn't some immutable property of email, mail> servers are run by humans and they can fix their configurations.
This isn't a configuration problem (unless you consider protonmail and possibly others as setting too strict a policy). This is just how modern email servers behave. It's unfortunate, but every single email server in existence isn't going to stop supporting dmarc. Meanwhile, sad as it is, your list software could behave like every other list software and fix the problem for a all your users.
You're asking almost every email server to change to accommodate a single piece of mailing list software and as much as I wish they would, it's never going to happen. In the mean time, please consider your users sanity and make a concession. That's the sad reality about how software development works.
—Sam
On 2018-09-14 2:07 PM, Sam Whited wrote:
> This isn't a configuration problem (unless you consider protonmail and> possibly others as setting too strict a policy).
I do.
> This is just how modern email servers behave.
No, this is how a few email servers behave. Most people don't have
issues receiving emails from sr.ht.
> It's unfortunate, but every single email server in existence isn't> going to stop supporting dmarc.
Again, it's just a couple of mail servers. I'd rather have email
delivery issues which cause users to complain to their bad mail
providers, leading to gradual change over the course of a year or two,
than cater to bad mail servers.
Don't complain to me. My software is correct. Complain to your mail
provider, whose software is incorrect.
> Meanwhile, sad as it is, your list software could behave like every> other list software and fix the problem for a all your users.
Many other lists face the same problem. Most of them, in fact. I can't
think of any mailing lists which use the mailman option you mentioned.
> You're asking almost every email server to change
No, I'm not. I'm asking a small few servers which are objectively broken.
> it's never going to happen.
I'm not so pessimistic. Until you try you cannot know.
> That's the sad reality about how software development works.
The sad reality is that many software developers like you are needlessly
defeatist. I am not, and the results speak for themselves. I've seen
problems you would have laid over and given up on fixed with persistence
and patience, and that's how I intend to fix this problem too.
On Friday, September 14, 2018 8:00 PM, Joe Nelson <joe@begriffs.com> wrote:
> Perhaps a better way to prevent email impersonation would be to sign messages with PGP rather than relying on SPF+DMARC. (In fact the latter does nothing to prevent employees of a shared mail host from spoofing an email from one of their subscribers...) But this is a digression.
As said in later replies, DKIM has been designed exactly for this
purpose. It's a signature added by the sending MTA that remains after
mail forwarding by mailing list software.
>> Perhaps a better way to prevent email impersonation would be to sign messages with PGP rather than relying on SPF+DMARC.> > As said in later replies, DKIM has been designed exactly for this> purpose. It's a signature added by the sending MTA that remains after> mail forwarding by mailing list software.
So if DKIM is present and correctly signs the message, then you can be assured that the message originated from a legitimate sending server and that the message wasn't modified during forwarding. However couldn't someone still spoof another person by just not including DKIM in the message (or could a forwarding server strip the DKIM and then make other modifications)?
Seems to me it's the DMARC where you specify preferences like, "Don't trust any message claiming to be from my domain if the message lacks DKIM" -- aka the "adkim=r" property. Without the DMARC declaration there's no automated signal that the person who is emailing you has adopted DKIM to protect their messages.
Is this correct?
On 2018-09-14 5:51 PM, Joe Nelson wrote:
> So if DKIM is present and correctly signs the message, then you can be> assured that the message originated from a legitimate sending server> and that the message wasn't modified during forwarding. However> couldn't someone still spoof another person by just not including DKIM> in the message (or could a forwarding server strip the DKIM and then> make other modifications)?>> Seems to me it's the DMARC where you specify preferences like, "Don't> trust any message claiming to be from my domain if the message lacks> DKIM" -- aka the "adkim=r" property. Without the DMARC declaration> there's no automated signal that the person who is emailing you has> adopted DKIM to protect their messages.
Aye, this would be a good solution. It seems reasonable to me to apply a
subset of DMARC which requires DKIM but doesn't break when my mail
server forwards the message.
> Thanks for the tip. I think you should write to your mail provider about> this.
Sure, happy to pass along a message to them once I understand the issue. In general I'm with you that developers should push back against bad practices and help other parts of the world improve, rather than degrade their own software. Can you explain what in particular Fastmail is doing wrong in this instance? I don't know how to describe the problem to them other than, "Hey your server quarantined a message due to SPF failure and the originating domain's DMARC policy."
On 2018-09-14 6:00 PM, Joe Nelson wrote:
> > Thanks for the tip. I think you should write to your mail provider about> > this.> > Can you explain what in particular Fastmail is doing wrong in this> instance? I don't know how to describe the problem to them other than,> "Hey your server quarantined a message due to SPF failure and the> originating domain's DMARC policy."
Feel free to copy me on the message to clarify anything, and maybe Simon
as well (he knows a lot about email). I'd say that messages forwarded by
a mailing list will often preserve the From header and use the Sender
header to indicate that the message is forwarded by another mail server,
per RFC 5322, but this breaks too-strict DMARC setups. Suggest that if
the DKIM signature is valid, they disregard the DMARC failure.
I'll also reach out to Protonmail and suggest they fix their DMARC
record to require DKIM but not fail messages forwarded by the mailing
list.
--
Drew DeVault
Note, I also filed a ticket to automate testing people's mail servers
and to let them know when their configuration is too strict:
https://todo.sr.ht/~sircmpwn/lists.sr.ht/50
> Suggest that if the DKIM signature is valid, they disregard the DMARC> failure.> > I'll also reach out to Protonmail and suggest they fix their DMARC> record to require DKIM but not fail messages forwarded by the mailing> list.
This has been a useful conversation, thanks. Here's my summary:
* Your original contention that "DMARC the standard is wrong" is not true, only that certain DMARC configurations do not work with mailing lists.
* In particular, it's SPF that breaks things.
* DKIM (along with the adkim=r setting in DMARC) can do everything SPF can, and does not break mailing lists.
* When DKIM is present and validates properly, then it is safe for a mail server such as Fastmail to ignore an SPF failure. There is no way that the message is a forgery.
The solution is a combination of two things
1) senders should update their DMARC, removing SPF and enforcing DKIM
2) mail servers should treat successful DKIM as good enough
Both of these items require participation by potentially big players, so the problem may affect some of your users for a long time to come. Probably something that should be documented.
I'll update settings for my own domain and write to Fastmail about their end of things, but I'll let this message sit on the list for a little while first to see if other people have any thoughts.
On 2018-09-14 6:46 PM, Joe Nelson wrote:
> * Your original contention that "DMARC the standard is wrong" is not> true, only that certain DMARC configurations do not work with mailing> lists.
I disagree, I think DMARC is wrong, and not of any use now that DKIM is
popular.
> * In particular, it's SPF that breaks things.
???
> * DKIM (along with the adkim=r setting in DMARC) can do everything SPF> can, and does not break mailing lists.
Not really, adkim=r makes DMARC more liberal but doesn't fix the problem
entirely. It's best to set p=none or just drop DMARC entirely.
> * When DKIM is present and validates properly, then it is safe for a> mail server such as Fastmail to ignore an SPF failure. There is no way> that the message is a forgery.
Yes.
> The solution is a combination of two things> 1) senders should update their DMARC, removing SPF and enforcing DKIM> 2) mail servers should treat successful DKIM as good enough
Yes, though I would extend #1 with "update OR remove DMARC"
> Both of these items require participation by potentially big players,> so the problem may affect some of your users for a long time to come.> Probably something that should be documented.
The biggest players do this right, it's just smaller ones like
Protonmail that do it wrong. That's what the ticket I filed is for, for
letting users know to reach out to their postmasters.
> I'll update settings for my own domain and write to Fastmail about> their end of things, but I'll let this message sit on the list for a> little while first to see if other people have any thoughts.
Cheers.
> I disagree, I think DMARC is wrong, and not of any use now that DKIM is> popular.
Hmm, my understanding is that DMARC says how to handle DKIM/SPF failures, as well as whether a source domain promises that its outgoing messages will include DKIM. So if a domain doesn't set any DMARC preferences, then someone could spoof an email from that domain by simply omitting a DKIM signature in the message. No DKIM = no signing failure. The destination mail server will accept the message since there is nothing anywhere to tell it that the source domain promises outgoing messages will be signed.
On 2018-09-14 9:00 PM, Joe Nelson wrote:
> Hmm, my understanding is that DMARC says how to handle DKIM/SPF> failures, as well as whether a source domain promises that its> outgoing messages will include DKIM. So if a domain doesn't set any> DMARC preferences, then someone could spoof an email from that domain> by simply omitting a DKIM signature in the message. No DKIM = no> signing failure. The destination mail server will accept the message> since there is nothing anywhere to tell it that the source domain> promises outgoing messages will be signed.
Okay, fair. But because DMARC is a crappy spec which doesn't handle
Sender correctly, I'd only process it _if_ DKIM is missing. If DKIM is
there and valid you should stop and take the email as legitimate. The
bug is in software which continues to the DMARC check anyway.
--
Drew DeVault
> I'd only process it _if_ DKIM is missing. If DKIM is> there and valid you should stop and take the email as legitimate. The> bug is in software which continues to the DMARC check anyway.
Sounds perfectly reasonable.
Sr.ht users will still have to deal with messages hitting the spam folder for a certain amount of time before servers get adjusted, but at least we know what those adjustments should be.
Continuing to do some research and found this article:
https://blog.returnpath.com/how-to-explain-dkim-in-plain-english-2/
From the article:
> The problem with DKIM is that because it’s more difficult to implement, fewer senders have adopted it. This spotty adoption means that the absence of a DKIM signature does not necessarily indicate the email is fraudulent. Therefore, DKIM alone is not a universally reliable way of authenticating the identity of a sender. In addition, the DKIM domain is not visible to the non-technical end user, and does nothing to prevent the spoofing of the visible "header from" domain.> DMARC [...] addresses that problem, by guaranteeing that the domain visible to the end user is the same as the domains validated by the SPF and DKIM checks. In addition, it provides mailbox providers with clear instruction about which emails they should hold to the DKIM authentication standard and which they should not.
Thus we were not entirely right in saying, "If DKIM passes then no further checks are necessary." Someone can impersonate another user (even signing the message with the DKIM for their own domain if they want to) by forging the "From" header to match someone else. The "mail from" in the SMTP envelope will show the truth, but apparently mail clients don't make that visible, showing the forgeable From header instead. Doing a DMARC check will scan to see that From matches the true "mail from."
Fun story - I got fooled once by a scammer who wrote to my work address from a random gmail account but set the display name to that of my boss, like "Boss Name <fakeuser123@gmail.com>". They were like, "Hey are you at the office?" My stupid Mac mail client didn't show the email address by default -- just the name -- so it looked fine. The person started saying weird things after the first exchange so I got suspicious, dug into the email, and discovered the situation. DMARC couldn't help me with that one, but it would at least prevent the scammer from impersonating the email address as well as the display name. A lot of scamming relies on user interface flaws plus basic social engineering.
W dniu 15.09.2018 o 04:00, Joe Nelson pisze:
> Hmm, my understanding is that DMARC says how to handle DKIM/SPF failures, as well as whether a source domain promises that its outgoing messages will include DKIM. So if a domain doesn't set any DMARC preferences, then someone could spoof an email from that domain by simply omitting a DKIM signature in the message. No DKIM = no signing failure. The destination mail server will accept the message since there is nothing anywhere to tell it that the source domain promises outgoing messages will be signed.>
Aside from above, there's one more thing that DMARC does.
There are multiple kinds of "from" addresses in an email.
One is the address provided by one SMTP server to the other
as part of the SMPT "MAIL FROM" command, also known as
"Envelope From", "Return Path", or "RFC5321.MailFrom".
This is the only address mail servers care about
in order to figure out how to deliver your email.
This is what SPF cares about.
The other one is the "From" header inside the email,
also known as RFC5322.From.
It is meant for humans and email clients.
It has semantics of "Address of the _author_ of the message"
This is what your email client displays.
SPF doesn't care about it.
DKIM signs it, but doesn't compare its content to anything.
DMARC says that the RFC5322.From header in the email content
must match the RFC5321.MailFrom
and must match the domain specified in DKIM signature as the signer.
And this is the part of DMARC that is wrong.
It should not look at RFC5322.From.
On 2018-09-15 12:55 AM, Joe Nelson wrote:
> Thus we were not entirely right in saying, "If DKIM passes then no> further checks are necessary." Someone can impersonate another user> (even signing the message with the DKIM for their own domain if they> want to) by forging the "From" header to match someone else. The "mail> from" in the SMTP envelope will show the truth, but apparently mail> clients don't make that visible, showing the forgeable From header> instead. Doing a DMARC check will scan to see that From matches the> true "mail from."
This is a bug in the mail clients that don't show this correctly.
> DMARC says that the RFC5322.From header in the email content> must match the RFC5321.MailFrom> and must match the domain specified in DKIM signature as the signer.> > And this is the part of DMARC that is wrong.> It should not look at RFC5322.From.
This appears to mean that (even disregarding SPF) DMARC is incompatible with mailing lists that don't munge RFC5322.From. I found a multi-part article about the topic that is pretty good: https://blogs.msdn.microsoft.com/tzink/2015/05/28/solving-the-problem-of-dmarcs-incompatibility-with-mailing-lists-part-1/
I guess at some level all this makes sense because when a mailing list sends out a message "as you," it is in fact a forgery -- a trusted forgery. Sr.ht isn't *my* server, and I don't want it to be able to just send mails that appear to be mine. In general when it's playing by the rules and simply forwarding my actual messages to other subscribers it's cool, but we must remember there is an element of trust here. It *could* send anything as me if it wanted to. DMARC prevents this.
Thus munging RFC5322.From is actually the honest thing to do:
From: John Doe <jdoe@foo.com> -> From: John Doe via sr.ht <~sircmpwn/sr.ht-discuss@lists.sr.ht>
This makes it perfectly clear that the mailing list sent the message, and a Reply-To will guide my mail client in how to respond.
Rather than fixating on how "DMARC is wrong," start instead from the hypothesis that emails can no longer be forged, and when you take that hypothesis seriously it shows that the classic mailing list behavior is no longer possible.
Perhaps the rebuttal is that a competent mail client should make the return-path visible to the user, so no vigilant user would be fooled about the true source. Which clients do this? Does mutt even do this? Maybe if you press H to view the headers.
On 2018-09-16 10:09 AM, Joe Nelson wrote:
> I guess at some level all this makes sense because when a mailing list> sends out a message "as you," it is in fact a forgery -- a trusted> forgery. Sr.ht isn't *my* server, and I don't want it to be able to> just send mails that appear to be mine. In general when it's playing> by the rules and simply forwarding my actual messages to other> subscribers it's cool, but we must remember there is an element of> trust here. It *could* send anything as me if it wanted to. DMARC> prevents this.
This isn't true, the email is *from* you. This is what the Sender header
is for. The spec is very clear on this, DMARC is just bullshit.
W dniu 16.09.2018 o 17:09, Joe Nelson pisze:
> Rather than fixating on how "DMARC is wrong," start instead from the hypothesis that emails can no longer be forged, and when you take that hypothesis seriously it shows that the classic mailing list behavior is no longer possible.
Ok, let's imagine every email is cryprographically signed by the author,
and therefore cannot be forged.
Then, a mailing list is still possible. The list server:
- takes a signed email it receives
- prepends some headers like List-ID and Sender at the top,
before the part that is covered by a signature,
and leaves the From header intact
- sends the resulting email to everyone subscribed
Then the receiving end:
- successfully verifies the signature
- verifies that the headers that are meant to be set only by the author
(eg. From, Subject) are covered by the signature
- shows the email to the user
Sounds possible to me.
> This isn't true, the email is *from* you. This is what the Sender header> is for. The spec is very clear on this, DMARC is just bullshit.
Good point, the RFC does use the example of a boss and secretary, where the email is From the boss, but the Sender is the secretary. So the mailing list is playing the role of secretary in this case.
But tell me, if your server can send mail From me, what's to stop it from omitting the Sender header and simply writing to my friends and impersonating me? The internet used to be a cordial place for universities and research labs, and now it's an angry spam factory. The original RFCs were not designed for this scenario.
On Sun, Sep 16, 2018, at 10:11, Drew DeVault wrote:
> This isn't true, the email is *from* you. This is what the Sender header> is for. The spec is very clear on this, DMARC is just bullshit.
This is correct, but in the mean time while we argue about whose fault it is, our users are seeing lots of emails going to spam from several different providers and our sysadmin is threatening to switch us back to whatever god awful list software we were using before. It was terrible, but it had a switch to appease dmarc and that matters more, I suspect, to most people than being principled about strictly following a standard or whose fault it actually is (regardless of whose fault it is, your customers are the ones feeling the pain which makes it your problem). If there is a way to do this with your software, please advise (I really hated whatever crappy mailman ripoff we were using before).
—Sam
On 2018-09-16 10:22 AM, Sam Whited wrote:
> This is correct, but in the mean time while we argue about whose fault> it is, our users are seeing lots of emails going to spam from several> different providers and our sysadmin is threatening to switch us back> to whatever god awful list software we were using before.
Your sysadmin should be fixing your DMARC settings. Why are you
complaining to me about things that are broken on your end? If DKIM
passes then DMARC should be ignored. Pass this along to your sysadmin
and have them fix their server.
> It was terrible, but it had a switch to appease dmarc and that matters> more, I suspect, to most people than being principled about strictly> following a standard or whose fault it actually is (regardless of> whose fault it is, your customers are the ones feeling the pain which> makes it your problem).
sr.ht is in alpha. If we find out by the stable release that getting
mail servers to play ball isn't feasible, then I will put in a
workaround. Until then, I'm going to take the principled stance. I
appreciate your feedback, but you should be prepared to deal with the
consequences of using alpha-quality software.
On 2018-09-16 10:22 AM, Joe Nelson wrote:
> But tell me, if your server can send mail From me, what's to stop it> from omitting the Sender header and simply writing to my friends and> impersonating me? The internet used to be a cordial place for> universities and research labs, and now it's an angry spam factory.> The original RFCs were not designed for this scenario.
The DKIM signature.
On Sun, Sep 16, 2018, at 10:30, Drew DeVault wrote:
> sr.ht is in alpha. If we find out by the stable release that getting> mail servers to play ball isn't feasible, then I will put in a> workaround. Until then, I'm going to take the principled stance. I> appreciate your feedback, but you should be prepared to deal with the> consequences of using alpha-quality software.
Thank you, that seems totally fair. I appreciate your consideration, and thanks for all the hard work you've put into this!
—Sam
> Ok, let's imagine every email is cryprographically signed by the author,> and therefore cannot be forged.> > Then, a mailing list is still possible. The list server:> - takes a signed email it receives> - prepends some headers like List-ID and Sender at the top,> before the part that is covered by a signature,> and leaves the From header intact> - sends the resulting email to everyone subscribed
Nice point about List-ID. Many lists modify the subject line, prepending [foo] as identification, and this subject change violates DKIM if the subject is covered in the signature. Similarly some lists append unsubscribe links to the body of an email, but setting a List-Unsubscribe header instead can allow the body to remain unchanged.
> Then the receiving end:> ...> Sounds possible to me.
I see what you mean. If there's a valid DKIM signature belonging to the RFC5322.From domain then all else should be forgiven, the email is legit. By setting the right headers the mailing list can leave the Subject, Body etc unchanged and can forward a message with DKIM intact.
I'm fully satisfied in my understanding now, and see why DMARC's requirement that RFC5321.MailFrom == RFC5322.From is overzealous and broken.
> I'm fully satisfied in my understanding now, and see why DMARC's requirement that RFC5321.MailFrom == RFC5322.From is overzealous and broken.
The plot thickens! It turns out that DMARC checks alignment in two different ways:
1. For SPF, it checks that RFC5321.MailFrom == RFC5322.From
2. For DKIM, it checks that the message has one valid DKIM signature with "d=RFC5322.From".
(nice illustration: https://dmarcian.com/wp-content/uploads/2018/03/Header-Aligned.png)
Secondly, (and importantly!) DMARC will pass the message if *either* SPF or DKIM passes, and only fail the message if both SPF and DKIM fail. That way SPF-only and and DKIM-only messages can pass DMARC, but messages without either SPF/DKIM will always fail.
Thus I hypothesize that any domain which signs with DKIM will work fine on sr.ht even with DMARC enabled. As long as sr.ht does not modify the message in a way that breaks DKIM then it can continue to use proper From:, Sender: etc headers.
So what made that previous message I was talking about go to my spam folder? (For reference, it was message-id cmu-lmtpd-2726116-1536590260-0@sloti2d1t21) I think it's because it was not signed by DKIM. Thus it had to fall back to an SPF check and that one failed alignment.
Return-Path: <extsmtp@sr.ht>
From: laumann <laumann@protonmail.com>
On Monday, September 17, 2018 6:50 AM, Joe Nelson <joe@begriffs.com> wrote:
> So what made that previous message I was talking about go to my spam folder?> (For reference, it was message-id cmu-lmtpd-2726116-1536590260-0@sloti2d1t21)> I think it's because it was not signed by DKIM. Thus it had to fall back to an> SPF check and that one failed alignment.>> Return-Path: <extsmtp@sr.ht>> From: laumann <laumann@protonmail.com>
This seems unlikely, ProtonMail always signs outgoing messages with DKIM. I
can't find the message you're referring to, can you point me to the lists.sr.ht
link?
>> I think it's because it was not signed by DKIM. Thus it had to fall back to an>> SPF check and that one failed alignment.> > This seems unlikely, ProtonMail always signs outgoing messages with DKIM. I> can't find the message you're referring to, can you point me to the lists.sr.ht> link?
Aha, do you know what it was? The message was generated from https://todo.sr.ht/~sircmpwn/lists.sr.ht/49#comment-346
I'm guessing the message was created through the web interface, so there was no DKIM signature to relay.
On Monday, September 17, 2018 5:57 PM, Joe Nelson <joe@begriffs.com> wrote:
> Aha, do you know what it was? The message was generated from> https://todo.sr.ht/~sircmpwn/lists.sr.ht/49#comment-346>> I'm guessing the message was created through the web interface, so there was> no DKIM signature to relay.
Ah, that makes sense. The message wasn't _relayed_ by mail.sr.ht, it has been
_sent_ by mail.sr.ht (and generated by todo.sr.ht).
I've already said sending emails on behalf of users is a bad idea. The From
address is the real user's email address, and mail.sr.ht is not authorized to
send emails from this address. It's a good thing these emails are classified
as spam, because todo.sr.ht "impersonates" you.
Instead, todo.sr.ht should send emails with an address it owns.
On 2018-09-17 4:30 PM, Simon Ser wrote:
> On Monday, September 17, 2018 5:57 PM, Joe Nelson <joe@begriffs.com> wrote:> > Aha, do you know what it was? The message was generated from> > https://todo.sr.ht/~sircmpwn/lists.sr.ht/49#comment-346> >> > I'm guessing the message was created through the web interface, so there was> > no DKIM signature to relay.
Okay, this makes sense. I'm willing to change how emails from todo.sr.ht
work.
