Hello all,
For the past year, I've been self hosting email (using postfix + dovecot
on a Vultr VPS) - both as an educational exercise and a chance to get
away from Google - which is tied to my domain, coderkalyan.com. However,
when I first set it up, I was unable to get GMail to not put me in spam,
so I decided to go with Amazon SES as an SMTP proxy so I could rest
assured my email was being delivered.
Recently, I came across the [maddy](https://maddy.email) email client
and have been using it for the past couple of weeks. I decided that
since I had to rip out my existing DNS configuration and redo it, I'd
take the opportunity to try to remove AWS from my workflow However,
despite multiple days of googling and trying, I'm unable to get gmail to
accept my messages into primary inbox. Even with all the usual anti-spam
measures implemented, they put me in spam (which is slightly better than
not receiving it at all, which is what they do if you don't have
everything set up). What I've tried so far:
* DKIM and DMARC set up
* SPF records
* configured reverse DNS through Vultr and it seems to be working
* google site verification TXT record
* TLSA/DANE record
* MTA-STS record and subdomain setup (mta-sts.coderkalyan.com)
* 10/10 score on https://mail-tester.com
* high scores on other mail testing platforms, in fact the only issue
I've seen is that I don't fill out a lists unsubscribe header (which
doesn't seem sensible for a private email server).
* I've tried to use Google Postmaster tools and other reputation sites
but apparently postmaster doesn't even bother to look at your domain
until you meet a certain email throughput, which is designed for
businesses sending to large mailing lists so I'll never meet that.
I suspect I'm not the only person on this mailing list who self-hosts,
so I thought I'd reach out to see if anyone could help me out. Correct
me if I'm wrong, not sure how exactly mailing lists are implemented, but
I assume lists.sr.ht itself needs to pass some sort of spam checks to
get delivered to GMail inboxes (and for the most part I haven't seen
issues with that in the past).
Any advice would be much appreciated. Please do not suggest that I move
to a hosted provider or use an SMTP proxy. I realize that these exist
(even ethical ones like Migadu) and do not need others to tell me that.
I will move back to an SMTP proxy if all else fails, but I'm interested
in solving this problem directly.
Thanks,
Kalyan Sriram
On Tue Dec 21, 2021 at 6:54 PM EST, Kalyan Sriram wrote:
> Hello all,> For the past year, I've been self hosting email (using postfix + dovecot> on a Vultr VPS) - both as an educational exercise and a chance to get> away from Google - which is tied to my domain, coderkalyan.com. However,> when I first set it up, I was unable to get GMail to not put me in spam,> so I decided to go with Amazon SES as an SMTP proxy so I could rest> assured my email was being delivered.>> Recently, I came across the [maddy](https://maddy.email) email client> and have been using it for the past couple of weeks. I decided that> since I had to rip out my existing DNS configuration and redo it, I'd> take the opportunity to try to remove AWS from my workflow However,> despite multiple days of googling and trying, I'm unable to get gmail to> accept my messages into primary inbox. Even with all the usual anti-spam> measures implemented, they put me in spam (which is slightly better than> not receiving it at all, which is what they do if you don't have> everything set up). What I've tried so far:>> * DKIM and DMARC set up> * SPF records> * configured reverse DNS through Vultr and it seems to be working> * google site verification TXT record> * TLSA/DANE record> * MTA-STS record and subdomain setup (mta-sts.coderkalyan.com)> * 10/10 score on https://mail-tester.com> * high scores on other mail testing platforms, in fact the only issue> I've seen is that I don't fill out a lists unsubscribe header (which> doesn't seem sensible for a private email server).> * I've tried to use Google Postmaster tools and other reputation sites> but apparently postmaster doesn't even bother to look at your domain> until you meet a certain email throughput, which is designed for> businesses sending to large mailing lists so I'll never meet that.>> I suspect I'm not the only person on this mailing list who self-hosts,> so I thought I'd reach out to see if anyone could help me out. Correct> me if I'm wrong, not sure how exactly mailing lists are implemented, but> I assume lists.sr.ht itself needs to pass some sort of spam checks to> get delivered to GMail inboxes (and for the most part I haven't seen> issues with that in the past).>> Any advice would be much appreciated. Please do not suggest that I move> to a hosted provider or use an SMTP proxy. I realize that these exist> (even ethical ones like Migadu) and do not need others to tell me that.> I will move back to an SMTP proxy if all else fails, but I'm interested> in solving this problem directly.>> Thanks,> Kalyan Sriram
Hi Kalyan,
I've self-hosted my email on my Vultr VPS for several months now with no
issue. I set it up with this script[0] which you may find useful.
-- Sebastian
[0]: https://github.com/LukeSmithxyz/emailwiz
(Sorry for double posting, forgot to reply to list)
Hey,
Just chiming in, my email client reported that your message's DKIM
failed. Maybe that is causing it to be flagged?
Regards,
I've had much better overall delivery using Hetzner servers than
Vultr. Vultr regularly gets listed on UCEProtect Level 2 and 3.
Hetzner is very attentive to abuse reports and has managed to keep
their reputation reasonably stable (though by no means pristine).
Daniel
On Wed, 22 Dec 2021 at 00:18, Sebastian LaVine <mail@smlavine.com> wrote:
>> On Tue Dec 21, 2021 at 6:54 PM EST, Kalyan Sriram wrote:> > Hello all,> > For the past year, I've been self hosting email (using postfix + dovecot> > on a Vultr VPS) - both as an educational exercise and a chance to get> > away from Google - which is tied to my domain, coderkalyan.com. However,> > when I first set it up, I was unable to get GMail to not put me in spam,> > so I decided to go with Amazon SES as an SMTP proxy so I could rest> > assured my email was being delivered.> >> > Recently, I came across the [maddy](https://maddy.email) email client> > and have been using it for the past couple of weeks. I decided that> > since I had to rip out my existing DNS configuration and redo it, I'd> > take the opportunity to try to remove AWS from my workflow However,> > despite multiple days of googling and trying, I'm unable to get gmail to> > accept my messages into primary inbox. Even with all the usual anti-spam> > measures implemented, they put me in spam (which is slightly better than> > not receiving it at all, which is what they do if you don't have> > everything set up). What I've tried so far:> >> > * DKIM and DMARC set up> > * SPF records> > * configured reverse DNS through Vultr and it seems to be working> > * google site verification TXT record> > * TLSA/DANE record> > * MTA-STS record and subdomain setup (mta-sts.coderkalyan.com)> > * 10/10 score on https://mail-tester.com> > * high scores on other mail testing platforms, in fact the only issue> > I've seen is that I don't fill out a lists unsubscribe header (which> > doesn't seem sensible for a private email server).> > * I've tried to use Google Postmaster tools and other reputation sites> > but apparently postmaster doesn't even bother to look at your domain> > until you meet a certain email throughput, which is designed for> > businesses sending to large mailing lists so I'll never meet that.> >> > I suspect I'm not the only person on this mailing list who self-hosts,> > so I thought I'd reach out to see if anyone could help me out. Correct> > me if I'm wrong, not sure how exactly mailing lists are implemented, but> > I assume lists.sr.ht itself needs to pass some sort of spam checks to> > get delivered to GMail inboxes (and for the most part I haven't seen> > issues with that in the past).> >> > Any advice would be much appreciated. Please do not suggest that I move> > to a hosted provider or use an SMTP proxy. I realize that these exist> > (even ethical ones like Migadu) and do not need others to tell me that.> > I will move back to an SMTP proxy if all else fails, but I'm interested> > in solving this problem directly.> >> > Thanks,> > Kalyan Sriram>> Hi Kalyan,>> I've self-hosted my email on my Vultr VPS for several months now with no> issue. I set it up with this script[0] which you may find useful.>> -- Sebastian>> [0]: https://github.com/LukeSmithxyz/emailwiz
On Tue Dec 21, 2021 at 4:15 PM PST, Noelle Leigh wrote:
> Interestingly, your message got sent to my spam folder (Fastmail).
Weird. Fastmail appears to use SpamAssassin, which is tested by
mail-tester.com and other sites and I haven't had any issues from it
before.
>> Here's the X-Spam-* headers that it contained:>> X-Spam-known-sender: no> X-Spam-sender-reputation: 0 (email; expectedauth)> X-Spam-score: 8.2> X-Spam-hits: DMARC_LIST_OVERRIDE_QUARANTINE 0.5,> HEADER_FROM_DIFFERENT_DOMAINS 0.25,> MAILING_LIST_MULTI -1, ME_HAS_VSSU 0.001, ME_SC_NH -0.001,> ME_SENDERREP_DENY 4, ME_VADESPAM_MED_NB 4.5, SPF_HELO_NONE 0.001,> SPF_PASS -0.001, LANGUAGES en, BAYES_USED none, SA_VERSION 3.4.2> X-Spam-source: IP='173.195.146.151', Host='mail-b.sr.ht', Country='US',> FromHeader='com',> MailFrom='ht'> X-Spam-charsets: plain='UTF-8'>> I don't know if that's helpful ¯\_(ツ)_/¯
Thanks a lot for the reply. I googled "ME_SENDERREP_DENY" and got this
documentation:
https://www.fastmail.help/hc/en-us/articles/360060591413-Spam-filtering
It states that this score is given based on past interaction with the
sender (I assume that refers to me). Not sure why this would causing
issues, any ideas? Does it just dislike new email addresses from unknown
domains? Also, so far I can't figure out the ME_VADESPAM_MED_NB means.
Kalyan
On Tue Dec 21, 2021 at 4:17 PM PST, Sebastian LaVine wrote:
> On Tue Dec 21, 2021 at 6:54 PM EST, Kalyan Sriram wrote:> > Hello all,> > For the past year, I've been self hosting email (using postfix + dovecot> > on a Vultr VPS) - both as an educational exercise and a chance to get> > away from Google - which is tied to my domain, coderkalyan.com. However,> > when I first set it up, I was unable to get GMail to not put me in spam,> > so I decided to go with Amazon SES as an SMTP proxy so I could rest> > assured my email was being delivered.> >> > Recently, I came across the [maddy](https://maddy.email) email client> > and have been using it for the past couple of weeks. I decided that> > since I had to rip out my existing DNS configuration and redo it, I'd> > take the opportunity to try to remove AWS from my workflow However,> > despite multiple days of googling and trying, I'm unable to get gmail to> > accept my messages into primary inbox. Even with all the usual anti-spam> > measures implemented, they put me in spam (which is slightly better than> > not receiving it at all, which is what they do if you don't have> > everything set up). What I've tried so far:> >> > * DKIM and DMARC set up> > * SPF records> > * configured reverse DNS through Vultr and it seems to be working> > * google site verification TXT record> > * TLSA/DANE record> > * MTA-STS record and subdomain setup (mta-sts.coderkalyan.com)> > * 10/10 score on https://mail-tester.com> > * high scores on other mail testing platforms, in fact the only issue> > I've seen is that I don't fill out a lists unsubscribe header (which> > doesn't seem sensible for a private email server).> > * I've tried to use Google Postmaster tools and other reputation sites> > but apparently postmaster doesn't even bother to look at your domain> > until you meet a certain email throughput, which is designed for> > businesses sending to large mailing lists so I'll never meet that.> >> > I suspect I'm not the only person on this mailing list who self-hosts,> > so I thought I'd reach out to see if anyone could help me out. Correct> > me if I'm wrong, not sure how exactly mailing lists are implemented, but> > I assume lists.sr.ht itself needs to pass some sort of spam checks to> > get delivered to GMail inboxes (and for the most part I haven't seen> > issues with that in the past).> >> > Any advice would be much appreciated. Please do not suggest that I move> > to a hosted provider or use an SMTP proxy. I realize that these exist> > (even ethical ones like Migadu) and do not need others to tell me that.> > I will move back to an SMTP proxy if all else fails, but I'm interested> > in solving this problem directly.> >> > Thanks,> > Kalyan Sriram>> Hi Kalyan,>> I've self-hosted my email on my Vultr VPS for several months now with no> issue. I set it up with this script[0] which you may find useful.>
Thanks a lot. I will look through this script to see if I forgot to
configure anything.
> -- Sebastian>> [0]: https://github.com/LukeSmithxyz/emailwiz
Migadu here also identifies you as spam:
X-Migadu-Spam-Score: 6.00
X-Spam: Yes
Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=lists.sr.ht header.s=srht header.b=cUPC2BOb; dkim=fail ("headers rsa verify failed") header.d=coderkalyan.com header.s=default header.b=e51XeYqY; dmarc=fail reason="SPF not aligned (relaxed), DKIM not aligned (relaxed)" header.from=coderkalyan.com (policy=quarantine); spf=pass (aspmx1.migadu.com: domain of lists@sr.ht designates 173.195.146.151 as permitted sender) smtp.mailfrom=lists@sr.ht
It would seem both the SPF and DKIM checks fail. In my short experience years ago, if one of them succeeds then you are not considered spam immediately. I'd try to check on those failures. It may be some slight misconfiguration.
Regards,
Miguel
On Tue Dec 21, 2021 at 10:51 PM PST, Miguel Bernabeu wrote:
> Migadu here also identifies you as spam:>> X-Migadu-Spam-Score: 6.00> X-Spam: Yes> Authentication-Results: aspmx1.migadu.com; dkim=pass> header.d=lists.sr.ht header.s=srht header.b=cUPC2BOb; dkim=fail> ("headers rsa verify failed") header.d=coderkalyan.com header.s=default> header.b=e51XeYqY; dmarc=fail reason="SPF not aligned (relaxed), DKIM> not aligned (relaxed)" header.from=coderkalyan.com (policy=quarantine);> spf=pass (aspmx1.migadu.com: domain of lists@sr.ht designates> 173.195.146.151 as permitted sender) smtp.mailfrom=lists@sr.ht>> It would seem both the SPF and DKIM checks fail. In my short experience> years ago, if one of them succeeds then you are not considered spam> immediately. I'd try to check on those failures. It may be some slight> misconfiguration.>> Regards,> Miguel
This is really strange, I haven't seen this in my other testing. Thanks
for this, I'll look into it.
Thanks so much, everyone, for the help so far. Hopefully I can get to
the bottom of this with everyone's spam reports. I seem to have
identified two problems:
1) When sending email through lists.sr.ht, my mail fails DKIM. This is
not because srht's own DKIM fails (it works) but my own DKIM fails.
However, when I am sending directly, that very same DKIM configuration
on my server passes. I have no clue yet what is causing this but will
investigate.
2) Even on "direct" emails to GMail domains, which claim that I am
passing DMARC, DKIM, and SPF, I still get put in spam.
Kalyan
Hi there,
For what's worth it, my mail server runs on Heztner for the past two years. It's postfix/dovecot/rspamd without reverse dns. Emails get delivered everywhere except gmx.de sofar, since they need a mandatory reverse dns record which I can't set unless I recreate my server with a newly ordered IP from Heztner.
Cheers,
Mehdi
> For what's worth it, my mail server runs on Heztner for the past two years.> Emails get delivered everywhere except gmx.de sofar
I'm curious, do you have fairly reliable deliverability to Outlook? I've run my
mail server with netcup.eu for ~2 years as well but it seems as if Microsoft
blocks their whole IP range and it's been a major pain point. Mail is delivered
to everyone else perfectly fine though.
On Wed, Dec 22, 2021 at 09:23:13AM +0100, Mehdi Sadeghi wrote:
> ... Emails get delivered everywhere except gmx.de sofar,> since they need a mandatory reverse DNS record
Reverse DNS record that matches the SMTP HELO hostname
is indeed a huge plus for thrusting / accepting incoming email.
Groeten
Geert Stappers
--
Silence is hard to parse