Bobby
ah yes thank you for this
This site's content lives in a headless instance of Ghost CMS hosted on Fly.io. I've been very happy with it for a number of reasons, two of which are its writing experience and built-in newsletter support.
But I hit an interesting issue after upgrading to a newer version of Ghost (from 5.36 to 5.82.2... it had been a while): I could suddenly only send emails to ~30% of my email list. After digging into a few member records, I saw this:
That led me to some documentation indicating that Ghost will automatically disable accounts it thinks can no longer receive emails. The problem was that, according to my metrics, several of my members had opened nearly all of my emails. Coupled with the fact that this issue only came up after upgrading, something didn't seem quite right.
I didn't do an extremely thorough investigation, but after crawling through Ghost's source a bit, I saw references to two events that are dispatched & handled in the application: EmailBouncedEvent
and SpamComplaintEvent
. Somewhere along the way, the email_recipients
table came up, which seems to serve as a logging mechanism for every email sent from Ghost (including when those emails fail). Also in this mix was the MailgunEmailSuppressionList
, which subscribed to those aforementioned events:
const handleEvent = async (event) => {
try {
await this.Suppression.add({
email_address: event.email,
email_id: event.emailId,
reason: 'bounce',
created_at: event.timestamp
});
} catch (err) {
if (err.code !== 'ER_DUP_ENTRY') {
logging.error(err);
}
}
};
DomainEvents.subscribe(EmailBouncedEvent, handleEvent);
DomainEvents.subscribe(SpamComplaintEvent, handleEvent);
}
That was good enough for me to assume: any email that had ever bounced was being suppressed.
That assumption got a bit stronger when a few more threads led to an email analytics service, which appears to have been added in a version of Ghost newer than the one I had been using. That service wires up a batch job pulling data from that email_recipients
table. Since none of the records in this table had ever been processed, it looks like they were all processed at once, disabling a bunch of emails, and giving way to this "sudden" issue.
After a little more snooping, I found that a member's email is disabled by setting the email_disabled
column on the members
table. This meant I'd be able reenable emails for all of my members with a single query. Scary. I know.
I'm using Sqlite, so after SSH-ing into my machine, it meant running these two commands:
sqlite3 ghost.db
sqlite> UPDATE members SET email_disabled = 0;
The admin looked a little better after that, leaving those members all set to receive emails again.
You have good reason to scrunch your nose a little bit at this. Making production database updates for an application I'm not familiar with isn't something I'd recommend making a habit. Two particular risks popped into my head as I was doing this:
#1: What if the member wanted their email disabled?
I dismissed this one pretty quickly. The user didn't take any action to do this. Ghost did. And several of those people had clearly been receiving & opening emails up until now. Plus, if a member didn't want to receive emails anymore, they could've unsubscribed. Or marked me as "spam."
#2: What if I harm my domain authority?
This was a bigger concern. I was worried that if I started sending emails again after being marked as spam (which might've been the reason a member's email was disabled), my authority as a sender would tank.
Fortunately, that concern was eased after finding an email_spam_complaint_events
table in the database as well. It was completely empty. This was affirming. I must be sending some non-spammy content.
Well, I monitor. If I'm right about this, the only reason an email address would be disabled again is if it bounces again, dispatching that EmailBouncedEvent
and adding them to the suppression list. But that's easy enough to visually keeps tabs on in the UI for a member:
I'm just glad further issues shouldn't pop up as a surprise again. But we'll see how things look moving forward. I'm willing to get a little messier in the database if I have to.
Alex MacArthur is a software engineer working for Dave Ramsey in
Nashville-ish, TN.
Soli Deo gloria.
Get irregular emails about new posts or projects.
No spam. Unsubscribe whenever.ah yes thank you for this
agreed
np