/usr/bin/mail buggy or smtp server picky?

awayand · 2009-11-16 23:20:45

I want to use mail to send email quickly from my terminal and decided to use msmtp as my smtp client to keep things light and straightforward.

However, when I use /usr/bin/mail to send an email to x@y.com, my smtp server (mail.messagingengine.com if you are interested) queues the email that was delivered by my /usr/bin/msmtp client fine (I get an OK for queue message), but after a couple of hours I get an NDR (Non Delivery Report) saying that my message couldn't be sent and will be retried until ultimately I get an NDR that my message couldn't be delivered.

After some debugging, I realized that the reason the smtp server was having trouble is that my message didn't contain the From: header. Since the from address is already specified in the envelope upon SMTP-initiation, it seemed to me illogical that the smtp server couldn't just use that email.

So I opened a ticket with those guys over at fastmail.fm, and they told me that the problem was not with the SMTP server, but with the mail client, which creates "invalid" messages (i.e. not sending the From: header). Since I trust my provider to know his stuff, here's my question:

How can I tell /usr/bin/mail to insert a specific From: header?

Thanks!

anrxc · 2009-11-16 23:43:30

You should install a better mailx implementation. The mailx-heirloom from [community] is much better (it's also the default in many distributions, like Slackware). If you install it you don't even need msmtp.

awayand · 2009-11-17 00:17:39

Thanks, I will try that! Is there a way to vote or otherwise get mailx-heirloom into the standard core installation? It seems to me that [mailx 8.1.1-7] is broken and cannot be used properly, as I couldn't find a way to configure the From: header using /usr/bin/mail from the mailx package...

fukawi2 · 2009-11-17 00:20:05

awayand wrote:

Since I trust my provider to know his stuff, here's my question:
How can I tell /usr/bin/mail to insert a specific From: header?

They should not be rejecting mail due to a missing header -- so long as the envelope is valid then they should pass the message forward like they're supposed to. UNLESS you they are providing a spam service to you. A missing FROM header is a pretty good sign of spam, but is still valid at an SMTP level.

Try ssmtp too, I use it on my desktops to relay cron etc and don't have any problems

awayand · 2009-11-17 00:50:22

Hello fukawi2,

I trust those guys over at fastmail.fm to know what they are doing, even though it did seem strange to me too, that a missing from: header was illegal. After all, I am using the from: envelope, and the smtp server does accept the message. So I have checked myself (RFC 822) at http://www.faqs.org/rfcs/rfc822.html (format of internet messages), and interestingly enough, an email sent without a From: header does seem illegal. For those that don't want to skip to section A.3.1 of the rfc, the minimum required for a standard internet mail message is a From:, Date: and Bcc: or To: header.

In light of these findings, should I open a bug report or what do you recommend to get mailx replaced by a non-broken version of mail (like mailx-heirloom as suggested by anrxc)? Apparently, mailx is unable to generate correct emails without the help of an MTA that would force-insert the required headers...

fukawi2 · 2009-11-17 02:06:18

4.4.1. FROM / RESENT-FROM
This field contains the identity of the person(s) who wished
this message to be sent. The message-creation process should
default this field to be a single, authenticated machine
address, indicating the AGENT (person, system or process)
entering the message. If this is not done, the "Sender" field
MUST be present. If the "From" field IS defaulted this way,
the "Sender" field is optional and is redundant with the
"From" field. In all cases, addresses in the "From" field
must be machine-usable (addr-specs) and may not contain named
lists (groups).

Note the use of "should" vs "must". As long as the "Sender" header is being created, then there is no requirement for a "From" header.

Last edited by fukawi2 (2009-11-17 02:06:41)

awayand · 2009-11-17 09:44:26

Right. But /usr/bin/mail from the mailx package doesn't include a Sender: header, either. And I don't see a way to configure it. Unless I type in the necessary headers into my email body every time I send an email using /usr/bin/mail...

fukawi2 · 2009-11-17 12:31:29

I'm not sure what's going on... I have the standard 'mailx' package installed, using SSMTP to relay mail to my SMTP server and the "From" header is added correctly:

Received: by phil-desktop.fukawi2.local (sSMTP sendmail emulation); Tue, 17 Nov 2009 23:28:24 +1100
From: "Phillip Smith" <fukawi2@phil-desktop.fukawi2.local>
Date: Tue, 17 Nov 2009 23:28:24 +1100
To: fukawi2@myemailaddress.com
Subject: test

Perhaps a bug in MSMTP, although it shouldn't alter any existing headers, or be responsible for adding the "From" header.

How are you sending the mail?

awayand · 2009-11-17 23:36:17

Hello fukawi2,

my ~/.mailrc contains one line:

set sendmail=/usr/bin/msmtp

Here's what happens when write an email using /usr/bin/mail from the mailx 8.1.1-7 package:

[x@eee ~]$ mail x@y.com
Subject: hello
Another test
.
EOT

Here's how /usr/bin/mail passes my email to msmtp:

/usr/bin/msmtp -i x@y.com
To: x@y.com
Subject: hello
Another test

Here's what my msmtp config looks like:

#~/.msmtp
account fastmail
from "x@y.com"
host mail.messagingengine.com
auth on
tls on
tls_certcheck off
user x@fastmail.fm
password "*"
logfile ~/msmtp.log

Here's what the msmtp log shows when I send the email:

nov 18 00:28:41 host=mail.messagingengine.com tls=on auth=on user=x@fastmail.fm from=x@u.com recipients=x@y.com mailsize=41 smtpstatus=250 smtpmsg='250 2.0.0 Ok: queued as C3A00A7B7' exitcode=EX_OK

I will send the NDR from the SMTP server when it arrives (takes a couple of hours) if you'd like to inspect it.

Maybe ssmtp has a way to insert a From: header automatically, I haven't taken a look at it. Maybe even msmtp has a way to do that, even though I haven't found that option in the man pages.

What I am trying to say is, if the /usr/bin/mail program cannot be configured to create a valid rfc-compliant email, then using an intermediary mail transport agent like ssmtp would only be a workaround but not a permanent fix. Hence my suggestion to replace mailx with a compliant, working package in the arch core install.

Any ideas on how to proceed from here?

Last edited by awayand (2009-11-17 23:37:45)

fukawi2 · 2009-11-18 00:06:57

awayand wrote:

my ~/.mailrc contains one line:
set sendmail=/usr/bin/msmtp

I see... I wasn't aware you were overriding the your sendmail. I'm not familiar with MSTMP so my only suggestion is to replace MSMTP with SSMTP and remove the above line from your .mailrc so /usr/bin/mail (from mailx package) can just hand off the mail to SSMTP.

Perhaps it is a bug in mailx if it doesn't create the From header when using a non-default sendmail...

Arch Linux

#1 2009-11-16 23:20:45

/usr/bin/mail buggy or smtp server picky?

#2 2009-11-16 23:43:30

Re: /usr/bin/mail buggy or smtp server picky?

#3 2009-11-17 00:17:39

Re: /usr/bin/mail buggy or smtp server picky?

#4 2009-11-17 00:20:05

Re: /usr/bin/mail buggy or smtp server picky?

#5 2009-11-17 00:50:22

Re: /usr/bin/mail buggy or smtp server picky?

#6 2009-11-17 02:06:18

Re: /usr/bin/mail buggy or smtp server picky?

#7 2009-11-17 09:44:26

Re: /usr/bin/mail buggy or smtp server picky?

#8 2009-11-17 12:31:29

Re: /usr/bin/mail buggy or smtp server picky?

#9 2009-11-17 23:36:17

Re: /usr/bin/mail buggy or smtp server picky?

#10 2009-11-18 00:06:57

Re: /usr/bin/mail buggy or smtp server picky?

Board footer