FUDforum
Fast Uncompromising Discussions. FUDforum will get your users talking.

Home » FUDforum Development » Bug Reports » Invalid email headers
Show: Today's Messages :: Polls :: Message Navigator
Switch to threaded view of this topic Create a new topic Submit Reply
Invalid email headers [message #188149] Fri, 16 July 2021 09:11 Go to next message
JeanAbouSamra is currently offline  JeanAbouSamra   France
Messages: 7
Registered: July 2021
Karma: 0
Junior Member
Hello,

On a locally hosted instance of FUDforum, I am experiencing issues with mailing list synchronization.

- When I do not configure a custom SMTP server, no emails are sent at all (not just the posts, but also account confirmation emails).

- When I do configure an SMTP server, it sometimes fails, depending on the server and the mailing list address. When the SMTP is auth.smtp.1and1.fr, and the mailing list is on lists.gnu.org, mail delivery fails with replies like the following:

This message was created automatically by mail delivery software.

A message that you sent could not be delivered to one or more of
its recipients. This is a permanent error.

The following address failed:

lilypond-user-fr(at)gnu(dot)org:
SMTP error from remote server for TEXT command, host: eggs.gnu.org (209.51.188.92) reason: 550 Message headers fail syntax check


--- The header of the original message is following. ---

Received: from fudforum.localhost ([90.29.105.171]) by
 mrelayeu.kundenserver.de (mreue012 [212.227.15.168]) with ESMTPA (Nemesis) id
 1M91Tq-1m1abU45Xa-006Agp for <lilypond-user-fr(at)gnu(dot)org>; Fri, 16 Jul 2021
 10:55:07 +0200
Subject: Nouveau test
Date: Fri, 16 Jul 2021 10:55:07 +0200
To: lilypond-user-fr(at)gnu(dot)org
From: Administrator <jean(at)abou-samra(dot)fr>
Errors-To: Administrator <jean(at)abou-samra(dot)fr>
Return-Path: Administrator <jean(at)abou-samra(dot)fr>
X-Mailer: FUDforum v3.1.1
MIME-Version: 1.0
Content-Type: text/plain;
  charset="utf-8"
Content-Transfer-Encoding: 8bit
Organization: My forum, my way!
X-FUDforum: 8c9495fe7974471f9dc0168115731210 <3>
Message-ID: <3(dot)60f14969(at)fudforum(dot)localhost>
X-Provags-ID: V03:K1:5IUNE5rBW50JR/81TCYz9GTY3JGrtugb39n86zLqkbinTWxPotE
 owOWlfK8g3pgNPnzxyaqlpyCTdT+TeBj1gr3YM8RMRKuiRVfQ8QIgb41M0UMY/7DXVLrtvq
 3MHiMocFi3A7+Vjon+ge8BM91x8Fnoxb0DUeyT7BzHQaxKRyMofv7Vg2G3y8dSZGU+9cueq
 spNsB1YCzToaDWI2+TUZQ==
X-Spam-Flag: NO
X-UI-Out-Filterresults: notjunk:1;V03:K0:4NmddZixQBc=:/dw3WJVXlOVY7tb70itGXS
 LECVjiuOoQmVVXykcbJ7AweIKtZZEFUxDOBKpkDl2X6oklMlNDBw1vtUyE4HF37IERuCpZ53b
 5oY8c7Sfw7XO2TVrYEsZzqCdG/Slp3Muv2duOmlJTZwtsAAD58foGWJGq4kZKVPCoJ5Bu1zNR
 Zjkb0PzKIpE0ltvjD0Hfv97FW0r5Ek5EAtHxLYr2kSx8ljxgS1oZwoAYkym4yc5QR6Axqk1+6
 KyptLRUacejA2uH/OC/RCoDnpTeIkjsSkHZhPQRiRm8lcvuZn2P0sZ5niXXZZodpUq1A564l+
 GIdQYg/PmmfkwAA+ZfNMYQabU0bWwlTrPhg6NDrkMaa7iL00Sq9+ILLxYPmxMGt/hZ3a9FJWl
 MVL7nWGzNKYiixb1OVpH7sAxudPusu+VsjQwjxlfnffu4C5QjjatOlh27eTvc5jjp0sLSE0cz
 A8niGgemci4sIUgSoXTpCe2Z3ZYh1cBF8vV6s6tqDsfd5Bgl4YaAeJFZQ1YODDDprF9A4S5Ml
 UQLQw4AtoESIc1jAZS7iKmTPnSdQJhtpXWK3psYgbQ1N+lVIz5nVjfiM9Qqh2CKqYgBAbj4fg
 Nnt6KhI35uYMyjYofbRLMALTisAcUWXg9yh3idjnRdjyNJtmE1q2znKZlSx3gVP0AOZBIh17z
 UZvv8D5J6uzQ+VuaeNcp+qWv4e3RAuQmpRDIBURvhq+qrhtHh4Jol8hDgzSC6ify2bkU5bRmO
 w7xW+z5G5US5QQ89/qSOBl5OeLeYTAED7qC+ZaEETHY8vPTi6ZVXJ90sRaapl6qk76GXgj11j
 E0m3IvtVSG5p8giWoia/UgsEK2XZ6jkNLia5lNitaS3yxZ7XA7R9BLUEosn+FSz+g1+D/5g

I've tried SMTP servers auth.smtp.1and1.fr and smtp.gmail.com, as well as recipients gmail.com, a personal address at 1and1, and gnu.org lists. Only the combination of sender auth.smtp.1and1.fr and recipient gnu.org triggers the failure.


As was suggested to me, I tried the following patch, to no avail:

diff --git a/install/forum_data/src/iemail.inc.t b/install/forum_data/src/iemail.inc.t
index be9d35037..ccff78d51 100644
--- a/install/forum_data/src/iemail.inc.t
+++ b/install/forum_data/src/iemail.inc.t
@@ -63,7 +63,14 @@ function send_email($from, $to, $subj, $body, $header='', $munge_newlines=1)
        if (strpos($header, 'MIME-Version') === false) {
                $extra_header = "\nMIME-Version: 1.0\nContent-Type: text/plain; charset={TEMPLATE: iemail_CHARSET}\nContent-Transfer-Encoding: 8bit". $header;
        }
-       $header = 'From: '. $from ."\nErrors-To: ". $from ."\nReturn-Path: ". $from ."\nX-Mailer: FUDforum v". $GLOBALS['FORUM_VERSION']. $extra_header. $header;
+
+    $addronly = preg_replace(".*<", "<", $from);
+    $header =
+        "From: $from\n" .
+        "Errors-To: $addronly\n" .
+        "Return-Path: $addronly\n" .
+        "X-Mailer: FUDforum v" . $GLOBALS['FORUM_VERSION'] .
+        $extra_header . $header;
 
        $body = str_replace("\r", '', $body);
        if ($munge_newlines) {
diff --git a/install/www_root/adm/admmassemail.php b/install/www_root/adm/admmassemail.php
index 4972fe148..9a96c02bc 100644
--- a/install/www_root/adm/admmassemail.php
+++ b/install/www_root/adm/admmassemail.php
@@ -71,8 +71,15 @@
                        $email_batch_cnt = 0;
                        $to = array();
 
+            $addronly = preg_replace(".*<", "<", $ADMIN_EMAIL);
                        if (!($FUD_OPT_1 & 512)) {      // Not USE_SMTP.
-                               $header = 'From: '. $ADMIN_EMAIL. "\r\nErrors-To: ". $ADMIN_EMAIL. "\r\nReply-To: ". $ADMIN_EMAIL. "\r\nX-Mailer: FUDforum v". $GLOBALS['FORUM_VERSION']. "\r\nMIME-Version: 1.0\r\nContent-Type: text/plain; charset=". $charset. "\r\nContent-Transfer-Encoding: 8bit";
+                               $header =
+                  'From: '. $addronly.
+                  "\r\nErrors-To: ". $addronly.
+                  "\r\nReply-To: ". $addronly.
+                  "\r\nX-Mailer: FUDforum v". $GLOBALS['FORUM_VERSION'].
+                  "\r\nMIME-Version: 1.0\r\nContent-Type: text/plain; charset=". $charset.
+                  "\r\nContent-Transfer-Encoding: 8bit";
                        
                                $_POST['body'] = str_replace("\n.", "\n..", $_POST['body']);
 
@@ -113,7 +120,7 @@
                                        }
                                }
                        } else {
-                               $header = 'Errors-To: '. $ADMIN_EMAIL. "\r\nReply-To: ". $ADMIN_EMAIL. "\r\nMIME-Version: 1.0\r\nContent-Type: text/plain; charset=". $charset. "\r\nContent-Transfer-Encoding: 8bit";
+                               $header = 'Errors-To: '. $addronly. "\r\nReply-To: ". $addronly. "\r\nMIME-Version: 1.0\r\nContent-Type: text/plain; charset=". $charset. "\r\nContent-Transfer-Encoding: 8bit";
 
                                $smtp = new fud_smtp;
                                $smtp->msg = str_replace("\n.", "\n..", $_POST['body']);
This was intended to strip the "Administrator" part from the fields, but they remain in the error message.

Any help would be appreciated.
Re: Invalid email headers [message #188152 is a reply to message #188149] Sat, 17 July 2021 15:32 Go to previous messageGo to next message
naudefj is currently offline  naudefj   
Messages: 3772
Registered: December 2004
Karma: 28
Senior Member
Administrator
Core Developer
I'm not entirely sure what you want to do, but this may help:

<?php
$from = 'Administrator <jean(at)abou-samra(dot)fr>';

$addronly = substr($from, strpos($from, '<')+1, -1);
print $addronly ."\n";
-- This will print: jean(at)abou-samra(dot)fr

$addronly = preg_replace("/.*</", "<", $from);
print $addronly ."\n";
-- This will print: <jean(at)abou-samra(dot)fr>

[Updated on: Sat, 17 July 2021 15:33]

Report message to a moderator

Re: Invalid email headers [message #188153 is a reply to message #188149] Sun, 18 July 2021 16:37 Go to previous messageGo to next message
omiak is currently offline  omiak
Messages: 4
Registered: July 2021
Karma: 0
Junior Member
Hello naudefj,

You are right about the proposed preg_replace, slashes were missing.

But I beleive the original code to be wrong about the Return-Path and Errors-To headers.


1) The Return-Path header should consist of an angle-address only, and not « name + angle-address » as in From header.

<https //datatracker.ietf.org/doc/html/rfc2822#section-3.6.7>
§
return = "Return-Path:" path CRLF

path = ([CFWS] "<" ([CFWS] / addr-spec) ">" [CFWS]) /
obs-path
§


2) The Errors-To header is non standard and discouraged, it shouldn't be set at all.

<https //datatracker.ietf.org/doc/html/rfc2076.txt#section-3.5>
§
Address to which notifications Errors-To:, Non-standard,
are to be sent and a request to Return- discouraged.
get delivery notifications. Receipt-To:
Internet standards recommend,
however, the use of RCPT TO and
Return-Path, not Errors-To, for
where delivery notifications are
to be sent.
§
(oups, it is unreadable, even with font=Courier... read the original RFC instead)


Regards,
Olivier Miakinen
Re: Invalid email headers [message #188154 is a reply to message #188153] Mon, 19 July 2021 11:59 Go to previous messageGo to next message
omiak is currently offline  omiak
Messages: 4
Registered: July 2021
Karma: 0
Junior Member
Hello,

I found the "code" balise...

RFC 2822 section 3.6.7:
return          =       "Return-Path:" path CRLF

path            =       ([CFWS] "<" ([CFWS] / addr-spec) ">" [CFWS]) /
                        obs-path
RFC 2076 section 3.5:
   Address to which notifications       Errors-To:,    Non-standard,
   are to be sent and a request to      Return-        discouraged.
   get delivery notifications.          Receipt-To:
   Internet standards recommend,
   however, the use of RCPT TO and
   Return-Path, not Errors-To, for
   where delivery notifications are
   to be sent.
Re: Invalid email headers [message #188155 is a reply to message #188154] Mon, 19 July 2021 20:09 Go to previous messageGo to next message
naudefj is currently offline  naudefj   
Messages: 3772
Registered: December 2004
Karma: 28
Senior Member
Administrator
Core Developer
Good info!
I'll try to compile and submit a patch later.
Re: Invalid email headers [message #188156 is a reply to message #188155] Mon, 19 July 2021 21:10 Go to previous messageGo to next message
naudefj is currently offline  naudefj   
Messages: 3772
Registered: December 2004
Karma: 28
Senior Member
Administrator
Core Developer
I hope this will do:
https://github.com/fudforum/FUDforum/commit/47c5ff298539a576aa0c5edfbfca5ac 353fe986b
Re: Invalid email headers [message #188157 is a reply to message #188149] Tue, 20 July 2021 07:08 Go to previous messageGo to next message
JeanAbouSamra is currently offline  JeanAbouSamra   France
Messages: 7
Registered: July 2021
Karma: 0
Junior Member
Hello,

Thanks for your prompt reaction. Unfortunately, after reinstalling the
forum on latest master, the error still shows up. This time, the
header is:

--- The header of the original message is following. ---

Received: from fudforum.localhost ([90.29.105.171]) by
 mrelayeu.kundenserver.de (mreue106 [212.227.15.184]) with ESMTPA (Nemesis) id
 1MQdI8-1li04w0850-00Nfqw for <lilypond-user-fr(at)gnu(dot)org>; Tue, 20 Jul 2021
 09:02:12 +0200
Subject: Nouveau test
Date: Tue, 20 Jul 2021 09:02:09 +0200
To: lilypond-user-fr(at)gnu(dot)org
From: Administrator <jean(at)abou-samra(dot)fr>
Return-Path: <jean(at)abou-samra(dot)fr>
X-Mailer: FUDforum v3.1.1
MIME-Version: 1.0
Content-Type: text/plain;
  charset="utf-8"
Content-Transfer-Encoding: 8bit
Organization: My forum, my way!
X-FUDforum: 8c9495fe7974471f9dc0168115731210 <3>
Message-ID: <3(dot)60f674f1(at)fudforum(dot)localhost>
X-Provags-ID: V03:K1:ekiZESGLr3TIjjZGjhHJBqitVZFid/BDhVAqJuyTDagf7bu9YDz
 bPM5CFlIecjE09IFflFLKb9AtR8K3tIGufyKHELfmIDddfx1ioTnjo4126NR0G6++QPUAlU
 AfuS+/OWJQ2MvK48WTjo7rY3XRKq431kYK7oJyNkh8iIEMIV5U27uurYuhrkd0EUU1MC810
 fpi96v4wG1s0DkJytqOKw==
X-Spam-Flag: NO
X-UI-Out-Filterresults: notjunk:1;V03:K0:chHrlL4rruQ=:S7XvznfAX1aLsBEbLbwTQI
 waNOYfa5FaPklckbmanGYx0Imi0NMfhKNL70vAkxzmcCY8ETitIhpFBzcHVzL7ELCdYY3vMHz
 ZnI68sMsgOIWB3xezIRqjzLWvx0fJhMrqoOvoNNUY8muIbsIMWgm43dETBFPF3wn7kfFf+Qpc
 isSGehvaUS6JHCd1rY/cTp9muqjEzs1jcbo95ZaFD8fBpL0EinPfUyb/Q2E6N7Wll/N2xUFLi
 88dQqxGaG0MMYm/1c6D+PJLbKPaMt01GxUIA0P7TjSZRVQRHKSnpYkEKZWcJtXSjxK3Ete9Lv
 5uYg9f5l/TgTNbMHpCSoJvNL5qgJ8pJiZ+UwB3VdxW4PNxzLDaZ0iiLgEXhH+ICK5uvuaTB6d
 ml763JdPlF0QrV4fr5pH3tTrkdbOyLxxMbnaWJwNfe/mzMKsOyOQbpToNZ7Y8mEgBBlSBt5Ns
 HNrDLjGxpvDceRI0kO2OluQumAbqUJlLHycBy4SGywLTLcsHN2diZtOWDF+tW/BncFQPAjk1w
 ykZlH14imCK0DUBf7dBSjzQFNJ6aDIXk3aRDGDR+g2NmGybY4Rstvbbhx8cj8zWCWesaHDbw7
 Pp+MuhP0rvMiVR0JDIhqvVgUa+g0MmwignCz/VeIKZOGJQlra7FqSsxbxoQR/uSlR6iDz3Z30
 uGdVCGbhyKmLuy2uaMIM+mciYNppiNKMk+DP311ytJ3vcGqsxKD+pAXDrJh1BKRC6OX9xQDd0
 45WbS/OW+M2s0afK8LPCVVQMdZ09maBjOMbFA5ZUi5/ZYALaWc7Qyb266XKYfzUWZXJOaj/vd
 UFledomhlcWA+bHlUOWkEP11V3gStjCtK10uUoVXaaYvc0yFDyNBc+yfCbz88DNBacCa8+M
The Return-Path is now correct, and Errors-To has disappeared.
Unfortunately, something else still seems to be wrong.

Best regards,
Jean Abou Samra
Re: Invalid email headers [message #188158 is a reply to message #188157] Tue, 20 July 2021 07:11 Go to previous messageGo to next message
JeanAbouSamra is currently offline  JeanAbouSamra   France
Messages: 7
Registered: July 2021
Karma: 0
Junior Member
(Is FUDforum obscuring email addresses to prevent them from being
recognized by web scrapers? The header had @ and . characters.)
Re: Invalid email headers [message #188159 is a reply to message #188158] Tue, 20 July 2021 13:33 Go to previous messageGo to next message
omiak is currently offline  omiak
Messages: 4
Registered: July 2021
Karma: 0
Junior Member
Hi all,

Provided that the addresses contained @ and . characters and not the replacement (at) and (dot), all those headers now seem to be correct. At least I can't see any incorrect syntax in them. Hence, if Jean's server refuses them I suppose that it is a bug in his server itself, not in FUDforum.

In order to try and find what the bug could be, I will go through each header.

Received:
This is correct according to RFC 5322 and 5321. Note however that the third line is 81 characters long, which should be too long for a MIME-encoded header (max 76), but there is no such limitation for other kinds of headers. The only one limit is 998 characters long, and we are far from it.

Subject:, Date:, To:, From:
They are all correct according to RFC 5322 (and 2822 which is the older version of 5322).

Return-Path:
Now correct.

X-Mailer:
Non standard, and as such it has no effective syntax requirement. But I will say more in another response to come.

MIME-Version:, Content-Type:, Content-Transfer-Encoding:
They are correct together, according to RFC 2045 defining MIME.

Organization:
Not defined in the standard RFC 5322 for email, it is defined in informational RFC 7681 for email, and also in standard RFC 5536 for news (usenet). In any case it is an unstructured header, thus any syntax is Ok.

Message-ID:
Correct according to RFC 5322.

X-FUDforum:, X-Provags-ID:, X-Spam-Flag:, X-UI-Out-Filterresults:
Non standard, hence with no specific syntax requirement.
However I checked the total length of X-UI-Out-Filterresults:. Indeed, even though all the lines lengths are less or equal to 76, there may exist bugged email servers which want the total length of the header (summing all individual lines) to be less than 998 characters. But no worry, the total length is really less than 998.

Regards,
Olivier Miakinen

[Updated on: Tue, 20 July 2021 13:34]

Report message to a moderator

Re: Invalid email headers [message #188160 is a reply to message #188159] Tue, 20 July 2021 14:26 Go to previous messageGo to next message
omiak is currently offline  omiak
Messages: 4
Registered: July 2021
Karma: 0
Junior Member
A specific remark about X-Mailer header.
This header is not standard. It should eventually be replaced by User-Agent.

Though the User-Agent header is not yet standard for email, Note that the syntax is specified, with a slash (/) replacing the space and the letter v:
User-Agent: FUDforum/3.1.1

Best Regards,
Olivier Miakinen
Re: Invalid email headers [message #188161 is a reply to message #188160] Tue, 20 July 2021 20:45 Go to previous message
naudefj is currently offline  naudefj   
Messages: 3772
Registered: December 2004
Karma: 28
Senior Member
Administrator
Core Developer
Thank you for the thorough analysis.
Here is the patch to change X-Mailer to User-Agent:
https://github.com/fudforum/FUDforum/commit/47a1b57edb8bae1c0277c603216f066 77b621310
  Switch to threaded view of this topic Create a new topic Submit Reply
Previous Topic: FUDforum only tor-onion hosting
Next Topic: Unlocking then locking files results in wrong permissions
Goto Forum:
  

-=] Back to Top [=-
[ Syndicate this forum (XML) ] [ RSS ]

Current Time: Sun Dec 22 11:53:14 GMT 2024

Total time taken to generate the page: 0.02537 seconds