| MAILMAN: using maillist.php as PRIVATE_EXTERNAL_ARCHIVER [message #10868] |
Thu, 12 June 2003 16:32  |
Tobias Eigen
Messages: 85
Registered: June 2003
Location: Seattle, WA USA
Karma: 0
|
Member |
|
|
Hi all,
I have been gratefully exploring FUD - it is a beautiful program. Many thanks for developing it and sharing it with the open source community. It will certainly help my organization to help African civil society organizations to make better use of mailing lists for networking and information sharing.
I have a suggestion for a feature that might make alot of mailman administrators happy. If it were possible to specify the list name instead of the forum topic number when running maillist.php, then we would be able to set up FUD as the Mailman archiver. This would be very slick indeed.
Mailman generates mbox archives as well as web-based archives, either with PIPERMAIL (the built-in archiver) or MHONARC. Below is the command I have set in ~mailman/Mailman/mm_cfg.py to use MHONARC as the private external archiver:
PRIVATE_EXTERNAL_ARCHIVER = '/usr/bin/mhonarc -add -outdir
/usr/local/mailman/archives/private/%(listname)s -rcfile
/usr/local/mailman/archives/private/mhonarc.rc -htmlext html -idxfname index.html -title
%(listname)s -reverse -spammode >> /usr/local/mailman/logs/mhonarc_private'
I'd propose to change this to something like this:
PRIVATE_EXTERNAL_ARCHIVER = '/path/to/maillist.php %(listname)s
In python, %(listname)s is replaced with the actual list name, ie. the part of the address before the @ symbol (ie. test-list for test-list(at)lists(dot)kabissa(dot)org). With this setup, I don't need to do anything special to configure mailman archives.. it simply works by default. No aliases to set up, etc.
This setup is also completely secure - nothing gets archived without going through mailman first. This means that mailman can bounce spam and moderators can reject unwanted posts so they are not archived. It's also not possible to send email directly to the archive - which the current FUD solutions all seem to make possible (ie. mail to foo-archive(at)foo(dot)org is posted to the forum automatically - not necessarily a good thing).
I don't know how FUD runs internally, but if we could specify the mailman listname in the mailing list admin interface and use it when running maillist.php, then mailman/fud integration would be pretty much seamless. An alternative would be to strip out the first part of the list's email address to get the list name.
Another angle altogether might be to emulate a slick little script I use on my qmail system called qmail-to-mailman.py. Found in the mailman /contrib directory, this script can be set up to be run via a .qmail file in a dedicated domain mailbox. It takes the message, strips the @foo.com from it to get the listname, then sends the message to mailman for handling on that list. If the list doesn't exist, it returns a helpful bounce message. Works completely flawlessly.
I'd be happy to help out in any way I can - I'm not a programmer but am happy to test out and troubleshoot.
Cheers,
Tobias
Kabissa - Space for change in Africa
|
|
|
|
| Re: MAILMAN: using maillist.php as PRIVATE_EXTERNAL_ARCHIVER [message #10880 is a reply to message #10868] |
Thu, 12 June 2003 17:05  |
Ilia
Messages: 13241
Registered: January 2002
Karma: 0
|
Senior Member
Administrator
Core Developer |
|
|
Cannot be done without adding mailman specific hacks and I refuse to do that. The only possiblity is to allow maillist.php to accept the list's e-mail address identifier rather then the list id. That can be done with fairly light alterations to the current code.
FUDforum Core Developer
|
|
|
|
|
|
| Re: MAILMAN: using maillist.php as PRIVATE_EXTERNAL_ARCHIVER [message #10884 is a reply to message #10882] |
Thu, 12 June 2003 17:19 |
Ilia
Messages: 13241
Registered: January 2002
Karma: 0
|
Senior Member
Administrator
Core Developer |
|
|
The forum name my contain special characters that would require all sort's of encoding to be safely passed via the shell. So for security reasons it is probably not a good idea to use.
That said, you could easily implement some go between function that you could add to maillist.php that would translate any input into the understood rule id.
For example:
<?php
$mailman_id = array('LIST1' => 1, 'LIST2' => 2);
if (isset($mailman_id[$_SERVER['argv'][1]])) {
$_SERVER['argv'][1] = $mailman_id[$_SERVER['argv'][1]];
} else {
$_SERVER['argv'][1] = NULL;
}
?>
FUDforum Core Developer
|
|
|
|
| Re: MAILMAN: using maillist.php as PRIVATE_EXTERNAL_ARCHIVER [message #10886 is a reply to message #10882] |
Thu, 12 June 2003 17:21 |
Tobias Eigen
Messages: 85
Registered: June 2003
Location: Seattle, WA USA
Karma: 0
|
Member |
|
|
You know - I just had a thought.. your suggestion might work if the line reads like this:
PRIVATE_EXTERNAL_ARCHIVER = '/path/to/maillist.php %(listname)s(at)lists(dot)kabissa(dot)org'
I'm not quite sure what happens to messages to lists that are set up to 'prefer' a domain name other than the default lists.kabissa.org, but I think it might just work.
How do we proceed? I'd love to try this out.
Cheers,
Tobias
Kabissa - Space for change in Africa
|
|
|
|
| Re: MAILMAN: using maillist.php as PRIVATE_EXTERNAL_ARCHIVER [message #10888 is a reply to message #10886] |
Thu, 12 June 2003 17:29 |
Ilia
Messages: 13241
Registered: January 2002
Karma: 0
|
Senior Member
Administrator
Core Developer |
|
|
Basically the e-mail address provided needs to be indetical the one specified as the 'Mailing List Email' on the mailling list manager.
To try it do the following changes:
Change:
if ($_SERVER['argc'] < 2 || (!($fid = (int)$_SERVER['argv'][1]))) {
To:
if ($_SERVER['argc'] < 2) {
Change:
$mlist = db_sab('SELECT * FROM '.sql_p.'mlist WHERE id='.$fid);
To:
$mlist = db_sab('SELECT * FROM '.sql_p.'mlist WHERE name=\''.addslashes($_SERVER['argv'][1]).'\'');
FUDforum Core Developer
|
|
|
|
|
|
|
|
|
|
| Re: MAILMAN: using maillist.php as PRIVATE_EXTERNAL_ARCHIVER [message #10979 is a reply to message #10868] |
Sun, 15 June 2003 18:51 |
|
Anonymous
|
|
|
|
Hi,
Just about to start testing maillist.php as external archiver for mailman.. and had a question for you. What happens when a message can't be imported for whatever reason? Ie. the forum was not properly identified on the command-line? Is there a log file that is added to, and is there a directory where "bad" messages get saved? If not, these features might be a good idea to add for error-tracking.
Cheers,
Tobias
(I can edit this anonymous post - why is that?)
[Updated on: Sun, 15 June 2003 22:17] Report message to a moderator |
|
|
|
|
|
|
|
| Re: MAILMAN: using maillist.php as PRIVATE_EXTERNAL_ARCHIVER [message #11077 is a reply to message #10868] |
Tue, 17 June 2003 16:43 |
Tobias Eigen
Messages: 85
Registered: June 2003
Location: Seattle, WA USA
Karma: 0
|
Member |
|
|
Hi,
I'm having a little trouble getting maillist.php working as PRIVATE_EXTERNAL_ARCHIVER in Mailman. I know this is a little specific, but I think other Mailman administrators would also appreciate having this figured out.
I'm getting the error message below when the list does not correspond with an existing forum -
<b>UNABLE TO WRITE TO SQL LOG FILE</b><br>
(maillist.php) 1064: You have an error in your SQL syntax near '' at line 1<br />
Query: SELECT id, tag_style, message_threshold, (max_attach_size * 1024) AS max_attach_size, max_file_attachments FROM fud25_forum WHERE id=<br />
Server Version: 3.23.54<br />
Can I have these messages dumped into a forum for me to review and/or move to their proper place, then set up properly?
I'm also trying to set it up so that both mhonarc and maillist.php work simultaneously, while I'm setting up FUD forum. This is not working for me. Here's the line I'm using in ~Mailman/mm_cfg.py:
PRIVATE_EXTERNAL_ARCHIVER = '(/usr/bin/mhonarc -add -outdir
/usr/local/mailman/archives/private/%(listname)s -rcfile
/usr/local/mailman/archives/private/mhonarc.rc -htmlext html -idxfname index.html -title
%(listname)s -reverse -spammode ;
/home/httpd/vhosts/kabissa.org/httpdocs/team/fud-data/scripts/maillist.php
%(listname)s(at)lists(dot)kabissa(dot)org)'
Maillist.php generates a line in the errors/.mlist/error_log file and creates a zero-length filename in the same directory like this:
ERROR :: Tue, 17 Jun 2003 11:29:45 -0500 :: no name or email for :: /home/httpd/vhosts/kabissa.org/httpdocs/team/fud-data/errors/.mlist/1055867 385_d41d8cd98f00b204e9800998ecf8427e
I presume this means it's not getting the message piped to it - so I'm clearly doing something wrong.
Any thoughts?
Cheers,
Tobias
Kabissa - Space for change in Africa
|
|
|
|
| Re: MAILMAN: using maillist.php as PRIVATE_EXTERNAL_ARCHIVER [message #11079 is a reply to message #11077] |
Tue, 17 June 2003 17:03 |
Ilia
Messages: 13241
Registered: January 2002
Karma: 0
|
Senior Member
Administrator
Core Developer |
|
|
The second error message indicates that the script failed to locate the header which would tell it who the message came from (From: header).
The 1st error message sounds like the list identifier that is passed is not correct and it failes to retrieve data from MySQL about this list rule.
FUDforum Core Developer
|
|
|
|
|
|
|
|
| Re: MAILMAN: using maillist.php as PRIVATE_EXTERNAL_ARCHIVER [message #11092 is a reply to message #11091] |
Tue, 17 June 2003 18:18 |
Tobias Eigen
Messages: 85
Registered: June 2003
Location: Seattle, WA USA
Karma: 0
|
Member |
|
|
 I'll keep plowing away at it.
Basically Mailman is piping the message to maillist.php in the same way that a .qmail file would. Mailman provides the list name via a variable, which I use to generate the listname(at)lists(dot)kabissa(dot)org on the command line. This works fine for lists configured in FUD, and when it's the only program configured as the external archiver.
If the listname(at)lists(dot)kabissa(dot)org rule isn't specified in FUD someplace, then it chokes on the message and saves it in the .mlist error directory. My preference would be for it to be posted to a default holding forum where I can then deal with it by deleting it or creating the appropriate forum and moving the message there.
I've had no luck getting Mailman to pipe the message to two external archivers at the same time - this is a disappointment since it hampers my testing and transition efforts. I must be missing something about how shell commands work. There must be a way to it, ie.
cat blah.txt | (program1 >> log1 ; program 2 >> log2)
Cheers,
Tobias
Kabissa - Space for change in Africa
[Updated on: Tue, 17 June 2003 18:18] Report message to a moderator |
|
|
|
|
|
| Re: MAILMAN: using maillist.php as PRIVATE_EXTERNAL_ARCHIVER [message #11104 is a reply to message #11092] |
Tue, 17 June 2003 19:30 |
Tobias Eigen
Messages: 85
Registered: June 2003
Location: Seattle, WA USA
Karma: 0
|
Member |
|
|
Sweet - a kind soul from the mailman-users list provided me with the answer to one of my MAILMAN/FUD questions:
http://www.mail-archive.com/mailman-users%40python.org/msg17520.html
The answer is to do it like this:
PRIVATE_EXTERNAL_ARCHIVER = 'cat > /tmp/mailman_pvt_tmp$$; archiver-1-command
%(listname)s-stuff < /tmp/teststuff$$; archiver-2-command
%(listname)s-stuff < /tmp/teststuff$$; rm /tmp/teststuff$$'
This isn't elegant for the long term, but at least now I can go ahead and continue testing FUD and migrating my lists into it. The only problem that remains is that mail to unknown forums is being treated as an error instead of being imported into a holding forum where I can deal with it manually.
This should be possible via maillist.php - can you possibly give me some guidance?
Cheers,
Tobias
Kabissa - Space for change in Africa
|
|
|
|
|
|
| Re: MAILMAN: using maillist.php as PRIVATE_EXTERNAL_ARCHIVER [message #11107 is a reply to message #11098] |
Tue, 17 June 2003 20:35 |
Tobias Eigen
Messages: 85
Registered: June 2003
Location: Seattle, WA USA
Karma: 0
|
Member |
|
|
Hi Protoss,
Thanks for the very proactive help today - I really appreciate it. I'm psyched regarding the ability to use the email address instead of the list id. Many thanks.
Regarding the default forum - above you wrote:
| Quote: |
You can easily hackup the default forum by doing:
<?php
if (isset($_SERVER['argv'][1])) {
$_SERVER['argv'][1] = default_rule_id;
$_SERVER['argc']++;
}
?>
|
Where can I include this? Sorry for being thick. Where do I specify the default forum id/name?
Cheers,
Tobias
Kabissa - Space for change in Africa
|
|
|
|
|
|
| Re: MAILMAN: using maillist.php as PRIVATE_EXTERNAL_ARCHIVER [message #11109 is a reply to message #11108] |
Tue, 17 June 2003 20:57 |
Tobias Eigen
Messages: 85
Registered: June 2003
Location: Seattle, WA USA
Karma: 0
|
Member |
|
|
<sigh> this isn't working for me - remember I had commented out the line just above the one you're talking about - perhaps that's causing me problems:
// if ($_SERVER['argc'] < 2 || (!($fid = (int)$_SERVER['argv'][1]))) {
if (isset($_SERVER['argv'][1])) {
$_SERVER['argv'][1] = 4;
$_SERVER['argc']++;
}
Anyhoo - no tragedy for the moment, I'm pleased with my successes today and will head home. I will take the time tomorrow to create a forum for each list hosted on this server.
Before I do that I will reinstall FUD with default values again - is there a new version I should get?
Thanks again,
Cheers,
Tobias
Kabissa - Space for change in Africa
|
|
|
|
|
|
|
|
|
|
| Re: MAILMAN: using maillist.php as PRIVATE_EXTERNAL_ARCHIVER [message #11205 is a reply to message #11204] |
Sat, 21 June 2003 20:14 |
Ilia
Messages: 13241
Registered: January 2002
Karma: 0
|
Senior Member
Administrator
Core Developer |
|
|
In that case change:
<?php
if (isset($_SERVER['argv'][1])) {
?>
to
<?php
if (!isset($_SERVER['argv'][1])) {
?>
Now only time maillist.php is called without a list id, will it try sending the message to the default forum.
FUDforum Core Developer
|
|
|
|
Calendar
Search
Help
Members
Register
Login
Home
I am not sure how you are executing the script so I don't know.
]