FUDforum: How To » Body mangling in a mailing list

Home » FUDforum » How To » Body mangling in a mailing list (

) 1 Vote

Show: Today's Messages :: Polls :: Message Navigator

Body mangling in a mailing list [message #14793]

Thu, 20 November 2003 22:15

Squeebee

Messages: 110
Registered: November 2003

Karma: 0

Senior Member

I am looking at the message body mangling, and I would like to remove any line that begins with a > to strip quoting (or maybe can it be converted to a fudcode quote block?) and also like to remove anything after a -- line:

Quote:

--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/mysql?unsub=mhlists(at)telus(dot)net

That is a standard footer I want to remove when I get mailing list importing working.

Thanks for your help, I haven't had time to learn regular expressions all that well yet.

--The pieces are coming together for a move away from PHPbb

Report message to a moderator

Re: Body mangling in a mailing list [message #14796 is a reply to message #14793]

Thu, 20 November 2003 23:21

Ilia

Messages: 13241
Registered: January 2002

Karma: 0

Senior Member
Administrator
Core Developer

"--[\r\n]+MySQL General Mailing List.*" regex and 'm' pattern modifier should do it.

P.S. phpbb conversion script is done, you can get a copy here:
http://fud.prohost.org/forum/index.php?t=msg&goto=1479

FUDforum Core Developer

Report message to a moderator

Re: Body mangling in a mailing list [message #14797 is a reply to message #14793]

Thu, 20 November 2003 23:30

Squeebee

Messages: 110
Registered: November 2003

Karma: 0

Senior Member

You link goes to a post titled "FUDforum 1.2.5 Released" I think it may be in error.

Report message to a moderator

Re: Body mangling in a mailing list [message #14798 is a reply to message #14797]

Thu, 20 November 2003 23:36

Ilia

Messages: 13241
Registered: January 2002

Karma: 0

Senior Member
Administrator
Core Developer

http://fud.prohost.org/forum/index.php?t=msg&goto=14795 is the correct link, I 'lost' a number when doing copy paste.

FUDforum Core Developer

Report message to a moderator

Re: Body mangling in a mailing list [message #14803 is a reply to message #14793]

Fri, 21 November 2003 00:07

Squeebee

Messages: 110
Registered: November 2003

Karma: 0

Senior Member

Nice, but I have to report that I lost 253 users in the process and 600+ posts, let me see what kind of information I can find...

Ok, here's my import script running:

----------------------------------------

Actually when I ran it the second time everything came in... odd but at least it worked.

Report message to a moderator

Re: Body mangling in a mailing list [message #14805 is a reply to message #14803]

Fri, 21 November 2003 00:22

Ilia

Messages: 13241
Registered: January 2002

Karma: 0

Senior Member
Administrator
Core Developer

Dunno, in all my tests it imported all the data (messages & users).

FUDforum Core Developer

Report message to a moderator

Re: Body mangling in a mailing list [message #14809 is a reply to message #14793]

Fri, 21 November 2003 02:42

Squeebee

Messages: 110
Registered: November 2003

Karma: 0

Senior Member

Well, it dropped a bunch during the consistency check. But as it worked on the second pass I am quite happy.

Report message to a moderator

Re: Body mangling in a mailing list [message #14889 is a reply to message #14793]

Mon, 24 November 2003 20:50

Squeebee

Messages: 110
Registered: November 2003

Karma: 0

Senior Member

Ok, back to the topic that was at hand.

I tried what you suggested as you see in the attachment, but the footer is not going away. Am I implementing your suggestion correctly?

Attachment: mailinglist.jpg
(Size: 397.90KB, Downloaded 790 times)

Report message to a moderator

Re: Body mangling in a mailing list [message #14893 is a reply to message #14889]

Mon, 24 November 2003 21:09

Ilia

Messages: 13241
Registered: January 2002

Karma: 0

Senior Member
Administrator
Core Developer

In my local test on the regular expression it appeard to work correctly...

Plus from what I can see in the picture you didn'r copy the regex correctly, you have _ and ! in it, which I didn't have.

FUDforum Core Developer

Report message to a moderator

Re: Body mangling in a mailing list [message #14895 is a reply to message #14793]

Mon, 24 November 2003 21:23

Squeebee

Messages: 110
Registered: November 2003

Karma: 0

Senior Member

Hmm, odd.

Just to clarify:

"--[\r\n]+MySQL General Mailing List.*"

Goes in the first box (with double quotes)

'm'

Goes in the second box (with quotes)

Third box below has nothing in it.

Right?

Report message to a moderator

Re: Body mangling in a mailing list [message #14897 is a reply to message #14895]

Mon, 24 November 2003 21:58

Ilia

Messages: 13241
Registered: January 2002

Karma: 0

Senior Member
Administrator
Core Developer

correct.

FUDforum Core Developer

Report message to a moderator

Re: Body mangling in a mailing list [message #14898 is a reply to message #14793]

Mon, 24 November 2003 22:02

Squeebee

Messages: 110
Registered: November 2003

Karma: 0

Senior Member

Ok, it is driving me nuts. Here is the line from the mlist table:

SELECT *
FROM `fud26_mlist`

+----+----------+-----------------------+------------------------+----------------------+--------------------------------------------+-------------------+--------------------+-----------+
| id | forum_id | name                  | subject_regex_haystack | subject_regex_needle | body_regex_haystack                        | body_regex_needle | additional_headers | mlist_opt |
+----+----------+-----------------------+------------------------+----------------------+--------------------------------------------+-------------------+--------------------+-----------+
|  1 |       36 | mysql(at)lists(dot)mysql(dot)com |                        |                      | /"--[\r\n]+MySQL General Mailing List.*"/m |                   |                    |        34 |
+----+----------+-----------------------+------------------------+----------------------+--------------------------------------------+-------------------+--------------------+-----------+

But I am not getting the tag stripped.

Report message to a moderator

Re: Body mangling in a mailing list [message #14899 is a reply to message #14898]

Mon, 24 November 2003 22:09

Ilia

Messages: 13241
Registered: January 2002

Karma: 0

Senior Member
Administrator
Core Developer

Hmm try removing the "m" flag, leave the flag window blank.

FUDforum Core Developer

Report message to a moderator

Re: Body mangling in a mailing list [message #14901 is a reply to message #14793]

Mon, 24 November 2003 22:19

Squeebee

Messages: 110
Registered: November 2003

Karma: 0

Senior Member

Thanks for all the help, in return I have paypaled you the measly amount in my PayPal account. It's not much, but hopefully it gets you a cup of coffee for your trouble.

Report message to a moderator

Re: Body mangling in a mailing list [message #14903 is a reply to message #14793]

Mon, 24 November 2003 22:21

Squeebee

Messages: 110
Registered: November 2003

Karma: 0

Senior Member

Oh, and just so you know the removal of the m did not help.

Report message to a moderator

Re: Body mangling in a mailing list [message #14904 is a reply to message #14903]

Mon, 24 November 2003 22:26

Ilia

Messages: 13241
Registered: January 2002

Karma: 0

Senior Member
Administrator
Core Developer

could you send me (As file attachment) 1 complete message from the mailing list and I'll try the regex myself.

FUDforum Core Developer

Report message to a moderator

Re: Body mangling in a mailing list [message #14905 is a reply to message #14793]

Mon, 24 November 2003 22:29

Squeebee

Messages: 110
Registered: November 2003

Karma: 0

Senior Member

Please find attached.

Attachment: listmessage.txt
(Size: 2.08KB, Downloaded 752 times)

Report message to a moderator

Re: Body mangling in a mailing list [message #14906 is a reply to message #14905]

Tue, 25 November 2003 00:53

Ilia

Messages: 13241
Registered: January 2002

Karma: 0

Senior Member
Administrator
Core Developer

Instead of "m" modifier use "s" modifier.

FUDforum Core Developer

Report message to a moderator

Re: Body mangling in a mailing list [message #14915 is a reply to message #14793]

Tue, 25 November 2003 04:50

Squeebee

Messages: 110
Registered: November 2003

Karma: 0

Senior Member

Sorry to be a pain, but I am still meeting with failure. I have tried various combinations of m, s, quotes, no quotes. Still no success. This is not a serious issue for me as the footer is little more than a basic message, but I would like to eventually handle it.

Report message to a moderator

Re: Body mangling in a mailing list [message #14919 is a reply to message #14915]

Tue, 25 November 2003 13:01

Ilia

Messages: 13241
Registered: January 2002

Karma: 0

Senior Member
Administrator
Core Developer

"--[\r\n]+MySQL General Mailing List.*" regex plus "s" modifier work fine, I've tested the code...

FUDforum Core Developer

Report message to a moderator

Re: Body mangling in a mailing list [message #14924 is a reply to message #14793]

Tue, 25 November 2003 13:46

Squeebee

Messages: 110
Registered: November 2003

Karma: 0

Senior Member

Ok I will try again. Would I include the double quotes or not?

Report message to a moderator

Re: Body mangling in a mailing list [message #14925 is a reply to message #14924]

Tue, 25 November 2003 14:00

Ilia

Messages: 13241
Registered: January 2002

Karma: 0

Senior Member
Administrator
Core Developer

ofcourse not.

FUDforum Core Developer

Report message to a moderator

Re: Body mangling in a mailing list [message #14926 is a reply to message #14793]

Tue, 25 November 2003 14:05

Squeebee

Messages: 110
Registered: November 2003

Karma: 0

Senior Member

Hehe, well you never know.

Report message to a moderator

Re: Body mangling in a mailing list [message #14927 is a reply to message #14793]

Tue, 25 November 2003 14:15

Squeebee

Messages: 110
Registered: November 2003

Karma: 0

Senior Member

Anyway, this is what I have in my table under the body haystack:

/--[\n\r]+MySQL General Mailing List/s

It is not working, but I take your word for it that it should work, I will therefore just assume something is awry with something else involved and assume the string provided is good.

Report message to a moderator

Re: Body mangling in a mailing list [message #23901 is a reply to message #14793]

Mon, 04 April 2005 21:20

BobB

Messages: 165
Registered: April 2005
Location: Tucson, AZ

Karma: 0

Senior Member

I can't seem to find an answer to the original question asked in this thread, namely how to strip over-quoting that comes in via a mailing list. That is, stripping anything following ">" or ">>".

Report message to a moderator

Re: Body mangling in a mailing list [message #23912 is a reply to message #23901]

Tue, 05 April 2005 14:12

Ilia

Messages: 13241
Registered: January 2002

Karma: 0

Senior Member
Administrator
Core Developer

Add the "(> )+" replace mask to body mangling.

FUDforum Core Developer

Report message to a moderator

Re: Body mangling in a mailing list [message #24031 is a reply to message #23912]

Sat, 09 April 2005 20:28

BobB

Messages: 165
Registered: April 2005
Location: Tucson, AZ

Karma: 0

Senior Member

Ilia wrote on Tue, 05 April 2005 07:12

Add the "(> )+" replace mask to body mangling.

This only eliminates the >'s themselves, not the text that follows them. Is that possible?

Report message to a moderator

Re: Body mangling in a mailing list [message #24032 is a reply to message #24031]

Sat, 09 April 2005 20:33

Ilia

Messages: 13241
Registered: January 2002

Karma: 0

Senior Member
Administrator
Core Developer

That's the purpose of that regex, just to eliminate things like > > > etc...

FUDforum Core Developer

Report message to a moderator

Re: Body mangling in a mailing list [message #24078 is a reply to message #24032]

Thu, 14 April 2005 00:39

BobB

Messages: 165
Registered: April 2005
Location: Tucson, AZ

Karma: 0

Senior Member

OK, no big deal. I'll continue to use majordomo for that.
Thanks.

Report message to a moderator

Re: Body mangling in a mailing list [message #24681 is a reply to message #24078]

Mon, 09 May 2005 09:42

_2ge_

Messages: 7
Registered: May 2005

Karma: 0

Junior Member

hello, you should use these:

for removing "-- " and after that:

"^--[\s\r]*\n"
"s"
""

for removing ">" lines (you should easily change that to fud quoting)

"^(>\s*)+.*"
"s"
""

Hope works. I didn't try them :)

Report message to a moderator

Previous Topic:	Title for my forum
Next Topic:	Template help

Goto Forum:

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

]

Current Time: Sat Nov 30 04:47:50 GMT 2024

Total time taken to generate the page: 0.02576 seconds