FUDforum
Fast Uncompromising Discussions. FUDforum will get your users talking.

Home » FUDforum » How To » problems with french - charset issue?
Show: Today's Messages :: Polls :: Message Navigator
Switch to threaded view of this topic Create a new topic Submit Reply
problems with french - charset issue? [message #11224] Mon, 23 June 2003 15:32 Go to next message
Tobias Eigen is currently offline  Tobias Eigen   United States
Messages: 85
Registered: June 2003
Location: Seattle, WA USA
Karma: 0
Member
Hi,

A message imported via maillist.php looks strange:

Quote:

Hi Baudouin,

Je serai contente de participer à cette conférence sur RDC, notre beau pays. Veuillez m'envoyer les détails sur cette conférence.

May God bless Congo,

Marie-Claire,


What am I doing wrong? The message looks fine in the regular Mailman archive. I need the standard European character-sets to work on my forum.

BTW, where do users select their preferred language? I don't see this on the profile page.

Cheers,

Tobias


Kabissa - Space for change in Africa
Re: problems with french - charset issue? [message #11225 is a reply to message #11224] Mon, 23 June 2003 16:17 Go to previous messageGo to next message
Ilia is currently offline  Ilia   Canada
Messages: 13241
Registered: January 2002
Karma: 0
Senior Member
Administrator
Core Developer
Users do not select a language, they select a theme, the admin can create themes with different language. So if I decided I want my forum to be in both French & English. I would go to the theme manage and create a new theme using a default template set & the French language. Then the users will have 2 themes to pick from in their profile, default (using english) and name_of_choice (using french).

FUDforum Core Developer
Re: problems with french - charset issue? [message #11227 is a reply to message #11225] Mon, 23 June 2003 16:21 Go to previous messageGo to next message
Tobias Eigen is currently offline  Tobias Eigen   United States
Messages: 85
Registered: June 2003
Location: Seattle, WA USA
Karma: 0
Member
Hi Protoss -

Thanks as always for the quick reply.

I have now figured that out - I have french, english and portuguese configured.

Still I have the problem with French posts being imported incorrectly via maillist.php. Any thoughts on how I can fix this?

Cheers,

Tobias


Kabissa - Space for change in Africa
Re: problems with french - charset issue? [message #11229 is a reply to message #11227] Mon, 23 June 2003 16:56 Go to previous messageGo to next message
Ilia is currently offline  Ilia   Canada
Messages: 13241
Registered: January 2002
Karma: 0
Senior Member
Administrator
Core Developer
Hmm find out what is the charset of the messages you are importing, in all likelyhood they are ISO-8859-1 or ISO-8859-15, however there is a small chance some messages are UTF-8, which is what could be causing the problem.

FUDforum Core Developer
Re: problems with french - charset issue? [message #11231 is a reply to message #11229] Mon, 23 June 2003 17:21 Go to previous messageGo to next message
Tobias Eigen is currently offline  Tobias Eigen   United States
Messages: 85
Registered: June 2003
Location: Seattle, WA USA
Karma: 0
Member
yup - you got it. It's UTF-8. What's the solution?

Cheers,

Tobias


Kabissa - Space for change in Africa
Re: problems with french - charset issue? [message #11233 is a reply to message #11231] Mon, 23 June 2003 17:44 Go to previous messageGo to next message
Ilia is currently offline  Ilia   Canada
Messages: 13241
Registered: January 2002
Karma: 0
Senior Member
Administrator
Core Developer
The solution is to convert the message from UTF-8 to ISO-8859-15, you have the recode (php) extension that's quite easy to do. If not you may need to do some tricks using the recode binary (unix util) to convert the message content's from UTF-8 to ISO-8859-15.

FUDforum Core Developer
Re: problems with french - charset issue? [message #11234 is a reply to message #11233] Mon, 23 June 2003 18:32 Go to previous messageGo to next message
Tobias Eigen is currently offline  Tobias Eigen   United States
Messages: 85
Registered: June 2003
Location: Seattle, WA USA
Karma: 0
Member
ok - this sounds a little complicated. Smile

How common are UTF-8 messages, anyway? Is this going to affect alot of my users?

Is there a way to do this systematically? Ie. by piping the message through another command before piping it to maillist.php?

Cheers,

Tobias


Kabissa - Space for change in Africa
Re: problems with french - charset issue? [message #11235 is a reply to message #11234] Mon, 23 June 2003 18:42 Go to previous messageGo to next message
Ilia is currently offline  Ilia   Canada
Messages: 13241
Registered: January 2002
Karma: 0
Senior Member
Administrator
Core Developer
Depends of the mail client software and what the user has specified and their default charset. I usually don't deal with multi-lingual mailing list so any info I could give you about the frequency of this would be completely theoretical.

FUDforum Core Developer
Re: problems with french - charset issue? [message #11236 is a reply to message #11235] Mon, 23 June 2003 19:00 Go to previous messageGo to next message
Tobias Eigen is currently offline  Tobias Eigen   United States
Messages: 85
Registered: June 2003
Location: Seattle, WA USA
Karma: 0
Member
Hi Protoss -

I just did a search of that mbox file and there are only a few instances.. 2 messages over the course of the last year or so. We may be able to just ignore the problem & when it comes up, suggest to people that they use a different encoding.

I was struck that mhonarc handled it ok while maillist.php did not - is it not possible to build in support for UTF-8?

Cheers,

tobias


Kabissa - Space for change in Africa
Re: problems with french - charset issue? [message #11237 is a reply to message #11236] Mon, 23 June 2003 19:08 Go to previous messageGo to next message
Ilia is currently offline  Ilia   Canada
Messages: 13241
Registered: January 2002
Karma: 0
Senior Member
Administrator
Core Developer
It is possible, but that would require adding a depency on an extremely rare php extension not avaliable to most people.

FUDforum Core Developer
Re: problems with french - charset issue? [message #11983 is a reply to message #11237] Thu, 24 July 2003 16:35 Go to previous messageGo to next message
Micha37 is currently offline  Micha37   Germany
Messages: 77
Registered: October 2002
Karma: 0
Member
Hi Prottoss,

since importing a German Newsgroup, I am facing the same issue as you can see at http://www.diabetesring.de/Forum/index.php?t=tree&th=3907&mid=23709 &rid=128

It would be nice, if you could think over some solution.

Best regards

Michael
Re: problems with french - charset issue? [message #11986 is a reply to message #11983] Thu, 24 July 2003 17:32 Go to previous messageGo to next message
Ilia is currently offline  Ilia   Canada
Messages: 13241
Registered: January 2002
Karma: 0
Senior Member
Administrator
Core Developer
French & German charsets both use ISO-8859-15 and are generally compatible. However, I suspect the newsgroup you've imported the messages from used an incompatible UTF-8 charset. The solution is to use the iconv extension to convert messages from UTF-8 to ISO-8859-15 before importing them into the forum.

FUDforum Core Developer
Re: problems with french - charset issue? [message #12008 is a reply to message #11986] Thu, 24 July 2003 19:43 Go to previous messageGo to next message
Micha37 is currently offline  Micha37   Germany
Messages: 77
Registered: October 2002
Karma: 0
Member
Hi Prottoss,

thanks for the hint but how do this? Is FUDforum offering this function or do you know any hack, which can do the thing??

Thanks for any advise.

Best regards


Michael
Re: problems with french - charset issue? [message #12010 is a reply to message #12008] Thu, 24 July 2003 19:46 Go to previous messageGo to next message
Ilia is currently offline  Ilia   Canada
Messages: 13241
Registered: January 2002
Karma: 0
Senior Member
Administrator
Core Developer
FUDforum does not offer this functionality because it would result in a depenecy on iconv extension that is rarely avalaible. Fortunately implementing iconv is very easy so if you have the extension it is simply a matter of adding something like this:
<?php
$text
= iconv("UTF-8", "ISO-8859-15", $text);
?>

inside the import script.


FUDforum Core Developer
Re: problems with french - charset issue? [message #12011 is a reply to message #12010] Thu, 24 July 2003 19:57 Go to previous messageGo to next message
Micha37 is currently offline  Micha37   Germany
Messages: 77
Registered: October 2002
Karma: 0
Member
It seems that my PHP installation is offering the iconv extension. Do you have any suggestion at which place of the script I should place the hack.

Thanks in advance for the help.

Michael
Re: problems with french - charset issue? [message #12013 is a reply to message #12011] Thu, 24 July 2003 20:17 Go to previous messageGo to next message
Ilia is currently offline  Ilia   Canada
Messages: 13241
Registered: January 2002
Karma: 0
Senior Member
Administrator
Core Developer
That's a little tricky, because you actually can't assume that every message will have the same encoding, the actual encoding is specified by the mail client. So you need to use the headers array to determine the character set of the message, and if it is not ISO-8859-1 and ISO-8859-15 convert it to ISO-8859-15.

The two variables you need to convert are:
$emsg->body
$emsg->subject


FUDforum Core Developer
Re: problems with french - charset issue? [message #12016 is a reply to message #12013] Thu, 24 July 2003 20:22 Go to previous message
Micha37 is currently offline  Micha37   Germany
Messages: 77
Registered: October 2002
Karma: 0
Member
Hi Prottoss,

thanks for the reply but that sounds too tricky for me. I am just happy that I got the nntp script to run, so I am not able to programm tricky routines.

Anyway, its just a few postings like that, so never mind.

Best regards

Michael
  Switch to threaded view of this topic Create a new topic Submit Reply
Previous Topic: showing only one category at a time
Next Topic: Tips on finding out users' identities
Goto Forum:
  

-=] Back to Top [=-
[ Syndicate this forum (XML) ] [ RSS ]

Current Time: Sat Nov 23 02:00:13 GMT 2024

Total time taken to generate the page: 0.02938 seconds