FUDforum: How To » NNTP Issue (New User)

Home » FUDforum » How To » NNTP Issue (New User) (New User)

Show: Today's Messages :: Polls :: Message Navigator

NNTP Issue (New User) [message #162382]

Wed, 05 May 2010 00:44

mmiller

Messages: 27
Registered: May 2010

Karma: 0

Junior Member

Hi Folks!

New user here, just finished doing an install. I've used IPB 1.3.1 for close to a decade now and first impressions with FUD are very good!

My issue is NNTP. I created a proper rule and now want to get it to retrieve posts.

I edited the first line in nntp.php to read like this;

#!C:\Progra~1\PHP -q

That's my path to the PHP executable. Is it OK? (programmatically)

My next issue is, how do I trigger an update? The docs don't show anything for Windows/IIS. They just mention Task Scheduler.

Sorry for the newby questions - can anyone help?

Report message to a moderator

NNTP Issue [message #162384 is a reply to message #162382]

Wed, 05 May 2010 00:51

mmiller

Messages: 27
Registered: May 2010

Karma: 0

Junior Member

Hi Folks!

Brand new user here Shocked

Shocked

I created an NNTP profile and changed the first line in nntp.php to;

#!C:\Progra~1\PHP -q

Is that OK? (it is the right path to my PHP directory)

The next question is how exactly do I trigger an update? The docs show a Cron job (for Unix) but nothing for Windows. I understand that I need to use task manager but what exactly am I trying to run?

Thanks!

Report message to a moderator

Re: NNTP Issue [message #162394 is a reply to message #162384]

Wed, 05 May 2010 06:57

naudefj

Messages: 3771
Registered: December 2004

Karma: 28

Senior Member
Administrator
Core Developer

See Windows example at http://www.php-cli.com/php-cli-tutorial.shtml

Feel free to expand the documentation at Newsgroup Manager

Report message to a moderator

Re: NNTP Issue [message #162403 is a reply to message #162394]

Wed, 05 May 2010 18:50

mmiller

Messages: 27
Registered: May 2010

Karma: 0

Junior Member

Thank you very much - I managed to get it working from the command line and as soon as figure out how to schedule it with Task Scheduler I'll be happy to update the docs accordingly as it's no difficult thing Smile

Smile

Question though, during testing I began to pull down a newsgroup and after some time got this error;

Importing microsoft.public.windowsxp.general message 2037635
Importing microsoft.public.windowsxp.general message 2037636
(C:\Inetpub\fudforum\include\theme\default\db.inc:72<br />
C:\Inetpub\fudforum\include\theme\default\db.inc:176<br />
C:\Inetpub\fudforum\include\theme\default\imsg_edt.inc:112<br />
C:\Inetpub\fudforum\include\nntp.inc:577<br />
C:\Inetpub\fudforum\scripts\nntp.php:94<br />
) 1366: Incorrect string value: '\xBB Alte...' for column 'subject' at row 1<br
/>
Query: INSERT INTO fud30_msg (
thread_id,
poster_id,
reply_to,
ip_addr,
host_name,
post_stamp,
subject,
attach_cnt,
poll_id,
icon,
msg_opt,
file_id,
foff,
length,
file_id_preview,
offset_preview,
length_preview,
mlist_msg_id,
poll_cache,
flag_cc,
flag_country
) VALUES(
274,
350,
1662,
'79.78.172.66',
NULL,
1266356162,
'Re: Sound Schemes ╗ Altering, etc. (XP Professional)',
0,
0,
NULL,
3,
20000,
6631745,
1915,
0,
0,
0,
'uVL93$0rKHA(dot)3944(at)TK2MSFTNGP06(dot)phx(dot)gbl',
NULL,
NULL,
NULL
)<br />
Server Version: 5.1.42-community<br />

Any ideas on why or how to avoid it?

Report message to a moderator

Re: NNTP Issue [message #162404 is a reply to message #162403]

Wed, 05 May 2010 19:13

naudefj

Messages: 3771
Registered: December 2004

Karma: 28

Senior Member
Administrator
Core Developer

One of the messages you are importing has a control character in the subject line that MySQL doesn't like. You may want to change the encoding of the table to accept it or filter it our in the PHP script.

Report message to a moderator

Re: NNTP Issue [message #162405 is a reply to message #162404]

Wed, 05 May 2010 20:12

mmiller

Messages: 27
Registered: May 2010

Karma: 0

Junior Member

Hmmm.... the easiest way (for me) would be to change the encoding of the table in MySQL to accept it.

What table would that be and what should the encoding be changed to?

Report message to a moderator

Re: NNTP Issue (New User) [message #162409 is a reply to message #162382]

Thu, 06 May 2010 01:33

mmiller

Messages: 27
Registered: May 2010

Karma: 0

Junior Member

Alternatively, how should the code be changed?

This issue is stopping me from importing data from almost every feed I set up Sad

Sad

Report message to a moderator

Re: NNTP Issue (New User) [message #162412 is a reply to message #162409]

Thu, 06 May 2010 05:38

naudefj

Messages: 3771
Registered: December 2004

Karma: 28

Senior Member
Administrator
Core Developer

Run the forum's MySQL Charset Changer to change the encoding of your tables to UTF-8. If they are already UTF-8, we can fix the script.

PS: I would need a test case or copy of the message to prepare a fix.

Best regards.

Frank

Report message to a moderator

Re: NNTP Issue (New User) [message #162414 is a reply to message #162382]

Thu, 06 May 2010 06:15

mmiller

Messages: 27
Registered: May 2010

Karma: 0

Junior Member

Remarkable - being able to change the character set through the forum admin! I find a lot of things about this forum very interesting.

I checked the properties of fud30_index and found that it is already UTF8 Shocked

Shocked

I guess that means a programmatic solution?

The news pull stopped again and I'm sure it was also due to a non-standard character in the subject line. This time it might be a Spanish post that caused it to cough.

A search on Google for some of the text (Id� del suceso: 36871) seems to show that it was a Spanish post.

That caused me to think, maybe he's using a spanish-specific character such as an upside down exclamation mark or a letter with a hat Laughing

Laughing

So, my theory is, if a foreigner made a post with a subject line that used non-standard characters maybe that will also cause the NNTP pull to cough up an error and stop?

This is the most recent error that seems to indicate a Spanish post;

Importing microsoft.public.exchange2000.general message 7475
(C:\Inetpub\fudforum\include\theme\default\db.inc:72
C:\Inetpub\fudforum\include\theme\default\db.inc:176
C:\Inetpub\fudforum\include\theme\default\imsg_edt.inc:112
C:\Inetpub\fudforum\include\nntp.inc:577
C:\Inetpub\fudforum\scripts\nntp.php:94

) 1366: Incorrect string value: '\xB7 del ...' for column 'subject' at row 1

Query: INSERT INTO fud30_msg (
thread_id,
poster_id,
reply_to,
ip_addr,
host_name,
post_stamp,
subject,
attach_cnt,
poll_id,
icon,
msg_opt,
file_id,
foff,
length,
file_id_preview,
offset_preview,
length_preview,
mlist_msg_id,
poll_cache,
flag_cc,
flag_country
) VALUES(
0,
2776,
0,
'200.16.138.46',
NULL,
1134395206,
'Id� del suceso: 36871',
0,
0,
NULL,
3,
110000,
13212695,
428,
0,
0,
0,
'OidmbJy$FHA(dot)3944(at)TK2MSFTNGP10(dot)phx(dot)gbl',
NULL,
NULL,
NULL
)

When I check Google for that post it shows many similar posts - all of them in Spanish.

Best;
Marvin

Report message to a moderator

Re: NNTP Issue (New User) [message #162415 is a reply to message #162382]

Thu, 06 May 2010 06:19

mmiller

Messages: 27
Registered: May 2010

Karma: 0

Junior Member

Hmmmm... found another one;

'Outlook doesn�t work'

I would assume that one was just an apostrophe?

Report message to a moderator

Re: NNTP Issue (New User) [message #162416 is a reply to message #162415]

Thu, 06 May 2010 15:56

naudefj

Messages: 3771
Registered: December 2004

Karma: 28

Senior Member
Administrator
Core Developer

Doesn't help me much. As explained above, I need a test case or copy of the message to prepare a fix.

Report message to a moderator

Re: NNTP Issue (New User) [message #162423 is a reply to message #162382]

Sat, 08 May 2010 02:49

mmiller

Messages: 27
Registered: May 2010

Karma: 0

Junior Member

OK, I got one Laughing

Laughing

This is the error;

(C:\Inetpub\fudforum\include\theme\default\db.inc:72
C:\Inetpub\fudforum\include\theme\default\db.inc:176
C:\Inetpub\fudforum\include\theme\default\imsg_edt.inc:112
C:\Inetpub\fudforum\include\nntp.inc:577
C:\Inetpub\fudforum\scripts\nntp.php:94

) 1366: Incorrect string value: '\xBB Alte...' for column 'subject' at row 1

'Re: Sound Schemes � Altering, etc. (XP Professional)',

and this is the message (note the arrows in the subject)

Subject: Re: Sound Schemes » Altering, etc. (XP Professional)
Did you save the sound scheme after you'd modified it?

"thricipio" <thricipio(at)discussions(dot)microsoft(dot)com[/email]> wrote in message
news:1BEFD643-F393-487E-8B9C-2F81B39A23FF(at)microsoft(dot)com...[/email]
> So. I've been trying to do something on XP Pro, that I know I've done
> before
> with no great difficulty on XP Home: that is, change a couple of the
> sound
> scheme settings - specifically, the "Windows Logoff" and "Windows Logon"
> settings.
>
> On XP Home, I recall having to go into the "Sounds and Audio Devices
> Properties" dialogue and selecting the desired item (e.g., Windows Logoff)
> and then using the [Browse] button to find the desired .wav file. I can
> do
> that, and I can check that I have the correct .wav file by clicking the
> [>]
> button. So far, so good. But when I actually Logoff/on, I don't get the
> newly selected .wav files.
>
> I've even renamed the desired .wav files to "Windows Logoff(/on)" and
> placed
> them in the default directory {<root_drive>\Windows\Media} and tested them
> with the [>] button. And that works (as one would expect). But again,
> when
> I logoff and logon, I still get the default Win logoff/on .wav file
> sounds.
>
> Obviously, I'm missing something basic here. just can't figure out what
> that
> is. Any help you can give will be appreciated. Thanks.
>
> Color me stymied,
> -Thri

[Updated on: Sat, 08 May 2010 02:54]

Report message to a moderator

Re: NNTP Issue (New User) [message #162426 is a reply to message #162423]

Sat, 08 May 2010 03:16

naudefj

Messages: 3771
Registered: December 2004

Karma: 28

Senior Member
Administrator
Core Developer

I actually need a full message with complete headers to test with. I'll try to construct one from your post, however, I suspect it won't help as the Subject line is encoded.

Is this message available on a public newsgroup that I can use for testing? If so, what is the NNTP server name and message number?

Report message to a moderator

Re: NNTP Issue (New User) [message #162428 is a reply to message #162426]

Sat, 08 May 2010 06:23

mmiller

Messages: 27
Registered: May 2010

Karma: 0

Junior Member

Hi Frank;

It came from news server msnews.microsoft.com
The group was microsoft.public.windowsxp.general
It was posted on Feb 16, 2010 by thricipio

Here's a link? to it;
news:1BEFD643-F393-487E-8B9C-2F81B39A23FF(at)microsoft(dot)com

Hopefully that will help?
The one thing I'm concerned about though is that it seems any message with goofy characters in the subject line causes the pull to fail.
So I'm thinking that if someone posts in Spanish or Japanese or something like that will also cause it to fail?
I'm just guessing in an effort to try and help Smile

Smile

Best & Thanks!
Marvin

Report message to a moderator

Re: NNTP Issue (New User) [message #162429 is a reply to message #162428]

Sat, 08 May 2010 07:59

naudefj

Messages: 3771
Registered: December 2004

Karma: 28

Senior Member
Administrator
Core Developer

Great this is exactly what I need. Looking at the message's source, the subject is:

Subject: =?Utf-8?Q?Sound_Schemes_=C2=BB_Altering=2C_etc._=28?=
=?Utf-8?Q?XP_Professional=29?=

Report message to a moderator

Re: NNTP Issue (New User) [message #162430 is a reply to message #162429]

Sat, 08 May 2010 13:32

naudefj

Messages: 3771
Registered: December 2004

Karma: 28

Senior Member
Administrator
Core Developer

I'm unable to reproduce the problem. I've tried on my Linux box and on Windows with FUD2Go. Both systems were able to import the message without a problem.

To proceed, we need to do some debugging. Please let me know if you would prefer 1 or 2 below:

1) If the forum is on the Net, create an admin user for me and mail me the password.

2) I can prepare a debugging patch for you. You will need to apply it, run the import and report back on the output.

Report message to a moderator

Re: NNTP Issue (New User) [message #162431 is a reply to message #162430]

Sat, 08 May 2010 16:51

mmiller

Messages: 27
Registered: May 2010

Karma: 0

Junior Member

Hi Frank;

Number 2 would be easier for me as the site is not available from the 'net yet.

Report message to a moderator

Re: NNTP Issue (New User) [message #162433 is a reply to message #162431]

Sun, 09 May 2010 09:12

naudefj

Messages: 3771
Registered: December 2004

Karma: 28

Senior Member
Administrator
Core Developer

For starters, open include/nntp.inc, and search for:
$this->subject = htmlspecialchars(trim(decode_header_value($this->headers['subject'])));

Replace this line with:

echo "FORUM CHARSET=[". $GLOBALS['CHARSET'] ."]\n";
echo "RAW SUBJ=[". $this->subject ."]\n";
$this->subject = htmlspecialchars(trim(decode_header_value($this->headers['subject'])));
echo "NEW SUBJ=[". $this->subject ."]\n";
die();

Reset the NNTP tracker to the problematic message by running this statement from the SQL Manager:
UPDATE fud30_nntp SET tracker = 2037610 WHERE id = 1;

Run the import script and you should see something like this:

Importing microsoft.public.windowsxp.general message 2037610
FORUM CHARSET=[utf-8]
RAW SUBJ=[=?Utf-8?Q?Sound_Schemes_=C2=BB_Altering=2C_etc._=28?= =?Utf-8?Q?XP_Professional=29?=]
NEW SUBJ=[Sound Schemes » Altering, etc. (XP Professional)]

You may also want to add extra echo statements to decode_header_value() in include/scripts_common.inc.

Report message to a moderator

Re: NNTP Issue (New User) [message #162434 is a reply to message #162433]

Sun, 09 May 2010 17:12

mmiller

Messages: 27
Registered: May 2010

Karma: 0

Junior Member

Hi Frank;

I did as you asked and this is what I got when I ran it;

Output of file C:/Inetpub//fudforum/scripts/nntp_1.log:
Importing microsoft.public.windowsxp.general message 2037610
FORUM CHARSET=[utf-8]
RAW SUBJ=[]
NEW SUBJ=[Sound Schemes » Altering, etc. (XP Professional)]

Does that help?

Report message to a moderator

Previous Topic:	Can't Log in to Admin
Next Topic:	PDF and code page

Goto Forum:

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

]

Current Time: Thu Oct 31 23:59:33 GMT 2024

Total time taken to generate the page: 0.04714 seconds