| Output status during processing [message #179527] |
Sun, 04 November 2012 19:23  |
Scott Johnson
Messages: 196
Registered: January 2012
Karma: 0
|
Senior Member |
|
|
Hello
I hate to come here with a question without any tried code but I really
do not even know where to start, I have never attempted this before
without a net.
What I would like to do is provide an output to the browser during a
long processing term.
Exactly what I am doing is this.
I am loading a csv file to update inventory.
During this process I am flushing the current Table and then loading the
new and then comparing the inventory table to a product table and would
like to provide status of where we are in the process what items matched
and changed and so forth in real time.
I have done some similar tasks in JS but with minimal steps in the
processing then what I am trying to accomplish.
I did some binging and found that it may have to do with output
buffering but nothing particular to my situation. (as far as I could find)
If this is possible could someone point me in the right direction and I
will do the leg work.
Thanks
Scotty
|
|
|
|
| Re: Output status during processing [message #179529 is a reply to message #179527] |
Sun, 04 November 2012 21:28  |
M. Strobel
Messages: 386
Registered: December 2011
Karma: 0
|
Senior Member |
|
|
Am 04.11.2012 20:23, schrieb Scott Johnson:
> Hello
>
> I hate to come here with a question without any tried code but I really do not even
> know where to start, I have never attempted this before without a net.
>
> What I would like to do is provide an output to the browser during a long processing
> term.
>
> Exactly what I am doing is this.
>
> I am loading a csv file to update inventory.
>
> During this process I am flushing the current Table and then loading the new and then
> comparing the inventory table to a product table and would like to provide status of
> where we are in the process what items matched and changed and so forth in real time.
>
> I have done some similar tasks in JS but with minimal steps in the processing then
> what I am trying to accomplish.
>
> I did some binging and found that it may have to do with output buffering but nothing
> particular to my situation. (as far as I could find)
>
> If this is possible could someone point me in the right direction and I will do the
> leg work.
Are you starting your long running job in the browser? Not good, the whole setup is
designed for request-response, and not long running jobs.
I would try to design the long running update-compare job as a separate process,
writing status or progress information to a file/a db table. Then you have a
corresponding script you call in your browser, showing you status and progress. This
could be done with whole page refreshs or with ajax.
/Str.
|
|
|
|
| Re: Output status during processing [message #179530 is a reply to message #179529] |
Sun, 04 November 2012 21:52 |
Scott Johnson
Messages: 196
Registered: January 2012
Karma: 0
|
Senior Member |
|
|
On 11/4/2012 1:28 PM, M. Strobel wrote:
> Am 04.11.2012 20:23, schrieb Scott Johnson:
>> Hello
>>
>> I hate to come here with a question without any tried code but I really do not even
>> know where to start, I have never attempted this before without a net.
>>
>> What I would like to do is provide an output to the browser during a long processing
>> term.
>>
>> Exactly what I am doing is this.
>>
>> I am loading a csv file to update inventory.
>>
>> During this process I am flushing the current Table and then loading the new and then
>> comparing the inventory table to a product table and would like to provide status of
>> where we are in the process what items matched and changed and so forth in real time.
>>
>> I have done some similar tasks in JS but with minimal steps in the processing then
>> what I am trying to accomplish.
>>
>> I did some binging and found that it may have to do with output buffering but nothing
>> particular to my situation. (as far as I could find)
>>
>> If this is possible could someone point me in the right direction and I will do the
>> leg work.
>
> Are you starting your long running job in the browser? Not good, the whole setup is
> designed for request-response, and not long running jobs.
>
> I would try to design the long running update-compare job as a separate process,
> writing status or progress information to a file/a db table. Then you have a
> corresponding script you call in your browser, showing you status and progress. This
> could be done with whole page refreshs or with ajax.
>
> /Str.
>
Ok that makes makes sense.
And yes it was started in the browser on a file upload from a form field.
I could use AJAX to load the file and then process it in the background
sending status points as it goes through AJAX.
I have done this is the past just not for so many steps to perform.
The more I read about the PHP buffer functions, I could not find a way
that would periodically send text without a header issue.
Guess why they have different tools for the different needs.
Thanks Str.
Scotty
|
|
|
|
| Re: Output status during processing [message #179531 is a reply to message #179530] |
Mon, 05 November 2012 01:21 |
Jerry Stuckle
Messages: 2598
Registered: September 2010
Karma: 0
|
Senior Member |
|
|
On 11/4/2012 4:52 PM, Scott Johnson wrote:
> On 11/4/2012 1:28 PM, M. Strobel wrote:
>> Am 04.11.2012 20:23, schrieb Scott Johnson:
>>> Hello
>>>
>>> I hate to come here with a question without any tried code but I
>>> really do not even
>>> know where to start, I have never attempted this before without a net.
>>>
>>> What I would like to do is provide an output to the browser during a
>>> long processing
>>> term.
>>>
>>> Exactly what I am doing is this.
>>>
>>> I am loading a csv file to update inventory.
>>>
>>> During this process I am flushing the current Table and then loading
>>> the new and then
>>> comparing the inventory table to a product table and would like to
>>> provide status of
>>> where we are in the process what items matched and changed and so
>>> forth in real time.
>>>
>>> I have done some similar tasks in JS but with minimal steps in the
>>> processing then
>>> what I am trying to accomplish.
>>>
>>> I did some binging and found that it may have to do with output
>>> buffering but nothing
>>> particular to my situation. (as far as I could find)
>>>
>>> If this is possible could someone point me in the right direction and
>>> I will do the
>>> leg work.
>>
>> Are you starting your long running job in the browser? Not good, the
>> whole setup is
>> designed for request-response, and not long running jobs.
>>
>> I would try to design the long running update-compare job as a
>> separate process,
>> writing status or progress information to a file/a db table. Then you
>> have a
>> corresponding script you call in your browser, showing you status and
>> progress. This
>> could be done with whole page refreshs or with ajax.
>>
>> /Str.
>>
>
> Ok that makes makes sense.
>
> And yes it was started in the browser on a file upload from a form field.
>
> I could use AJAX to load the file and then process it in the background
> sending status points as it goes through AJAX.
>
> I have done this is the past just not for so many steps to perform.
>
> The more I read about the PHP buffer functions, I could not find a way
> that would periodically send text without a header issue.
>
> Guess why they have different tools for the different needs.
>
> Thanks Str.
>
> Scotty
>
>
Nope, but it has nothing to do with php. HTTP is a request/response
protocol; the client sends a request and expects a response within the
time specified by the browser's timeout value.
But then this causes a problem also, because PHP has a buffer (for
efficiency), the server has a buffer (also for efficiency), routers
along the way may have buffers (again, for efficiency), and the client's
system may have a buffer (hmmm... efficiency?) and finally the browser
may have a buffer.
The bottom line is - the HTTP protocol is pretty much useless for
long-running tasks. The time it takes to send data to the browser is
dependent on a lot of things, and the browser's timeout values are
independent of anything the server can control.
If you need such processing, I suggest you look into other methods of
communications. Java applets, for instance, can work with non-http
protocols.
--
==================
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.
jstucklex(at)attglobal(dot)net
==================
|
|
|
|
| Re: Output status during processing [message #179532 is a reply to message #179531] |
Mon, 05 November 2012 03:10 |
Scott Johnson
Messages: 196
Registered: January 2012
Karma: 0
|
Senior Member |
|
|
On 11/4/2012 5:21 PM, Jerry Stuckle wrote:
> On 11/4/2012 4:52 PM, Scott Johnson wrote:
>> On 11/4/2012 1:28 PM, M. Strobel wrote:
>>> Am 04.11.2012 20:23, schrieb Scott Johnson:
>>>> Hello
>>>>
>>>> I hate to come here with a question without any tried code but I
>>>> really do not even
>>>> know where to start, I have never attempted this before without a net.
>>>>
>>>> What I would like to do is provide an output to the browser during a
>>>> long processing
>>>> term.
>>>>
>>>> Exactly what I am doing is this.
>>>>
>>>> I am loading a csv file to update inventory.
>>>>
>>>> During this process I am flushing the current Table and then loading
>>>> the new and then
>>>> comparing the inventory table to a product table and would like to
>>>> provide status of
>>>> where we are in the process what items matched and changed and so
>>>> forth in real time.
>>>>
>>>> I have done some similar tasks in JS but with minimal steps in the
>>>> processing then
>>>> what I am trying to accomplish.
>>>>
>>>> I did some binging and found that it may have to do with output
>>>> buffering but nothing
>>>> particular to my situation. (as far as I could find)
>>>>
>>>> If this is possible could someone point me in the right direction and
>>>> I will do the
>>>> leg work.
>>>
>>> Are you starting your long running job in the browser? Not good, the
>>> whole setup is
>>> designed for request-response, and not long running jobs.
>>>
>>> I would try to design the long running update-compare job as a
>>> separate process,
>>> writing status or progress information to a file/a db table. Then you
>>> have a
>>> corresponding script you call in your browser, showing you status and
>>> progress. This
>>> could be done with whole page refreshs or with ajax.
>>>
>>> /Str.
>>>
>>
>> Ok that makes makes sense.
>>
>> And yes it was started in the browser on a file upload from a form field.
>>
>> I could use AJAX to load the file and then process it in the background
>> sending status points as it goes through AJAX.
>>
>> I have done this is the past just not for so many steps to perform.
>>
>> The more I read about the PHP buffer functions, I could not find a way
>> that would periodically send text without a header issue.
>>
>> Guess why they have different tools for the different needs.
>>
>> Thanks Str.
>>
>> Scotty
>>
>>
>
> Nope, but it has nothing to do with php. HTTP is a request/response
> protocol; the client sends a request and expects a response within the
> time specified by the browser's timeout value.
>
> But then this causes a problem also, because PHP has a buffer (for
> efficiency), the server has a buffer (also for efficiency), routers
> along the way may have buffers (again, for efficiency), and the client's
> system may have a buffer (hmmm... efficiency?) and finally the browser
> may have a buffer.
>
> The bottom line is - the HTTP protocol is pretty much useless for
> long-running tasks. The time it takes to send data to the browser is
> dependent on a lot of things, and the browser's timeout values are
> independent of anything the server can control.
>
> If you need such processing, I suggest you look into other methods of
> communications. Java applets, for instance, can work with non-http
> protocols.
>
Yeah I can see there can be a bottle-neck/lag issue anywhere which would
be useless for anything real time (which I use loosely).
Thanks for the input.
Scotty
|
|
|
|
| Re: Output status during processing [message #179533 is a reply to message #179530] |
Mon, 05 November 2012 12:40 |
Denis McMahon
Messages: 634
Registered: September 2010
Karma: 0
|
Senior Member |
|
|
On Sun, 04 Nov 2012 13:52:35 -0800, Scott Johnson wrote:
> I could use AJAX to load the file and then process it in the background
> sending status points as it goes through AJAX.
Back up a step.
The server can't "send" status updates to the browser using ajax in the
normal case, rather the browser would request "request" status updates
from the server, usually in relation to some browser event (which might
be a javascript timer timeout, or the user clicking a "get update" button
or link).
Ajax, which is built on the http request / response, which is in turn
built on a tcp/ip connection, is not a technology which generally allows
the server to push unsolicited data to the browser, rather it is a
technology that allows the browser to make requests to the server and
receive data back without reloading the page being displayed.
Specifically, when you make an http request to a server, a tcp connection
is opened to the server. The connection is usually closed after the
server has sent the response. Making a connection to the server is easy,
because tcp/ip knows that the server is listening to http on port 80. In
the other direction, there is no official port that a browser listens out
on for a server to make a connection to it, which makes it difficult for
the server to open a connection to the browser to just send it something.
I know it may sound like a petty distinction to make, but making the
distinction helps you keep it clear in your mind how it all fits
together, and in the concept of php running behind a web-server, the
context is generally servicing and responding to requests from the
browser, not pushing data to the browser.
Rgds
Denis McMahon
|
|
|
|
| Re: Output status during processing [message #179534 is a reply to message #179527] |
Mon, 05 November 2012 13:22 |
Thomas 'PointedEars'
Messages: 701
Registered: October 2010
Karma: 0
|
Senior Member |
|
|
Scott Johnson wrote:
> What I would like to do is provide an output to the browser during a
> long processing term.
>
> Exactly what I am doing is this.
>
> I am loading a csv file to update inventory.
>
> During this process I am flushing the current Table and then loading the
> new and then comparing the inventory table to a product table and would
> like to provide status of where we are in the process what items matched
> and changed and so forth in real time.
>
> I have done some similar tasks in JS but with minimal steps in the
> processing then what I am trying to accomplish.
>
> I did some binging and found that it may have to do with output
> buffering but nothing particular to my situation. (as far as I could find)
>
> If this is possible could someone point me in the right direction and I
> will do the leg work.
Contrary to popular belief, it is possible with PHP alone to provide step-
by-step feedback to the client (as well as to a server-side log file) while
a server-side script is being executed.
About one year ago, I have done something very similar to what you want to
do: I have written a PHP script to import product information from a CSV
file into a Magento eCommerce database (in a way that Magento's built-in
import feature was not capable of at the time).
Here is basically what I have done:
1. Enable PHP's output buffering (ob_start()).
2. Process a record in the CSV (with fgetcsv() and the Magento API),
writing status information to the server-side log file and (optionally)
to the standard output (with file_put_contents() and `echo',
respectively).
3. Flush and clear the output buffer:
/**
* Flushes all output buffers and restarts output buffering
*
* @author sebastian(at)jcompare(dot)com
* @link http://php.net/manual/en/function.ob-flush.php
*/
function flush_buffers()
{
ob_end_flush();
ob_flush();
flush();
ob_start();
}
That displays what is written in step 2 in the console or the browser
window (depending on how the script is run). In order to have the last
row visible, client-side scripting is used to scroll down to it in
regular intervals.
4. Continue with step 2 if there are more records or files;
exit successfully otherwise.
That has worked beautifully both with text/plain and text/html.
The output-buffering approach does _not_ provide you with the ability to
update arbitrary parts of an (X)HTML document, though; it would create the
response document content on the fly instead. If you need arbitrary display
updates, you should look into asynchronous XMLHttpRequest, with which you
would poll status information from the server accessing a (PHP-)generated
resource in regular intervals, and update your (X)HTML document accordingly.
(The output-buffering approach was adequate and sufficient in my case; the
XHR approach was and is employed by the Magento UI but was not sufficient
for the used data format at the time.)
Also, you should be aware that no matter how you do it, neither a database
connection is likely to be stable indefinitely nor a PHP script is likely to
run forever. In particular, on the server my script ran, the import was
supposed to run regularly (from CSV files that were uploaded by the customer
at arbitrary times, with the provision that the shop information needed to
be as up-to-date as possible). IOW, a cron job was set up which started PHP
CLI to run the PHP script if it not already ran.
But because a CSV file could have had many (thousands of) records (read:
product variants), and there were several such files, it was not possible to
import all records from all files in one run. Either the database
connection used by the Magento API was severed or the PHP script would be
terminated early (because of max_execution_time in php.ini, which for
security reasons could not be set to 0 for the Web version that was still
needed for "manual" import).
I have solved this with storing the position of the last import in another
file, so that when the script was executed next time it would check if a
previous position was saved and continue from there, or it would continue
from the beginning of the file(s).
HTH
PointedEars
--
When all you know is jQuery, every problem looks $(olvable).
|
|
|
|
| Re: Output status during processing [message #179535 is a reply to message #179532] |
Mon, 05 November 2012 14:24 |
crankypuss
Messages: 147
Registered: March 2011
Karma: 0
|
Senior Member |
|
|
On 11/04/2012 08:10 PM, Scott Johnson wrote:
> On 11/4/2012 5:21 PM, Jerry Stuckle wrote:
>> On 11/4/2012 4:52 PM, Scott Johnson wrote:
>>> On 11/4/2012 1:28 PM, M. Strobel wrote:
>>>> Am 04.11.2012 20:23, schrieb Scott Johnson:
>>>> > Hello
>>>> >
>>>> > I hate to come here with a question without any tried code but I
>>>> > really do not even
>>>> > know where to start, I have never attempted this before without a net.
>>>> >
>>>> > What I would like to do is provide an output to the browser during a
>>>> > long processing
>>>> > term.
>>>> >
>>>> > Exactly what I am doing is this.
>>>> >
>>>> > I am loading a csv file to update inventory.
>>>> >
>>>> > During this process I am flushing the current Table and then loading
>>>> > the new and then
>>>> > comparing the inventory table to a product table and would like to
>>>> > provide status of
>>>> > where we are in the process what items matched and changed and so
>>>> > forth in real time.
>>>> >
>>>> > I have done some similar tasks in JS but with minimal steps in the
>>>> > processing then
>>>> > what I am trying to accomplish.
>>>> >
>>>> > I did some binging and found that it may have to do with output
>>>> > buffering but nothing
>>>> > particular to my situation. (as far as I could find)
>>>> >
>>>> > If this is possible could someone point me in the right direction and
>>>> > I will do the
>>>> > leg work.
>>>>
>>>> Are you starting your long running job in the browser? Not good, the
>>>> whole setup is
>>>> designed for request-response, and not long running jobs.
>>>>
>>>> I would try to design the long running update-compare job as a
>>>> separate process,
>>>> writing status or progress information to a file/a db table. Then you
>>>> have a
>>>> corresponding script you call in your browser, showing you status and
>>>> progress. This
>>>> could be done with whole page refreshs or with ajax.
>>>>
>>>> /Str.
>>>>
>>>
>>> Ok that makes makes sense.
>>>
>>> And yes it was started in the browser on a file upload from a form
>>> field.
>>>
>>> I could use AJAX to load the file and then process it in the background
>>> sending status points as it goes through AJAX.
>>>
>>> I have done this is the past just not for so many steps to perform.
>>>
>>> The more I read about the PHP buffer functions, I could not find a way
>>> that would periodically send text without a header issue.
>>>
>>> Guess why they have different tools for the different needs.
>>>
>>> Thanks Str.
>>>
>>> Scotty
>>>
>>>
>>
>> Nope, but it has nothing to do with php. HTTP is a request/response
>> protocol; the client sends a request and expects a response within the
>> time specified by the browser's timeout value.
>>
>> But then this causes a problem also, because PHP has a buffer (for
>> efficiency), the server has a buffer (also for efficiency), routers
>> along the way may have buffers (again, for efficiency), and the client's
>> system may have a buffer (hmmm... efficiency?) and finally the browser
>> may have a buffer.
>>
>> The bottom line is - the HTTP protocol is pretty much useless for
>> long-running tasks. The time it takes to send data to the browser is
>> dependent on a lot of things, and the browser's timeout values are
>> independent of anything the server can control.
>>
>> If you need such processing, I suggest you look into other methods of
>> communications. Java applets, for instance, can work with non-http
>> protocols.
>>
>
> Yeah I can see there can be a bottle-neck/lag issue anywhere which would
> be useless for anything real time (which I use loosely).
>
> Thanks for the input.
>
> Scotty
You can also have the browser request a reload every few seconds, it
isn't exactly what you're asking for but it can give more or less the
same benefits.
|
|
|
|
| Re: Output status during processing [message #179536 is a reply to message #179533] |
Mon, 05 November 2012 16:32 |
Thomas 'PointedEars'
Messages: 701
Registered: October 2010
Karma: 0
|
Senior Member |
|
|
Denis McMahon wrote:
> On Sun, 04 Nov 2012 13:52:35 -0800, Scott Johnson wrote:
>> I could use AJAX to load the file and then process it in the background
>> sending status points as it goes through AJAX.
>
> Back up a step.
>
> The server can't "send" status updates to the browser using ajax in the
> normal case,
_Usual_, not normal.
> rather the browser would request "request" status updates from the server,
> usually in relation to some browser event (which might be a javascript
There is no javascript.
> timer timeout, or the user clicking a "get update" button or link).
Yes.
> Ajax,
… is a misnomer used only by people who do not know better. It does not
need to be asynchronous, it does not have to be JavaScript (or any other
ECMAScript implementation), and it often does not use XML.
> which is built on the http request / response, which is in turn
> built on a tcp/ip connection,
_HTTP_ and _TCP/IP_
> is not a technology which generally allows
_Usually_, not generally.
> the server to push unsolicited data to the browser, rather it is a
> technology that allows the browser to make requests to the server and
> receive data back without reloading the page being displayed.
>
> Specifically, when you make an http request to a server, a tcp connection
> is opened to the server.
Correct.
> The connection is usually closed after the server has sent the response.
Utter nonsense. HTTP/1.1 uses persistent TCP connections by default.
HTTP/1.1 (RFC 2616) is currently an IETF Draft Standard. It was first
issued in 1999 and is widely implemented today: HTTP/1.1 is the default
protocol version in modern Web browsers.
> Making a connection to the server is easy, because tcp/ip knows that the
> server is listening to http on port 80.
Or any other port. 80/tcp is the default port for HTTP, not the only one.
> In the other direction, there is no official port that a browser listens
> out on for a server to make a connection to it, which makes it difficult
> for the server to open a connection to the browser to just send it
> something.
An official port number is not required. A privileged application in the
browser (a browser extension) can listen for push notifications from a
remote client on an unused TCP port > 1024. Which is done.
However, it is also possible that an HTTP client keeps a listen-connection
(and therefore a local TCP port) open for a specific HTTP server, waiting
for push updates from that server through that connection. That application
model is currently called Comet, among other names.
<http://en.wikipedia.org/wiki/Comet_(programming)>
(The caveat there is that an HTTP client is supposed to limit the number of
connections to the same server or proxy, meaning that this "connection port"
of the browser's HTTP client cannot be used for accessing other resources as
Web browsers follow that recommendation in the RFCs.)
There are other approaches (like server-sent events and the HTML5 WebSocket
API) that accomplish the same. They are referred to at the bottom of that
article.
> I know it may sound like a petty distinction to make, but making the
> distinction helps you keep it clear in your mind how it all fits
> together, and in the concept of php running behind a web-server, the
> context is generally servicing and responding to requests from the
> browser, not pushing data to the browser.
If only you knew what you are talking about.
PointedEars
--
Anyone who slaps a 'this page is best viewed with Browser X' label on
a Web page appears to be yearning for the bad old days, before the Web,
when you had very little chance of reading a document written on another
computer, another word processor, or another network. -- Tim Berners-Lee
|
|
|
|
| Re: Output status during processing [message #179537 is a reply to message #179534] |
Mon, 05 November 2012 19:59 |
Scott Johnson
Messages: 196
Registered: January 2012
Karma: 0
|
Senior Member |
|
|
On 11/5/2012 5:22 AM, Thomas 'PointedEars' Lahn wrote:
> Scott Johnson wrote:
>
>> What I would like to do is provide an output to the browser during a
>> long processing term.
>>
>> Exactly what I am doing is this.
>>
>> I am loading a csv file to update inventory.
>>
>> During this process I am flushing the current Table and then loading the
>> new and then comparing the inventory table to a product table and would
>> like to provide status of where we are in the process what items matched
>> and changed and so forth in real time.
>>
>> I have done some similar tasks in JS but with minimal steps in the
>> processing then what I am trying to accomplish.
>>
>> I did some binging and found that it may have to do with output
>> buffering but nothing particular to my situation. (as far as I could find)
>>
>> If this is possible could someone point me in the right direction and I
>> will do the leg work.
>
> Contrary to popular belief, it is possible with PHP alone to provide step-
> by-step feedback to the client (as well as to a server-side log file) while
> a server-side script is being executed.
>
> About one year ago, I have done something very similar to what you want to
> do: I have written a PHP script to import product information from a CSV
> file into a Magento eCommerce database (in a way that Magento's built-in
> import feature was not capable of at the time).
>
> Here is basically what I have done:
>
> 1. Enable PHP's output buffering (ob_start()).
>
> 2. Process a record in the CSV (with fgetcsv() and the Magento API),
> writing status information to the server-side log file and (optionally)
> to the standard output (with file_put_contents() and `echo',
> respectively).
>
> 3. Flush and clear the output buffer:
>
> /**
> * Flushes all output buffers and restarts output buffering
> *
> * @author sebastian(at)jcompare(dot)com
> * @link http://php.net/manual/en/function.ob-flush.php
> */
> function flush_buffers()
> {
> ob_end_flush();
> ob_flush();
> flush();
> ob_start();
> }
>
> That displays what is written in step 2 in the console or the browser
> window (depending on how the script is run). In order to have the last
> row visible, client-side scripting is used to scroll down to it in
> regular intervals.
>
> 4. Continue with step 2 if there are more records or files;
> exit successfully otherwise.
>
> That has worked beautifully both with text/plain and text/html.
>
> The output-buffering approach does _not_ provide you with the ability to
> update arbitrary parts of an (X)HTML document, though; it would create the
> response document content on the fly instead. If you need arbitrary display
> updates, you should look into asynchronous XMLHttpRequest, with which you
> would poll status information from the server accessing a (PHP-)generated
> resource in regular intervals, and update your (X)HTML document accordingly.
> (The output-buffering approach was adequate and sufficient in my case; the
> XHR approach was and is employed by the Magento UI but was not sufficient
> for the used data format at the time.)
>
> Also, you should be aware that no matter how you do it, neither a database
> connection is likely to be stable indefinitely nor a PHP script is likely to
> run forever. In particular, on the server my script ran, the import was
> supposed to run regularly (from CSV files that were uploaded by the customer
> at arbitrary times, with the provision that the shop information needed to
> be as up-to-date as possible). IOW, a cron job was set up which started PHP
> CLI to run the PHP script if it not already ran.
>
> But because a CSV file could have had many (thousands of) records (read:
> product variants), and there were several such files, it was not possible to
> import all records from all files in one run. Either the database
> connection used by the Magento API was severed or the PHP script would be
> terminated early (because of max_execution_time in php.ini, which for
> security reasons could not be set to 0 for the Web version that was still
> needed for "manual" import).
>
> I have solved this with storing the position of the last import in another
> file, so that when the script was executed next time it would check if a
> previous position was saved and continue from there, or it would continue
> from the beginning of the file(s).
>
>
> HTH
>
> PointedEars
>
Ahh very interesting.
Thank you very much for the information. This may be just the ticket
for me. I will let you know how it ends up.
Again thanks
Scotty
|
|
|
|
| Re: Output status during processing [message #179538 is a reply to message #179537] |
Mon, 05 November 2012 23:21 |
Thomas 'PointedEars'
Messages: 701
Registered: October 2010
Karma: 0
|
Senior Member |
|
|
Scott Johnson wrote:
> On 11/5/2012 5:22 AM, Thomas 'PointedEars' Lahn wrote:
>> [Using output buffering to generate a response step-by-step]
>
> Ahh very interesting.
>
> Thank you very much for the information. This may be just the ticket
> for me. I will let you know how it ends up.
You are welcome.
But *please* stop wasting resources (and trying your readers' nerves) and
*trim* *your* *quotes* to the relevant parts next time. It is absolutely
unacceptable that you post 103 lines of quoted text for 9 lines of new (own)
text (of which one line is an attribution line even):
<http://www.netmeister.org/news/learn2quote.html>
PointedEars
P.S.: You have e-mail.
--
var bugRiddenCrashPronePieceOfJunk = (
navigator.userAgent.indexOf('MSIE 5') != -1
&& navigator.userAgent.indexOf('Mac') != -1
) // Plone, register_function.js:16
|
|
|
|
Calendar
Search
Help
Members
Register
Login
Home
]