| Efficiency of a lot of variables [message #170856] |
Sat, 04 December 2010 00:52  |
jwcarlton
Messages: 76
Registered: December 2010
Karma: 0
|
Member |
|
|
I have 1,000 variables, written like this:
// Block 1
$hash['arr1']['var1'] = "whatever1";
$hash['arr1']['var2'] = "whatever2";
:
$hash['arr1']['var10'] = "whatever10";
// Block 2
$hash['arr2']['var1'] = "whateverelse1";
$hash['arr2']['var2'] = "whateverelse2";
:
$hash['arr2']['var10'] = "whateverelse10";
There are 100 blocks, and 10 variables in each block. The block used
is determined based on the domain used to access the site (there are
99 domains parked on top of the primary account). In the sample above,
'arr1' and 'arr2' represent the domains, and these are loaded on every
page of the site.
I've been researching this for awhile, and most seem to agree that
it's faster / more efficient to store all 1,000 variables into arrays,
instead of 100 blocks of if-else or switch-case. I had also considered
having 100 text files, and then just loading the appropriate text file
based on the domain (like below), but most believed that the I/O speed
would be worse than just loading all 1,000 at once:
$domain = "whatever";
list($var1, $var2,..., $var10) = FILE("/path/to/variables-
$domain.txt");
So, I guess my first question is, do you guys agree that storing them
all at once is faster / more efficient than the 3 alternatives? Or can
you suggest another option that I haven't considered?
If this is the best method, my next question is in regards to the
construct of the arrays. In the sample above, I have a single
multidimensional array; is that better than, for example, 100 arrays
with 10 dimensions:
$arr1['var1'] = "whatever1";
$arr1['var2'] = "whatever2";
:
$arr1['var10'] = "whatever10";
Or vice versa, 10 arrays with 100 dimensions:
$var1['arr1'] = "whatever1";
$var2['arr1'] = "whatever2";
:
$var10['arr1'] = "whatever10";
I know that I'm probably worrying too much about microseconds, but I
currently have an average of 600,000 pageviews a day, and expect it to
increase to at least 12,000,000 within the next year, so I'm trying to
make sure everything is as fast as possible before the problems set
in :-)
TIA,
Jason
|
|
|
|
| Re: Efficiency of a lot of variables [message #170858 is a reply to message #170856] |
Sat, 04 December 2010 02:07  |
Magno
Messages: 49
Registered: October 2010
Karma: 0
|
Member |
|
|
On 12/03/2010 09:52 PM, jwcarlton wrote:
> I have 1,000 variables, written like this:
>
> // Block 1
> $hash['arr1']['var1'] = "whatever1";
> $hash['arr1']['var2'] = "whatever2";
> :
> $hash['arr1']['var10'] = "whatever10";
>
> // Block 2
> $hash['arr2']['var1'] = "whateverelse1";
> $hash['arr2']['var2'] = "whateverelse2";
> :
> $hash['arr2']['var10'] = "whateverelse10";
>
>
> There are 100 blocks, and 10 variables in each block. The block used
> is determined based on the domain used to access the site (there are
> 99 domains parked on top of the primary account). In the sample above,
> 'arr1' and 'arr2' represent the domains, and these are loaded on every
> page of the site.
>
> I've been researching this for awhile, and most seem to agree that
> it's faster / more efficient to store all 1,000 variables into arrays,
> instead of 100 blocks of if-else or switch-case. I had also considered
> having 100 text files, and then just loading the appropriate text file
> based on the domain (like below), but most believed that the I/O speed
> would be worse than just loading all 1,000 at once:
>
> $domain = "whatever";
> list($var1, $var2,..., $var10) = FILE("/path/to/variables-
> $domain.txt");
>
> So, I guess my first question is, do you guys agree that storing them
> all at once is faster / more efficient than the 3 alternatives? Or can
> you suggest another option that I haven't considered?
>
> If this is the best method, my next question is in regards to the
> construct of the arrays. In the sample above, I have a single
> multidimensional array; is that better than, for example, 100 arrays
> with 10 dimensions:
>
> $arr1['var1'] = "whatever1";
> $arr1['var2'] = "whatever2";
> :
> $arr1['var10'] = "whatever10";
>
>
> Or vice versa, 10 arrays with 100 dimensions:
>
> $var1['arr1'] = "whatever1";
> $var2['arr1'] = "whatever2";
> :
> $var10['arr1'] = "whatever10";
>
>
> I know that I'm probably worrying too much about microseconds, but I
> currently have an average of 600,000 pageviews a day, and expect it to
> increase to at least 12,000,000 within the next year, so I'm trying to
> make sure everything is as fast as possible before the problems set
> in :-)
>
> TIA,
>
> Jason
What I do in cases like this is benchmarking.
Make thousand of calls for every case and compare their response time.
|
|
|
|
| Re: Efficiency of a lot of variables [message #170860 is a reply to message #170858] |
Sat, 04 December 2010 02:36 |
jwcarlton
Messages: 76
Registered: December 2010
Karma: 0
|
Member |
|
|
On Dec 3, 9:07 pm, Magno <marbar...@gmail.com> wrote:
> On 12/03/2010 09:52 PM, jwcarlton wrote:
>
>
>
>> I have 1,000 variables, written like this:
>
>> // Block 1
>> $hash['arr1']['var1'] = "whatever1";
>> $hash['arr1']['var2'] = "whatever2";
>> :
>> $hash['arr1']['var10'] = "whatever10";
>
>> // Block 2
>> $hash['arr2']['var1'] = "whateverelse1";
>> $hash['arr2']['var2'] = "whateverelse2";
>> :
>> $hash['arr2']['var10'] = "whateverelse10";
>
>> There are 100 blocks, and 10 variables in each block. The block used
>> is determined based on the domain used to access the site (there are
>> 99 domains parked on top of the primary account). In the sample above,
>> 'arr1' and 'arr2' represent the domains, and these are loaded on every
>> page of the site.
>
>> I've been researching this for awhile, and most seem to agree that
>> it's faster / more efficient to store all 1,000 variables into arrays,
>> instead of 100 blocks of if-else or switch-case. I had also considered
>> having 100 text files, and then just loading the appropriate text file
>> based on the domain (like below), but most believed that the I/O speed
>> would be worse than just loading all 1,000 at once:
>
>> $domain = "whatever";
>> list($var1, $var2,..., $var10) = FILE("/path/to/variables-
>> $domain.txt");
>
>> So, I guess my first question is, do you guys agree that storing them
>> all at once is faster / more efficient than the 3 alternatives? Or can
>> you suggest another option that I haven't considered?
>
>> If this is the best method, my next question is in regards to the
>> construct of the arrays. In the sample above, I have a single
>> multidimensional array; is that better than, for example, 100 arrays
>> with 10 dimensions:
>
>> $arr1['var1'] = "whatever1";
>> $arr1['var2'] = "whatever2";
>> :
>> $arr1['var10'] = "whatever10";
>
>> Or vice versa, 10 arrays with 100 dimensions:
>
>> $var1['arr1'] = "whatever1";
>> $var2['arr1'] = "whatever2";
>> :
>> $var10['arr1'] = "whatever10";
>
>> I know that I'm probably worrying too much about microseconds, but I
>> currently have an average of 600,000 pageviews a day, and expect it to
>> increase to at least 12,000,000 within the next year, so I'm trying to
>> make sure everything is as fast as possible before the problems set
>> in :-)
>
>> TIA,
>
>> Jason
>
> What I do in cases like this is benchmarking.
> Make thousand of calls for every case and compare their response time.
That's my next step, but I was concerned that the current server load
would give me false results. I don't have an unused server to test
with.
I also just stumbled across "session handling". That's a totally new
concept for me. Would it be practical to load the variables on the
first session, then store them as a $_SESSION? Like this:
session_start();
if (!$_SESSION['var1']) {
switch ($domain) {
case "arr1" :
default :
$_SESSION['var1'] = "whatever1";
$_SESSION['var2'] = "whatever2";
:
$_SESSION['var10'] = "whatever10";
break;
case "arr2" :
$_SESSION['var1'] = "whatever1";
$_SESSION['var2'] = "whatever2";
:
$_SESSION['var10'] = "whatever10";
break;
}
}
|
|
|
|
| Re: Efficiency of a lot of variables [message #170862 is a reply to message #170856] |
Sat, 04 December 2010 03:24 |
Norman Peelman
Messages: 126
Registered: September 2010
Karma: 0
|
Senior Member |
|
|
jwcarlton wrote:
> I have 1,000 variables, written like this:
>
> // Block 1
> $hash['arr1']['var1'] = "whatever1";
> $hash['arr1']['var2'] = "whatever2";
> :
> $hash['arr1']['var10'] = "whatever10";
>
> // Block 2
> $hash['arr2']['var1'] = "whateverelse1";
> $hash['arr2']['var2'] = "whateverelse2";
> :
> $hash['arr2']['var10'] = "whateverelse10";
>
>
> There are 100 blocks, and 10 variables in each block. The block used
> is determined based on the domain used to access the site (there are
> 99 domains parked on top of the primary account). In the sample above,
> 'arr1' and 'arr2' represent the domains, and these are loaded on every
> page of the site.
>
> I've been researching this for awhile, and most seem to agree that
> it's faster / more efficient to store all 1,000 variables into arrays,
> instead of 100 blocks of if-else or switch-case. I had also considered
> having 100 text files, and then just loading the appropriate text file
> based on the domain (like below), but most believed that the I/O speed
> would be worse than just loading all 1,000 at once:
>
> $domain = "whatever";
> list($var1, $var2,..., $var10) = FILE("/path/to/variables-
> $domain.txt");
>
> So, I guess my first question is, do you guys agree that storing them
> all at once is faster / more efficient than the 3 alternatives? Or can
> you suggest another option that I haven't considered?
>
> If this is the best method, my next question is in regards to the
> construct of the arrays. In the sample above, I have a single
> multidimensional array; is that better than, for example, 100 arrays
> with 10 dimensions:
>
> $arr1['var1'] = "whatever1";
> $arr1['var2'] = "whatever2";
> :
> $arr1['var10'] = "whatever10";
>
>
> Or vice versa, 10 arrays with 100 dimensions:
>
> $var1['arr1'] = "whatever1";
> $var2['arr1'] = "whatever2";
> :
> $var10['arr1'] = "whatever10";
>
>
> I know that I'm probably worrying too much about microseconds, but I
> currently have an average of 600,000 pageviews a day, and expect it to
> increase to at least 12,000,000 within the next year, so I'm trying to
> make sure everything is as fast as possible before the problems set
> in :-)
>
> TIA,
>
> Jason
I'd look into using shared memory:
http://us3.php.net/manual/en/book.apc.php
--
Norman
Registered Linux user #461062
AMD64X2 6400+ Ubuntu 8.04 64bit
|
|
|
|
| Re: Efficiency of a lot of variables [message #170880 is a reply to message #170856] |
Sat, 04 December 2010 17:46 |
Twayne
Messages: 135
Registered: September 2010
Karma: 0
|
Senior Member |
|
|
In news:605cbbdc-3c65-4d1f-ba19-8b676cb39fcb(at)u9g2000pra(dot)googlegroups(dot)com,
jwcarlton <jwcarlton(at)gmail(dot)com> typed:
> I have 1,000 variables, written like this:
>
> // Block 1
> $hash['arr1']['var1'] = "whatever1";
> $hash['arr1']['var2'] = "whatever2";
>>
> $hash['arr1']['var10'] = "whatever10";
>
> // Block 2
> $hash['arr2']['var1'] = "whateverelse1";
> $hash['arr2']['var2'] = "whateverelse2";
>>
> $hash['arr2']['var10'] = "whateverelse10";
>
>
> There are 100 blocks, and 10 variables in each block. The
> block used is determined based on the domain used to access
> the site (there are 99 domains parked on top of the primary
> account). In the sample above, 'arr1' and 'arr2' represent
> the domains, and these are loaded on every page of the site.
>
> I've been researching this for awhile, and most seem to
> agree that it's faster / more efficient to store all 1,000
> variables into arrays, instead of 100 blocks of if-else or
> switch-case. I had also considered having 100 text files,
> and then just loading the appropriate text file based on
> the domain (like below), but most believed that the I/O
> speed would be worse than just loading all 1,000 at once:
>
> $domain = "whatever";
> list($var1, $var2,..., $var10) = FILE("/path/to/variables-
> $domain.txt");
>
> So, I guess my first question is, do you guys agree that
> storing them all at once is faster / more efficient than
> the 3 alternatives? Or can you suggest another option that
> I haven't considered?
>
> If this is the best method, my next question is in regards
> to the construct of the arrays. In the sample above, I have
> a single multidimensional array; is that better than, for
> example, 100 arrays with 10 dimensions:
>
> $arr1['var1'] = "whatever1";
> $arr1['var2'] = "whatever2";
>>
> $arr1['var10'] = "whatever10";
>
>
> Or vice versa, 10 arrays with 100 dimensions:
>
> $var1['arr1'] = "whatever1";
> $var2['arr1'] = "whatever2";
>>
> $var10['arr1'] = "whatever10";
>
>
> I know that I'm probably worrying too much about
> microseconds, but I currently have an average of 600,000
> pageviews a day, and expect it to increase to at least
> 12,000,000 within the next year, so I'm trying to make sure
> everything is as fast as possible before the problems set
> in :-)
>
> TIA,
>
> Jason
Due to the size/quantity of files you have, and variances being great
between servers, I don't think you have much choice but to benchmark the
various possibilities. Do the testing on the first locally using a local
server, and then go online to the same server-set where the site is actually
going to live. And don't forget time of day traffic variables, etc..
Do your timing measurements with a stopwatch, onscreen or manually held
(best). If measurements are so small as to not be accurate using a stopwatch
then the differences/variances will be negligible. If you get into
double-digit minute differences, then you know for sure you have a
noticeable diff in your methods.
Also, if it's too long a wait to start with, say over 15 seconds to be a
useful screen, consider it a failure to start with and look for better ways.
You MUST find something you can use as a control to compare all the rest of
the numbers against.
Use a local server for proof-of-concept since you'll likely have to do a
lot of troubleshooting but with smaller sets of data. Then when it's all
running well go online and upload it to its intended home and start testing
load and page display times.
IMO arrays and Session()'s are the best combination to start with but I've
never had anything so huge I had to worry about it much escepting images.
If you're new to sessions and based on another allusioin you made, I'd
suggest you go slowly and research carefully; there are likely a gazillion
things you don't know yet. Don't benchmark anything that isn't working 100%
or you'll waste your time.
Definitely do NOT load data more than once! Do it once only; so after the
first time, succeeding times are much faster.
With the numbers of files etc. to start with, there has to be some way to
categorize the optons even if some appear in more than one category. Now you
have a LOT less data to search thru or address. So toss in a page tochoose
a category from and make it easy to return to that page at any time.
And if you use images at all on any of those pages, you're going to be
able to easily sink yourself; don't do it. Keep latency in mind at all
times.
PHP: Get "The Manual".
Check out w3schools.com and php.net. Great resources.
http://www.websiteoptimization.com/services/analyze/ is a great site
optimizer IMO, and gives a lot of timing numbers.
HTH,
Twayne`
|
|
|
|
Calendar
Search
Help
Members
Register
Login
Home
]