| signed int64 pack/unpack [message #184577] |
Sun, 12 January 2014 02:18  |
cameron7
Messages: 8
Registered: January 2014
Karma: 0
|
Junior Member |
|
|
Hi,
I'm having a bit of trouble figuring out a straightforward way to "pack" / "unpack" a signed int64.
I'm on a 64-bit machine, but the 'i' and 'I' are coming back with 4-byte length. And of course even though double comes back with 8 length it's a floating point and thus loses precision.
Any thoughts? Any help would be greatly appreciated.
BTW, unsigned int64 seems relatively straightforward, but signed int64 has been unusually difficult to figure out.
Thanks
|
|
|
|
| Re: signed int64 pack/unpack [message #184578 is a reply to message #184577] |
Sun, 12 January 2014 02:35  |
Jerry Stuckle
Messages: 2598
Registered: September 2010
Karma: 0
|
Senior Member |
|
|
On 1/11/2014 9:18 PM, cameron7(at)gmail(dot)com wrote:
> Hi,
>
> I'm having a bit of trouble figuring out a straightforward way to "pack" / "unpack" a signed int64.
>
> I'm on a 64-bit machine, but the 'i' and 'I' are coming back with 4-byte length. And of course even though double comes back with 8 length it's a floating point and thus loses precision.
>
> Any thoughts? Any help would be greatly appreciated.
>
> BTW, unsigned int64 seems relatively straightforward, but signed int64 has been unusually difficult to figure out.
>
> Thanks
>
You're on a 64 bit machine, but are you using a 64 bit OS and 64 bit PHP?
As for thoughts - how about some detail? What's you code? What do you
expect? What do you get?
--
==================
Remove the "x" from my email address
Jerry Stuckle
jstucklex(at)attglobal(dot)net
==================
|
|
|
|
| Re: signed int64 pack/unpack [message #184579 is a reply to message #184577] |
Sun, 12 January 2014 02:42 |
Ben Bacarisse
Messages: 82
Registered: November 2013
Karma: 0
|
Member |
|
|
cameron7(at)gmail(dot)com writes:
> I'm having a bit of trouble figuring out a straightforward way to
> "pack" / "unpack" a signed int64.
>
> I'm on a 64-bit machine, but the 'i' and 'I' are coming back with
> 4-byte length.
There is no explicit format for 64-bit ints. The i/I formats are used
for a native "int" but that need not be 64 bits on a 64-bit machine.
The function is borrowed from Perl whose documentation says a bit more
This 'integer' is _at_least_ 32 bits wide. Its exact size depends on
what a local C compiler calls 'int'.
On my 64-bit Linux machine, 'int' is 32 bits.
> And of course even though double comes back with 8
> length it's a floating point and thus loses precision.
>
> Any thoughts? Any help would be greatly appreciated.
You could, perhaps, just pack the two halves. Of course doing it
yourself rather defeats the purpose, but I am not sure there is another
way.
Depending on what you need this for, you could consider another format
altogether (ASCII, for example).
> BTW, unsigned int64 seems relatively straightforward, but signed int64
> has been unusually difficult to figure out.
How are you doing unsigned 64-bit ints?
--
Ben.
|
|
|
|
| Re: signed int64 pack/unpack [message #184580 is a reply to message #184578] |
Sun, 12 January 2014 02:46 |
cameron7
Messages: 8
Registered: January 2014
Karma: 0
|
Junior Member |
|
|
Hi, yes, sorry I am running 64-bit OS and 64-bit PHP. PHP_INT_MAX is giving appropriate value.
Specifics from `php -i`: PHP Version => 5.4.6-1ubuntu1.5
The code has become extremely ugly, and still not working perfectly, thus the reason for trying to find others who may have experience with this. The pack / unpack functions don't provide support for 64-bit integers directly so I'm looking for at least a sound "hack".
I've tried all sorts of things, from base_convert to simply trying to extract and iterate over a bitwise comparison against each bit in 4 separate 16-bit ints, but nothing I've tried so far is working correctly.
Basically, the context for this is I'm writing a client for a custom protocol in which several of the components of that protocol in requests and responses require big endian 64-bit signed int, thus the reason for the need to pack / unpack.
|
|
|
|
| Re: signed int64 pack/unpack [message #184581 is a reply to message #184580] |
Sun, 12 January 2014 02:59 |
The Natural Philosoph
Messages: 993
Registered: September 2010
Karma: 0
|
Senior Member |
|
|
On 12/01/14 02:46, cameron7(at)gmail(dot)com wrote:
> Hi, yes, sorry I am running 64-bit OS and 64-bit PHP. PHP_INT_MAX is
> giving appropriate value.
>
> Specifics from `php -i`: PHP Version => 5.4.6-1ubuntu1.5
>
> The code has become extremely ugly, and still not working perfectly,
> thus the reason for trying to find others who may have experience
> with this. The pack / unpack functions don't provide support for
> 64-bit integers directly so I'm looking for at least a sound "hack".
>
> I've tried all sorts of things, from base_convert to simply trying to
> extract and iterate over a bitwise comparison against each bit in 4
> separate 16-bit ints, but nothing I've tried so far is working
> correctly.
>
> Basically, the context for this is I'm writing a client for a custom
> protocol in which several of the components of that protocol in
> requests and responses require big endian 64-bit signed int, thus the
> reason for the need to pack / unpack.
>
I have to say this is where I would probably be trying to write a C
library and wrap it in PHP..
Or write a C daemon and interrogate that..via a named pipe or system
type call.
This is system level programming and PHP is simply not the right tool
for it.
--
Ineptocracy
(in-ep-toc’-ra-cy) – a system of government where the least capable to
lead are elected by the least capable of producing, and where the
members of society least likely to sustain themselves or succeed, are
rewarded with goods and services paid for by the confiscated wealth of a
diminishing number of producers.
|
|
|
|
| Re: signed int64 pack/unpack [message #184583 is a reply to message #184579] |
Sun, 12 January 2014 03:01 |
Jerry Stuckle
Messages: 2598
Registered: September 2010
Karma: 0
|
Senior Member |
|
|
On 1/11/2014 9:42 PM, Ben Bacarisse wrote:
> cameron7(at)gmail(dot)com writes:
>
>> I'm having a bit of trouble figuring out a straightforward way to
>> "pack" / "unpack" a signed int64.
>>
>> I'm on a 64-bit machine, but the 'i' and 'I' are coming back with
>> 4-byte length.
>
> There is no explicit format for 64-bit ints. The i/I formats are used
> for a native "int" but that need not be 64 bits on a 64-bit machine.
> The function is borrowed from Perl whose documentation says a bit more
>
> This 'integer' is _at_least_ 32 bits wide. Its exact size depends on
> what a local C compiler calls 'int'.
>
> On my 64-bit Linux machine, 'int' is 32 bits.
>
On my 64 bit Linux system, 'int' is 64 bits. But then I have the 64 bit
PHP version (and 64 bit OS) loaded.
What do you get when you do:
<?php
echo PHP_INT_MAX . "\n";
?>
--
==================
Remove the "x" from my email address
Jerry Stuckle
jstucklex(at)attglobal(dot)net
==================
|
|
|
|
| Re: signed int64 pack/unpack [message #184584 is a reply to message #184581] |
Sun, 12 January 2014 03:03 |
Jerry Stuckle
Messages: 2598
Registered: September 2010
Karma: 0
|
Senior Member |
|
|
On 1/11/2014 9:59 PM, The Natural Philosopher wrote:
> On 12/01/14 02:46, cameron7(at)gmail(dot)com wrote:
>> Hi, yes, sorry I am running 64-bit OS and 64-bit PHP. PHP_INT_MAX is
>> giving appropriate value.
>>
>> Specifics from `php -i`: PHP Version => 5.4.6-1ubuntu1.5
>>
>> The code has become extremely ugly, and still not working perfectly,
>> thus the reason for trying to find others who may have experience
>> with this. The pack / unpack functions don't provide support for
>> 64-bit integers directly so I'm looking for at least a sound "hack".
>>
>> I've tried all sorts of things, from base_convert to simply trying to
>> extract and iterate over a bitwise comparison against each bit in 4
>> separate 16-bit ints, but nothing I've tried so far is working
>> correctly.
>>
>> Basically, the context for this is I'm writing a client for a custom
>> protocol in which several of the components of that protocol in
>> requests and responses require big endian 64-bit signed int, thus the
>> reason for the need to pack / unpack.
>>
>
> I have to say this is where I would probably be trying to write a C
> library and wrap it in PHP..
>
> Or write a C daemon and interrogate that..via a named pipe or system
> type call.
>
> This is system level programming and PHP is simply not the right tool
> for it.
>
>
>
>
>
You'd write a 5,000 line C program to say "Hello, World".
--
==================
Remove the "x" from my email address
Jerry Stuckle
jstucklex(at)attglobal(dot)net
==================
|
|
|
|
| Re: signed int64 pack/unpack [message #184585 is a reply to message #184580] |
Sun, 12 January 2014 03:04 |
Jerry Stuckle
Messages: 2598
Registered: September 2010
Karma: 0
|
Senior Member |
|
|
On 1/11/2014 9:46 PM, cameron7(at)gmail(dot)com wrote:
> Hi, yes, sorry I am running 64-bit OS and 64-bit PHP. PHP_INT_MAX is giving appropriate value.
>
> Specifics from `php -i`: PHP Version => 5.4.6-1ubuntu1.5
>
> The code has become extremely ugly, and still not working perfectly, thus the reason for trying to find others who may have experience with this. The pack / unpack functions don't provide support for 64-bit integers directly so I'm looking for at least a sound "hack".
>
> I've tried all sorts of things, from base_convert to simply trying to extract and iterate over a bitwise comparison against each bit in 4 separate 16-bit ints, but nothing I've tried so far is working correctly.
>
> Basically, the context for this is I'm writing a client for a custom protocol in which several of the components of that protocol in requests and responses require big endian 64-bit signed int, thus the reason for the need to pack / unpack.
>
That may be, but without knowing exactly what you're doing and what you
expect, any attempt to help you would be a waste of both your and our time.
--
==================
Remove the "x" from my email address
Jerry Stuckle
jstucklex(at)attglobal(dot)net
==================
|
|
|
|
|
|
| Re: signed int64 pack/unpack [message #184587 is a reply to message #184581] |
Sun, 12 January 2014 03:10 |
cameron7
Messages: 8
Registered: January 2014
Karma: 0
|
Junior Member |
|
|
|
I definitely have mixed view on this. While I agree PHP isn't system-level programming language, I'm seeing more and more binary protocols for web infrastructure, which leads me to believe PHP should have better binary support.
|
|
|
|
| Re: signed int64 pack/unpack [message #184588 is a reply to message #184583] |
Sun, 12 January 2014 03:11 |
Ben Bacarisse
Messages: 82
Registered: November 2013
Karma: 0
|
Member |
|
|
Jerry Stuckle <jstucklex(at)attglobal(dot)net> writes:
> On 1/11/2014 9:42 PM, Ben Bacarisse wrote:
>> cameron7(at)gmail(dot)com writes:
>>
>>> I'm having a bit of trouble figuring out a straightforward way to
>>> "pack" / "unpack" a signed int64.
>>>
>>> I'm on a 64-bit machine, but the 'i' and 'I' are coming back with
>>> 4-byte length.
>>
>> There is no explicit format for 64-bit ints. The i/I formats are used
>> for a native "int" but that need not be 64 bits on a 64-bit machine.
>> The function is borrowed from Perl whose documentation says a bit more
>>
>> This 'integer' is _at_least_ 32 bits wide. Its exact size depends on
>> what a local C compiler calls 'int'.
^^^^^^^^^^^^^^^^^^^^^^^^^^^
>> On my 64-bit Linux machine, 'int' is 32 bits.
>
> On my 64 bit Linux system, 'int' is 64 bits. But then I have the 64
> bit PHP version (and 64 bit OS) loaded.
Yes, me too.
> What do you get when you do:
>
> <?php
> echo PHP_INT_MAX . "\n";
> ?>
Same as you but I don't think that's relevant. PHP stores values
internally using a C long. I think the Perl document describes what's
really happening.
--
Ben.
|
|
|
|
| Re: signed int64 pack/unpack [message #184589 is a reply to message #184588] |
Sun, 12 January 2014 03:15 |
Jerry Stuckle
Messages: 2598
Registered: September 2010
Karma: 0
|
Senior Member |
|
|
On 1/11/2014 10:11 PM, Ben Bacarisse wrote:
> Jerry Stuckle <jstucklex(at)attglobal(dot)net> writes:
>
>> On 1/11/2014 9:42 PM, Ben Bacarisse wrote:
>>> cameron7(at)gmail(dot)com writes:
>>>
>>>> I'm having a bit of trouble figuring out a straightforward way to
>>>> "pack" / "unpack" a signed int64.
>>>>
>>>> I'm on a 64-bit machine, but the 'i' and 'I' are coming back with
>>>> 4-byte length.
>>>
>>> There is no explicit format for 64-bit ints. The i/I formats are used
>>> for a native "int" but that need not be 64 bits on a 64-bit machine.
>>> The function is borrowed from Perl whose documentation says a bit more
>>>
>>> This 'integer' is _at_least_ 32 bits wide. Its exact size depends on
>>> what a local C compiler calls 'int'.
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^
>>> On my 64-bit Linux machine, 'int' is 32 bits.
>>
>> On my 64 bit Linux system, 'int' is 64 bits. But then I have the 64
>> bit PHP version (and 64 bit OS) loaded.
>
> Yes, me too.
>
>> What do you get when you do:
>>
>> <?php
>> echo PHP_INT_MAX . "\n";
>> ?>
>
> Same as you but I don't think that's relevant. PHP stores values
> internally using a C long. I think the Perl document describes what's
> really happening.
>
It is totally relevant. You can't operate on 64 bit values if you don't
have 64 bit support.
And this is PHP, not Perl. Things have changed a *lot* since the
original versions were created. Much of the Perl documentation no
longer applies to PHP (if it ever did).
--
==================
Remove the "x" from my email address
Jerry Stuckle
jstucklex(at)attglobal(dot)net
==================
|
|
|
|
| Re: signed int64 pack/unpack [message #184590 is a reply to message #184579] |
Sun, 12 January 2014 05:54 |
cameron7
Messages: 8
Registered: January 2014
Karma: 0
|
Junior Member |
|
|
Ok, so here's some code where we can start from:
This is giving me:
OUTPUT:
------------
PACKING: 8223372036854775807
UNPACKED: 8223372036854775807
However, when we're dealing with negative numbers:
OUTPUT:
------------
PACKING: -8223372036854775807
UNPACKED: 10223372036854776028
This likely will simply require some experimentation, for example I'm using "unsigned" flags to unpack, but it's not obvious, since when I go with a "signed" flag for negative values (ie. 'l' for signed long) I'm getting the same. If someone on the list has experience hacking binary protocols in PHP would love to know if there's a better way...
<?php
$i64 = 8223372036854775807;
echo "PACKING: ".$i64.PHP_EOL;
$binary = getBinaryString($i64, 64);
list(,$i64a) = unpack('N', pack('N', base_convert(substr($binary, 0, 32), 2, 10)));
list(,$i64b) = unpack('N', pack('N', base_convert(substr($binary, 32, 32), 2, 10)));
$i64c = getBinaryString($i64a, 32);
$i64d = getBinaryString($i64b, 32);
echo "UNPACKED: ".base_convert($i64c.$i64d,2,10).PHP_EOL;
function getBinaryString($packed,$bits){
$a = 0x01;
$binary = '';
for($x=0; $x<$bits; $x++){
$binary .= (int)(bool)($a & $packed);
$a = $a << 1;
}
return strrev($binary);
}
|
|
|
|
| Re: signed int64 pack/unpack [message #184592 is a reply to message #184589] |
Sun, 12 January 2014 12:37 |
Ben Bacarisse
Messages: 82
Registered: November 2013
Karma: 0
|
Member |
|
|
Jerry Stuckle <jstucklex(at)attglobal(dot)net> writes:
> On 1/11/2014 10:11 PM, Ben Bacarisse wrote:
>> Jerry Stuckle <jstucklex(at)attglobal(dot)net> writes:
>>
>>> On 1/11/2014 9:42 PM, Ben Bacarisse wrote:
>>>> cameron7(at)gmail(dot)com writes:
>>>>
>>>> > I'm having a bit of trouble figuring out a straightforward way to
>>>> > "pack" / "unpack" a signed int64.
>>>> >
>>>> > I'm on a 64-bit machine, but the 'i' and 'I' are coming back with
>>>> > 4-byte length.
>>>>
>>>> There is no explicit format for 64-bit ints. The i/I formats are used
>>>> for a native "int" but that need not be 64 bits on a 64-bit machine.
>>>> The function is borrowed from Perl whose documentation says a bit more
>>>>
>>>> This 'integer' is _at_least_ 32 bits wide. Its exact size depends on
>>>> what a local C compiler calls 'int'.
>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^
>>>> On my 64-bit Linux machine, 'int' is 32 bits.
>>>
>>> On my 64 bit Linux system, 'int' is 64 bits. But then I have the 64
>>> bit PHP version (and 64 bit OS) loaded.
>>
>> Yes, me too.
>>
>>> What do you get when you do:
>>>
>>> <?php
>>> echo PHP_INT_MAX . "\n";
>>> ?>
>>
>> Same as you but I don't think that's relevant. PHP stores values
>> internally using a C long. I think the Perl document describes what's
>> really happening.
>
> It is totally relevant. You can't operate on 64 bit values if you
> don't have 64 bit support.
The Perl documentation seems to describe what's happening. The PHP
documentation leaves the nature of the integer of "machine dependent
size" unspecified so some clarification is helpful.
64 bit support may be necessary but it seems not to be sufficient. Many
64 systems leave C's "int" as 32 bits, so if pack is still doing what
the Perl version describes, it's at least an explanation of what's
happening.
> And this is PHP, not Perl. Things have changed a *lot* since the
> original versions were created. Much of the Perl documentation no
> longer applies to PHP (if it ever did).
I am sure that's true, but, other than the source, I don't think there
is much more to go on. What is your explanation of what's going on?
--
Ben.
|
|
|
|
| Re: signed int64 pack/unpack [message #184593 is a reply to message #184590] |
Sun, 12 January 2014 15:25 |
cameron7
Messages: 8
Registered: January 2014
Karma: 0
|
Junior Member |
|
|
I guess I'll reply with my own solution. Turns out there are 2 separate issues here.
1) the "signed" bit needs to be flipped back to 0 before converting to int. This is because it appears the conversion from base 2 to base 10 will always assume the end value is positive.
2) Because of this, the rest of the bits need to be flipped.
So I added 2 arguments to the function, one will tell us if we're supposed to be checking for a "signed bit", and the other will tell us if we need to flip bits.
As you'll see I'm still off by 1 in the end result, but I'm sure that's just in the details.
I hope you'll all agree, this is really ugly, but seems to work :)
<?php
$i64 = -8223372036854775807;
echo "PACKING: ".$i64.PHP_EOL;
list(,$binary) = getBinaryString($i64, 64);
list(,$i64a) = unpack('N', pack('N', base_convert(substr($binary, 0, 32), 2, 10)));
list(,$i64b) = unpack('N', pack('N', base_convert(substr($binary, 32, 32), 2, 10)));
list($flag,$i64c)= getBinaryString($i64a, 32, true, true);
list(,$i64d) = getBinaryString($i64b, 32, false, true);
$unpack = base_convert($i64c.$i64d,2,10);
$unpack *= $flag ? -1 : 1;
echo "UNPACKED: ".$unpack.PHP_EOL;
function getBinaryString($packed,$bits,$check_signed = false, $flip = false){
$a = 0x01;
$binary = '';
for($x=0; $x<$bits; $x++){
$binary .= (int)(bool)($a & $packed);
$a = $a << 1;
}
$signed_flag = false;
if($check_signed && substr($binary,$bits-1,1) === '1'){
$signed_flag = true;
}
if($flip){
for($x=0;$x<strlen($binary);$x++){
$binary[$x] = $binary[$x] === '1' ? '0' : '1';
}
}
if($signed_flag){
$binary = substr($binary,0,$bits-1).'0';
}
return array($signed_flag, strrev($binary));
}
|
|
|
|
| Re: signed int64 pack/unpack [message #184594 is a reply to message #184592] |
Sun, 12 January 2014 15:51 |
Jerry Stuckle
Messages: 2598
Registered: September 2010
Karma: 0
|
Senior Member |
|
|
On 1/12/2014 7:37 AM, Ben Bacarisse wrote:
> Jerry Stuckle <jstucklex(at)attglobal(dot)net> writes:
>
>> On 1/11/2014 10:11 PM, Ben Bacarisse wrote:
>>> Jerry Stuckle <jstucklex(at)attglobal(dot)net> writes:
>>>
>>>> On 1/11/2014 9:42 PM, Ben Bacarisse wrote:
>>>> > cameron7(at)gmail(dot)com writes:
>>>> >
>>>> >> I'm having a bit of trouble figuring out a straightforward way to
>>>> >> "pack" / "unpack" a signed int64.
>>>> >>
>>>> >> I'm on a 64-bit machine, but the 'i' and 'I' are coming back with
>>>> >> 4-byte length.
>>>> >
>>>> > There is no explicit format for 64-bit ints. The i/I formats are used
>>>> > for a native "int" but that need not be 64 bits on a 64-bit machine.
>>>> > The function is borrowed from Perl whose documentation says a bit more
>>>> >
>>>> > This 'integer' is _at_least_ 32 bits wide. Its exact size depends on
>>>> > what a local C compiler calls 'int'.
>>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^
>>>> > On my 64-bit Linux machine, 'int' is 32 bits.
>>>>
>>>> On my 64 bit Linux system, 'int' is 64 bits. But then I have the 64
>>>> bit PHP version (and 64 bit OS) loaded.
>>>
>>> Yes, me too.
>>>
>>>> What do you get when you do:
>>>>
>>>> <?php
>>>> echo PHP_INT_MAX . "\n";
>>>> ?>
>>>
>>> Same as you but I don't think that's relevant. PHP stores values
>>> internally using a C long. I think the Perl document describes what's
>>> really happening.
>>
>> It is totally relevant. You can't operate on 64 bit values if you
>> don't have 64 bit support.
>
> The Perl documentation seems to describe what's happening. The PHP
> documentation leaves the nature of the integer of "machine dependent
> size" unspecified so some clarification is helpful.
>
> 64 bit support may be necessary but it seems not to be sufficient. Many
> 64 systems leave C's "int" as 32 bits, so if pack is still doing what
> the Perl version describes, it's at least an explanation of what's
> happening.
>
>> And this is PHP, not Perl. Things have changed a *lot* since the
>> original versions were created. Much of the Perl documentation no
>> longer applies to PHP (if it ever did).
>
> I am sure that's true, but, other than the source, I don't think there
> is much more to go on. What is your explanation of what's going on?
>
You can argue, or you can answer our questions and get help.
I see you've already got the answers and all you want to do is argue.
Sorry I wasted my time and the network's bandwidth trying to help you.
--
==================
Remove the "x" from my email address
Jerry Stuckle
jstucklex(at)attglobal(dot)net
==================
|
|
|
|
| Re: signed int64 pack/unpack [message #184595 is a reply to message #184594] |
Sun, 12 January 2014 16:00 |
Christoph Michael Bec
Messages: 207
Registered: June 2013
Karma: 0
|
Senior Member |
|
|
Jerry Stuckle wrote:
> On 1/12/2014 7:37 AM, Ben Bacarisse wrote:
>> Jerry Stuckle <jstucklex(at)attglobal(dot)net> writes:
>>
>>> On 1/11/2014 10:11 PM, Ben Bacarisse wrote:
>>>> Jerry Stuckle <jstucklex(at)attglobal(dot)net> writes:
>>>>
>>>> > On 1/11/2014 9:42 PM, Ben Bacarisse wrote:
>>>> >> cameron7(at)gmail(dot)com writes:
>>>> >>
>>>> >>> I'm having a bit of trouble figuring out a straightforward way to
>>>> >>> "pack" / "unpack" a signed int64.
>>>> >>>
>>>> >>> I'm on a 64-bit machine, but the 'i' and 'I' are coming back with
>>>> >>> 4-byte length.
>>>> >>
>>>> >> There is no explicit format for 64-bit ints. The i/I formats are
>>>> >> used
>>>> >> for a native "int" but that need not be 64 bits on a 64-bit machine.
>>>> >> The function is borrowed from Perl whose documentation says a bit
>>>> >> more
>>>> >>
>>>> >> This 'integer' is _at_least_ 32 bits wide. Its exact size
>>>> >> depends on
>>>> >> what a local C compiler calls 'int'.
>>>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^
>>>> >> On my 64-bit Linux machine, 'int' is 32 bits.
>>>> >
>>>> > On my 64 bit Linux system, 'int' is 64 bits. But then I have the 64
>>>> > bit PHP version (and 64 bit OS) loaded.
>>>>
>>>> Yes, me too.
>>>>
>>>> > What do you get when you do:
>>>> >
>>>> > <?php
>>>> > echo PHP_INT_MAX . "\n";
>>>> > ?>
>>>>
>>>> Same as you but I don't think that's relevant. PHP stores values
>>>> internally using a C long. I think the Perl document describes what's
>>>> really happening.
>>>
>>> It is totally relevant. You can't operate on 64 bit values if you
>>> don't have 64 bit support.
>>
>> The Perl documentation seems to describe what's happening. The PHP
>> documentation leaves the nature of the integer of "machine dependent
>> size" unspecified so some clarification is helpful.
>>
>> 64 bit support may be necessary but it seems not to be sufficient. Many
>> 64 systems leave C's "int" as 32 bits, so if pack is still doing what
>> the Perl version describes, it's at least an explanation of what's
>> happening.
>>
>>> And this is PHP, not Perl. Things have changed a *lot* since the
>>> original versions were created. Much of the Perl documentation no
>>> longer applies to PHP (if it ever did).
>>
>> I am sure that's true, but, other than the source, I don't think there
>> is much more to go on. What is your explanation of what's going on?
>>
>
> You can argue, or you can answer our questions and get help.
>
> I see you've already got the answers and all you want to do is argue.
> Sorry I wasted my time and the network's bandwidth trying to help you.
Jerry, please note that Ben, to whom you've replied, is not the OP. :)
--
Christoph M. Becker
|
|
|
|
| Re: signed int64 pack/unpack [message #184596 is a reply to message #184593] |
Sun, 12 January 2014 16:11 |
Jerry Stuckle
Messages: 2598
Registered: September 2010
Karma: 0
|
Senior Member |
|
|
On 1/12/2014 10:25 AM, cameron7(at)gmail(dot)com wrote:
> I guess I'll reply with my own solution. Turns out there are 2 separate issues here.
>
> 1) the "signed" bit needs to be flipped back to 0 before converting to int. This is because it appears the conversion from base 2 to base 10 will always assume the end value is positive.
> 2) Because of this, the rest of the bits need to be flipped.
>
True. base_convert
> So I added 2 arguments to the function, one will tell us if we're supposed to be checking for a "signed bit", and the other will tell us if we need to flip bits.
>
> As you'll see I'm still off by 1 in the end result, but I'm sure that's just in the details.
>
That's because when you use 2's compliment, you need to flip the bits
and add one.
>
> I hope you'll all agree, this is really ugly, but seems to work :)
>
> <?php
>
> $i64 = -8223372036854775807;
>
> echo "PACKING: ".$i64.PHP_EOL;
>
> list(,$binary) = getBinaryString($i64, 64);
>
> list(,$i64a) = unpack('N', pack('N', base_convert(substr($binary, 0, 32), 2, 10)));
> list(,$i64b) = unpack('N', pack('N', base_convert(substr($binary, 32, 32), 2, 10)));
>
> list($flag,$i64c)= getBinaryString($i64a, 32, true, true);
> list(,$i64d) = getBinaryString($i64b, 32, false, true);
>
> $unpack = base_convert($i64c.$i64d,2,10);
> $unpack *= $flag ? -1 : 1;
>
> echo "UNPACKED: ".$unpack.PHP_EOL;
>
> function getBinaryString($packed,$bits,$check_signed = false, $flip = false){
> $a = 0x01;
> $binary = '';
> for($x=0; $x<$bits; $x++){
> $binary .= (int)(bool)($a & $packed);
> $a = $a << 1;
> }
> $signed_flag = false;
> if($check_signed && substr($binary,$bits-1,1) === '1'){
> $signed_flag = true;
> }
> if($flip){
> for($x=0;$x<strlen($binary);$x++){
> $binary[$x] = $binary[$x] === '1' ? '0' : '1';
> }
> }
> if($signed_flag){
> $binary = substr($binary,0,$bits-1).'0';
> }
> return array($signed_flag, strrev($binary));
> }
>
I've seen worse :)
However, you might want to check some other functions. The gmp_xxx
functions will work on longer numbers; so will the BCD functions.
There are some suggestions and links under base_convert() in the online
documentation at http://us1.php.net/manual/en/function.base-convert.php.
Of course, if you're on a shared host and can't get them to install the
libraries for you, you're out of luck.
--
==================
Remove the "x" from my email address
Jerry Stuckle
jstucklex(at)attglobal(dot)net
==================
|
|
|
|
| Re: signed int64 pack/unpack [message #184598 is a reply to message #184577] |
Sun, 12 January 2014 16:37 |
Christoph Michael Bec
Messages: 207
Registered: June 2013
Karma: 0
|
Senior Member |
|
|
cameron7(at)gmail(dot)com wrote:
> I'm having a bit of trouble figuring out a straightforward way to
> "pack" / "unpack" a signed int64.
>
> I'm on a 64-bit machine, but the 'i' and 'I' are coming back with
> 4-byte length. And of course even though double comes back with 8
> length it's a floating point and thus loses precision.
>
> Any thoughts? Any help would be greatly appreciated.
>
> BTW, unsigned int64 seems relatively straightforward, but signed
> int64 has been unusually difficult to figure out.
It seems to me that it should be possible to pack 64bit numbers by
splitting them into two 32bit numbers and packing the "halves", as Ben
already noticed. Unpacking should be possible by doing the reverse
operations. It shouldn't really matter in this case if you're dealing
with signed or unsigned values, if your platform supports the respective
64bit values.
I can't test with 64bit Integers as I'm on Windows with
PHP_INT_MAX==2147483647. However, I've written the following functions
for working with 32bit integers by relying on the 'n' format of
pack()/unpack(). You may try to adjust the functions for 64bit by using
the 'N' format and replacing the literal "16" with "32".
function packInt($number)
{
$high = $number >> 16;
$low = $number & ((1 << 16) - 1);
return pack('n2', $high, $low);
}
function unpackInt($string)
{
$strings = unpack('n2', $string);
return ($strings[1] << 16) | $strings[2];
}
--
Christoph M. Becker
|
|
|
|
| Re: signed int64 pack/unpack [message #184608 is a reply to message #184595] |
Sun, 12 January 2014 21:09 |
Jerry Stuckle
Messages: 2598
Registered: September 2010
Karma: 0
|
Senior Member |
|
|
On 1/12/2014 11:00 AM, Christoph Michael Becker wrote:
> Jerry Stuckle wrote:
>
>> On 1/12/2014 7:37 AM, Ben Bacarisse wrote:
>>> Jerry Stuckle <jstucklex(at)attglobal(dot)net> writes:
>>>
>>>> On 1/11/2014 10:11 PM, Ben Bacarisse wrote:
>>>> > Jerry Stuckle <jstucklex(at)attglobal(dot)net> writes:
>>>> >
>>>> >> On 1/11/2014 9:42 PM, Ben Bacarisse wrote:
>>>> >>> cameron7(at)gmail(dot)com writes:
>>>> >>>
>>>> >>>> I'm having a bit of trouble figuring out a straightforward way to
>>>> >>>> "pack" / "unpack" a signed int64.
>>>> >>>>
>>>> >>>> I'm on a 64-bit machine, but the 'i' and 'I' are coming back with
>>>> >>>> 4-byte length.
>>>> >>>
>>>> >>> There is no explicit format for 64-bit ints. The i/I formats are
>>>> >>> used
>>>> >>> for a native "int" but that need not be 64 bits on a 64-bit machine.
>>>> >>> The function is borrowed from Perl whose documentation says a bit
>>>> >>> more
>>>> >>>
>>>> >>> This 'integer' is _at_least_ 32 bits wide. Its exact size
>>>> >>> depends on
>>>> >>> what a local C compiler calls 'int'.
>>>> > ^^^^^^^^^^^^^^^^^^^^^^^^^^^
>>>> >>> On my 64-bit Linux machine, 'int' is 32 bits.
>>>> >>
>>>> >> On my 64 bit Linux system, 'int' is 64 bits. But then I have the 64
>>>> >> bit PHP version (and 64 bit OS) loaded.
>>>> >
>>>> > Yes, me too.
>>>> >
>>>> >> What do you get when you do:
>>>> >>
>>>> >> <?php
>>>> >> echo PHP_INT_MAX . "\n";
>>>> >> ?>
>>>> >
>>>> > Same as you but I don't think that's relevant. PHP stores values
>>>> > internally using a C long. I think the Perl document describes what's
>>>> > really happening.
>>>>
>>>> It is totally relevant. You can't operate on 64 bit values if you
>>>> don't have 64 bit support.
>>>
>>> The Perl documentation seems to describe what's happening. The PHP
>>> documentation leaves the nature of the integer of "machine dependent
>>> size" unspecified so some clarification is helpful.
>>>
>>> 64 bit support may be necessary but it seems not to be sufficient. Many
>>> 64 systems leave C's "int" as 32 bits, so if pack is still doing what
>>> the Perl version describes, it's at least an explanation of what's
>>> happening.
>>>
>>>> And this is PHP, not Perl. Things have changed a *lot* since the
>>>> original versions were created. Much of the Perl documentation no
>>>> longer applies to PHP (if it ever did).
>>>
>>> I am sure that's true, but, other than the source, I don't think there
>>> is much more to go on. What is your explanation of what's going on?
>>>
>>
>> You can argue, or you can answer our questions and get help.
>>
>> I see you've already got the answers and all you want to do is argue.
>> Sorry I wasted my time and the network's bandwidth trying to help you.
>
> Jerry, please note that Ben, to whom you've replied, is not the OP. :)
>
Yes, I know. But he keeps referring to information that is > 15 years
old. PHP has changed a lot since then, but we just wants to argue.
--
==================
Remove the "x" from my email address
Jerry Stuckle
jstucklex(at)attglobal(dot)net
==================
|
|
|
|
| Re: signed int64 pack/unpack [message #184609 is a reply to message #184594] |
Sun, 12 January 2014 22:30 |
Ben Bacarisse
Messages: 82
Registered: November 2013
Karma: 0
|
Member |
|
|
Jerry Stuckle <jstucklex(at)attglobal(dot)net> writes:
<snip>
> You can argue, or you can answer our questions and get help.
What questions didn't I answer? Who is included in "our"?
As best I can tell you asked me one (which I answered) and I don't think
anyone else as has asked me any.
> I see you've already got the answers and all you want to do is
> argue. Sorry I wasted my time and the network's bandwidth trying to
> help you.
Have I done something to make you angry? We were having what I though
was civil exchange about a technical matter, and then this.
--
Ben.
|
|
|
|
| Re: signed int64 pack/unpack [message #184610 is a reply to message #184608] |
Sun, 12 January 2014 22:35 |
Ben Bacarisse
Messages: 82
Registered: November 2013
Karma: 0
|
Member |
|
|
Jerry Stuckle <jstucklex(at)attglobal(dot)net> writes:
<snip>
> Yes, I know. But he keeps referring to information that is > 15 years
> old. PHP has changed a lot since then, but we just wants to argue.
Old information can be accurate. I don't know if it is in this case,
but it explains the OP's results better than the more recent PHP
documentation does. When I asked you your opinion of what's happening,
you went all angry and weird.
--
Ben.
|
|
|
|
| Re: signed int64 pack/unpack [message #184612 is a reply to message #184610] |
Sun, 12 January 2014 22:59 |
Jerry Stuckle
Messages: 2598
Registered: September 2010
Karma: 0
|
Senior Member |
|
|
On 1/12/2014 5:35 PM, Ben Bacarisse wrote:
> Jerry Stuckle <jstucklex(at)attglobal(dot)net> writes:
> <snip>
>> Yes, I know. But he keeps referring to information that is > 15 years
>> old. PHP has changed a lot since then, but we just wants to argue.
>
> Old information can be accurate. I don't know if it is in this case,
> but it explains the OP's results better than the more recent PHP
> documentation does. When I asked you your opinion of what's happening,
> you went all angry and weird.
>
Exactly. You don't know if the information is accurate. Yet you
continue to argue about it.
And more recent PHP documentation is more accurate because it relates to
the CURRENT code base, not one that is over 15 years old.
As for getting "angry and weird" - maybe I'm just tired of arguing with
people who base their position on old and/or inaccurate information (or
translation thereof), and refuse to look at any other possibility.
--
==================
Remove the "x" from my email address
Jerry Stuckle
jstucklex(at)attglobal(dot)net
==================
|
|
|
|
| Re: signed int64 pack/unpack [message #184615 is a reply to message #184593] |
Sun, 12 January 2014 23:27 |
Ben Bacarisse
Messages: 82
Registered: November 2013
Karma: 0
|
Member |
|
|
cameron7(at)gmail(dot)com writes:
> I guess I'll reply with my own solution. Turns out there are 2
> separate issues here.
Here's mine. You might have to alter the order of the two 32-bit
components to match your external format. I've used the same names you
might be able to drop it into your tests.
echo "PACKING: ".$i64.PHP_EOL;
$ps = pack('N2', $i64 >> 32, $i64 & 0xffffffff);
list(, $hi, $lo) = unpack('N2', $ps);
$unpack = ($hi << 32) + $lo;
echo "UNPACKED: ".$unpack.PHP_EOL;
--
Ben.
|
|
|
|
|
|
|
|
| Re: signed int64 pack/unpack [message #184618 is a reply to message #184617] |
Mon, 13 January 2014 01:20 |
Ben Bacarisse
Messages: 82
Registered: November 2013
Karma: 0
|
Member |
|
|
cameron7(at)gmail(dot)com writes:
> Although it does lead me to the question, how will we do unsigned
> int64 :)
:) indeed! PHP does not support unsigned integers so it's a genuine
problem. In languages without unsigned integers, you can get by only
when the larges unsigned quantity you every use is <= INT_MAX.
> For another day I guess...
The answer will depend on what the code must do with the results.
Keeping them as signed might work for a lot of what gets done with it.
--
Ben.
|
|
|
|
| Re: signed int64 pack/unpack [message #184621 is a reply to message #184612] |
Mon, 13 January 2014 01:59 |
Ben Bacarisse
Messages: 82
Registered: November 2013
Karma: 0
|
Member |
|
|
Jerry Stuckle <jstucklex(at)attglobal(dot)net> writes:
> On 1/12/2014 5:35 PM, Ben Bacarisse wrote:
>> Jerry Stuckle <jstucklex(at)attglobal(dot)net> writes:
>> <snip>
>>> Yes, I know. But he keeps referring to information that is > 15 years
>>> old. PHP has changed a lot since then, but we just wants to argue.
>>
>> Old information can be accurate. I don't know if it is in this case,
>> but it explains the OP's results better than the more recent PHP
>> documentation does. When I asked you your opinion of what's happening,
>> you went all angry and weird.
>>
>
> Exactly. You don't know if the information is accurate. Yet you
> continue to argue about it.
>
> And more recent PHP documentation is more accurate because it relates
> to the CURRENT code base, not one that is over 15 years old.
I could not find an explanation in the PHP documentation. I am sure it
is more up-to-date (and more applicable) but there was no answer there
that I could see.
As it happens, the current code base references Perl for the behaviour.
A comment at the start of pack.c says:
/* pack() idea stolen from Perl (implemented formats behave the same as there)
* Implemented formats are Z, A, a, h, H, c, C, s, S, i, I, l, L, n, N, f, d, x, X, @.
*/
So the code authors would have pointed me to Perl's pack function had I
not already gone there and, as documented, the code for the i and I are
based on the size of a C int (on the build system):
case 'i':
case 'I':
while (arg-- > 0) {
php_pack(argv[currentarg++], sizeof(int), int_map, &output[outputpos]);
outputpos += sizeof(int);
}
break;
> As for getting "angry and weird" - maybe I'm just tired of arguing
> with people who base their position on old and/or inaccurate
> information (or translation thereof), and refuse to look at any other
> possibility.
I am open to other possibilities. What is your opinion about what i and
I do and the size of integer they work with?. I didn't pretend to know
what is actually going on, just what seemed to be going on.
--
Ben.
|
|
|
|
Calendar
Search
Help
Members
Register
Login
Home
]