Filesize >2GB Linux NAS

Locked
Pernod
getID3() contributor
Posts: 103
Joined: Sat Mar 21, 2009 5:30 pm
Are you a spambot?: no
Location: London, UK
Contact:

Re: Filesize >2GB Linux NAS

Post by Pernod » Wed Feb 13, 2013 4:40 pm

My preferred patch to getid3.php would be:

Code: Select all

$commandline = 'ls -l "'.$this->info['filenamepath'].'" | awk \'{print $5}\'';
$dir_output = `$commandline`;
$real_filesize = (float) trim($dir_output);

soulhunter
getID3() v1 developer
Posts: 40
Joined: Mon Jan 11, 2010 5:37 pm
Are you a spambot?: no
Location: Kaliningrad, Russia
Contact:

Re: Filesize >2GB Linux NAS

Post by soulhunter » Wed Feb 13, 2013 8:28 pm

Why are you using 32-bit linux?

Pernod
getID3() contributor
Posts: 103
Joined: Sat Mar 21, 2009 5:30 pm
Are you a spambot?: no
Location: London, UK
Contact:

Re: Filesize >2GB Linux NAS

Post by Pernod » Wed Feb 13, 2013 10:40 pm

soulhunter wrote:Why are you using 32-bit linux?
Because that's what comes pre-installed on Synology/QNAP NAS, problem?

soulhunter
getID3() v1 developer
Posts: 40
Joined: Mon Jan 11, 2010 5:37 pm
Are you a spambot?: no
Location: Kaliningrad, Russia
Contact:

Re: Filesize >2GB Linux NAS

Post by soulhunter » Fri Feb 15, 2013 9:07 am

ls may exist and may NOT exist in the system, so this is not the best solution.
The most cross-platform solution is to use 'stat' system call, which is available in all POSIX systems and is used internally by PHP itself.

Try this command: 'stat -c%s filepath' (on Darwin system it would look like 'stat -f%z filepath').
Appropreate fix will be included in the next release very soon.

Pernod
getID3() contributor
Posts: 103
Joined: Sat Mar 21, 2009 5:30 pm
Are you a spambot?: no
Location: London, UK
Contact:

Re: Filesize >2GB Linux NAS

Post by Pernod » Fri Feb 15, 2013 11:24 am

soulhunter wrote:Try this command: 'stat -c%s filepath'
This works well, so hope to see it in next release.

In getid3.php after the line $this->info['filesize'] = $real_filesize it sets an error, shouldn't this be a warning?

Not specifically GetID3 related... I have users running getid3 on QNAP NAS (TS-259 PRO+) and PHP is failing to recognise files over 2GB. Functions is_file(), is_readable() both return false, files under 2GB return true as expected. Any ideas what could cause this as I don't have the same issue on Synology NAS.

soulhunter
getID3() v1 developer
Posts: 40
Joined: Mon Jan 11, 2010 5:37 pm
Are you a spambot?: no
Location: Kaliningrad, Russia
Contact:

Re: Filesize >2GB Linux NAS

Post by soulhunter » Fri Feb 15, 2013 12:02 pm

Pernod wrote:In getid3.php after the line $this->info['filesize'] = $real_filesize it sets an error, shouldn't this be a warning?
I think It should. We'll fix it.
Pernod wrote:Not specifically GetID3 related... I have users running getid3 on QNAP NAS (TS-259 PRO+) and PHP is failing to recognise files over 2GB. Functions is_file(), is_readable() both return false, files under 2GB return true as expected. Any ideas what could cause this as I don't have the same issue on Synology NAS.
I heard about this problem, but did not have a test machine to reproduce. This is due to the same 32bit integer problem, as functions is_file(), filesize(), etc internally use stat() php function, which is subject to those restrictions. What PHP versions do you, and your users, use? I'll look for the solution. I think this behavior differs in various PHP versions.

Pernod
getID3() contributor
Posts: 103
Joined: Sat Mar 21, 2009 5:30 pm
Are you a spambot?: no
Location: London, UK
Contact:

Re: Filesize >2GB Linux NAS

Post by Pernod » Fri Feb 15, 2013 12:29 pm

soulhunter wrote:What PHP versions do you, and your users, use? I'll look for the solution. I think this behavior differs in various PHP versions.
The QNAP users use PHP 5.3.14. I'll ask them to try the stat workaround for filesize and report back. I use 5.3.15 on my Synology and don't have any unexpected issues.

Pernod
getID3() contributor
Posts: 103
Joined: Sat Mar 21, 2009 5:30 pm
Are you a spambot?: no
Location: London, UK
Contact:

Re: Filesize >2GB Linux NAS

Post by Pernod » Fri Feb 15, 2013 3:48 pm

I said previously that 'stat -c%s filepath' worked on the Synology, I was mistaken, it also fails on a QNAP. It works when I telnet into the NAS but fails from PHP with the following error:

Code: Select all

BusyBox v1.16.1 (2012-12-11 13:29:57 CST) multi-call binary.

Usage: stat [OPTIONS] FILE...

Display file (default) or filesystem status

Options:
	-f	Display filesystem status
	-L	Follow links
	-t	Display info in terse form

stat: invalid option -- c
So I'd still prefer 'ls -l "'.$this->info['filenamepath'].'" | awk \'{print $5}\'';' to be used, or parse the result of stat without any commandline options.

The following works but not sure if 'Size:' is locale dependant:

Code: Select all

$commandline = 'stat '.escapeshellarg($this->info['filenamepath']);
$dir_output = `$commandline`;
if (preg_match('#Size:[ ]+([0-9]+)#i', $dir_output, $matches)) {
	$real_filesize = (float) $matches[1];
}

soulhunter
getID3() v1 developer
Posts: 40
Joined: Mon Jan 11, 2010 5:37 pm
Are you a spambot?: no
Location: Kaliningrad, Russia
Contact:

Re: Filesize >2GB Linux NAS

Post by soulhunter » Fri Feb 15, 2013 4:03 pm

Let me see the result of 'stat filepath' call from your machine.

Pernod
getID3() contributor
Posts: 103
Joined: Sat Mar 21, 2009 5:30 pm
Are you a spambot?: no
Location: London, UK
Contact:

Re: Filesize >2GB Linux NAS

Post by Pernod » Fri Feb 15, 2013 4:05 pm

soulhunter wrote:Let me see the result of 'stat filepath' call from your machine.

Code: Select all

  File: `2012.2009.720p.BluRay.x264-METiS.mkv'
  Size: 7042759227      Blocks: 13755400   IO Block: 4096   regular file
Device: fd00h/64768d    Inode: 12771668    Links: 1
Access: (0755/-rwxr-xr-x)  Uid: ( 1024/   admin)   Gid: (  100/   users)
Access: 2013-02-14 23:38:08.000000019 +0000
Modify: 2010-02-03 15:37:58.000000001 +0000
Change: 2012-10-20 02:39:24.1349288946 +0100

soulhunter
getID3() v1 developer
Posts: 40
Joined: Mon Jan 11, 2010 5:37 pm
Are you a spambot?: no
Location: Kaliningrad, Russia
Contact:

Re: Filesize >2GB Linux NAS

Post by soulhunter » Fri Feb 15, 2013 4:38 pm

Hm, stat -t, please

Pernod
getID3() contributor
Posts: 103
Joined: Sat Mar 21, 2009 5:30 pm
Are you a spambot?: no
Location: London, UK
Contact:

Re: Filesize >2GB Linux NAS

Post by Pernod » Fri Feb 15, 2013 4:45 pm

soulhunter wrote:Hm, stat -t, please

Code: Select all

2012.2009.720p.BluRay.x264-METiS.mkv 7042759227 13755400 81ed 1024 100 fd00 12771668 1 0 0 1360885088 1265211478 1350697164 4096
When you're happy with a 'filesize' solution could we also have 'date_modified' returned in a similar way?

soulhunter
getID3() v1 developer
Posts: 40
Joined: Mon Jan 11, 2010 5:37 pm
Are you a spambot?: no
Location: Kaliningrad, Russia
Contact:

Re: Filesize >2GB Linux NAS

Post by soulhunter » Tue Feb 19, 2013 4:04 pm

try 'du filepath' (both from PHP and from console) and let me see the result, please

Pernod
getID3() contributor
Posts: 103
Joined: Sat Mar 21, 2009 5:30 pm
Are you a spambot?: no
Location: London, UK
Contact:

Re: Filesize >2GB Linux NAS

Post by Pernod » Tue Feb 19, 2013 4:13 pm

soulhunter wrote:try 'du filepath' (both from PHP and from console) and let me see the result, please
Both produce:

Code: Select all

6877700 /volume1/video/Movies HD/2012.2009.720p.BluRay.x264-METiS.mkv 
I'd prefer to use stat then we can also return other file details such as modified date.

soulhunter
getID3() v1 developer
Posts: 40
Joined: Mon Jan 11, 2010 5:37 pm
Are you a spambot?: no
Location: Kaliningrad, Russia
Contact:

Re: Filesize >2GB Linux NAS

Post by soulhunter » Tue Feb 19, 2013 5:05 pm

The problem with stat is that it's output and options differ much between systems, so we cannot write a universal command ith options or propely parse it's raw output with regular expressions.
So is with du (options differ).

ls output is locale-sensitive and ls may not exist on some systems, but it appears to be the best way for now, so I finally agree with Pernold's solution:
ls -l filepath | awk '{print $5}'

We'll implement this call for now, but will continue investigating to find the best solution. It will be addressed to v2.0

James Heinrich
getID3() v1 developer
Posts: 1444
Joined: Fri May 04, 2001 4:00 pm
Are you a spambot?: no
Location: Northern Ontario, Canada
Contact:

Re: Filesize >2GB Linux NAS

Post by James Heinrich » Wed Feb 20, 2013 9:05 pm

Code: Select all

ls -l filepath | awk '{print $5}'
is the accepted solution included in v1.9.5

Marking issue as resolved (for now).

Locked