excessive memory allocation - quicktime module

[filmo]

Got the following error:

Fatal error: Allowed memory size of 67108864 bytes exhausted (tried to allocate 64 bytes) in /home/product/phpinc/getID3-1.9.7/getid3/module.audio-video.quicktime.php on line 844

I'm assuming that the file in question is poorly formed, but there should probably be some tests placed in this module to prevent it from entering a race condition when a poorly presented file scanned to prevent this error.

Unfortunately, I do not even know where to begin on this one in terms of fixing it.

Here's the relevant code section:

Code: Select all

			case 'stts': // Sample Table Time-to-Sample atom
				$atom_structure['version']        = getid3_lib::BigEndian2Int(substr($atom_data,  0, 1));
				$atom_structure['flags_raw']      = getid3_lib::BigEndian2Int(substr($atom_data,  1, 3)); // hardcoded: 0x0000
				$atom_structure['number_entries'] = getid3_lib::BigEndian2Int(substr($atom_data,  4, 4));
				$sttsEntriesDataOffset = 8;
				//$FrameRateCalculatorArray = array();
				$frames_count = 0;
				for ($i = 0; $i < $atom_structure['number_entries']; $i++) {
					$atom_structure['time_to_sample_table'][$i]['sample_count']    = getid3_lib::BigEndian2Int(substr($atom_data, $sttsEntriesDataOffset, 4));
					$sttsEntriesDataOffset += 4;
					$atom_structure['time_to_sample_table'][$i]['sample_duration'] = getid3_lib::BigEndian2Int(substr($atom_data, $sttsEntriesDataOffset, 4));
					$sttsEntriesDataOffset += 4;

I seem to recall a similar problem a few versions ago. Basically either too many reads of a particular atom or the end of the Atom is not correctly recognized and it attempts to continue reading the file. (I'm assuming there's some structure to the atom that getID3 is reading and that it looks for the 'end' point to determine when to stop reading. If the end point were corrupted or missing, perhaps it's continuing to scan the file looking for it?? On a large 400mb file, it would easily exceed our 64mb per script memory limit.)

[filmo]

On further testing it appears that this particular file enters the case 'stts' 67865 times and that the length of atomdata = 382584 bytes. There should probably be some check to make sure this doesn't happen, but I'm not sure where to put it. If it enters this fault condition, getID3 should return an error or fault condition rather than dying.

[filmo]

This is the line 844:
$atom_structure['time_to_sample_table'][$i]['sample_count'] = getid3_lib::BigEndian2Int(substr($atom_data, $sttsEntriesDataOffset, 4));

[James Heinrich]

Thanks for the bug report. I'll look into it next week sometime.

If you can provide a copy of the sample file that would be a great help. You can email or PM me a link to it.

[James Heinrich]

Unfortunately this data ($atom_structure['time_to_sample_table'][$i]['sample_count']) isn't some mostly-irrelevant value that can be easily skipped -- it is used to calculate frame_rate.

Quicktime files are generally scanned the whole way through due to the way they are structured, and the not-interesting-to-getID3 parts are skipped over. I have just added a small change that should prevent running out of memory when parsing large Quicktime files, where it's not uncommon to have very-large 'mdat' atoms. I now limit the read data to at most half of the php.ini configured memory_limit:

Code: Select all

$info['quicktime'][$atomname] = $this->QuicktimeParseAtom($atomname, $atomsize, $this->fread(min($atomsize, round($this->getid3->memory_limit / 2))), $offset, $atomHierarchy, $this->ParseAllPossibleAtoms);

I also added a similar check to the number of "stts" entries scanned, this time scaled to the amount of RAM available. I just scanned some large (around 2GB) Quicktime video files and they appear to work just fine now. One file had 74050 entries for the 'stts' atom and that scanned fine with the original code when PHP had 1GB RAM available, but failed as you describe when configured at 64MB. With the new limiting code it now scans fine with 64MB, with a warning generated to note that not all entries were scanned:

QuickTime atom "stts" has 74050 but only scanning the first 6710 entries due to limited PHP memory available (64MB).

Code: Select all

$max_stts_entries_to_scan = min(floor($this->getid3->memory_limit / 10000), $atom_structure['number_entries']);
if ($max_stts_entries_to_scan < $atom_structure['number_entries']) {
	$info['warning'][] = 'QuickTime atom "stts" has '.$atom_structure['number_entries'].' but only scanning the first '.$max_stts_entries_to_scan.' entries due to limited PHP memory available ('.floor($this->getid3->memory_limit / 1048576).'MB).';
}
for ($i = 0; $i < $max_stts_entries_to_scan; $i++) {

Fix applied:
https://github.com/JamesHeinrich/getID3 ... 16c7d3bb80