Large PDF with large images causing `malloc` warnings

[Ryan Daulton]

What did you do?

When generating a PDF — particularly large ones — I get consistent warnings, and sometimes crashes that I cannot find solutions for. I have been searching through the issues in this repo for some time now, and cannot find adequate solutions elsewhere. I am getting the feeling that I'm simply using the package incorrectly and could use some help.

Here is an example of the logs I'm getting:

my_app(2358,0x16d473000) malloc: can't allocate region
:*** mach_vm_map(size=41582592, flags: 100) failed (error code=3)
my_app((2358,0x16d473000) malloc: *** set a breakpoint in malloc_error_break to debug
my_app((2358,0x16d473000) malloc: can't allocate region
:*** mach_vm_map(size=6488064, flags: 100) failed (error code=3)
my_app((2358,0x16d473000) malloc: *** set a breakpoint in malloc_error_break to debug
my_app((2358,0x16d473000) malloc: can't allocate region
:*** mach_vm_map(size=8372224, flags: 100) failed (error code=3)
/// etc...

Each page in my PDF contains an image.

It looks like the memory footprint climbs well outside of what the app expects, and that's why I get the issues. My feeling is that this originates from the compression of each image in the PDF.

When I place an exception breakpoint to catch the crash, it often catches line 214 of PDFGraphics.swift, where the compression occurs.

What did you expect to happen?

Given the note in the README...

Generate PDF files directly to handle large PDF files (Details)

...I was not expecting this particular workload to cause memory issues.

What happened instead?

It seems that this only begins to happen when I approach ~200 pages in the PDF, most of which contain an image. The PDF gen works completely fine when dealing with ~100 pages.

TPPDF Environment

TPPDF version: SPM 2.3.2
Xcode version: 12.3
Swift version: 5

Demo Code / Project

Each page in my PDF corresponds to a record. The imagery for each record in the PDF comes from a Data object, originating from the Core Data store. Each core data managed object has been retrieved (non-fault) before the PDF generation begins.

For each PDF, I loop through the records and create a 2-column page for each; left column contains textual information like expense title, date, notes, etc, while the right column is an image. Each raw image can range from 600px X 900px to upwards of 2000px X 4000px at the high end - though rare.

Naturally, I use the built in compression to bring the images down to size.

for i in 0..<records {
    let record = records[i]
    let section: PDFSection = PDFSection(columnWidths: [0.36, 0.64])
    section.columnMargin = 20

    // Header
    self.document.add(text: TITLE)

    // - - Column 1 - - //
    /// Date
    self.document.add(text: DATE)
    /// Total
    self.document.add(text: EXPENSE_TOTAL)
    /// Description
    self.document.add(text: DESC)
    
    // - - Column 2 - - //
    /// Record snapshot /// ---> This is where the issue seems to originate from
    var recordImage: PDFImage = PDFImage(image: UIImage(data: record.imageData!)!)
    recordImage.options = [.compress]
    recordImage.sizeFit = .widthHeight
    recordImage.quality = 0.85
    section.columns[1].add(.center, image: recordImage)
    document.add(section: section)
}

It is worth noting that I have placed the PDF process in an async block, similar to what was discussed in #162.

Questions

Is there a better way to go about this? Or some way for me to tell the PDFGenerator how many images and of what size to expect? Perhaps I am missing something, or just going about this very inefficiently. Either way, some feedback would be very helpful. Thank you.

[Philip Niedertscheider]

Thanks for the detailed issue report.
On first thought this seems like a memory issue in UIKit, can you please add device info such as device type, OS version, memory-usage etc.?

[Ryan Daulton]

Happens on both devices:

Device: iPhone 7
OS: iOS 14.4
Peak Memory Usage: 368 MB

Device: iPhone 12 Pro
OS: iOS 14.3
Peak Memory Usage: 736 MB

[Ryan Daulton]

@philprime any ideas on this?

[Philip Niedertscheider]

The error malloc: can't allocate region is new to me, and I also can't see a specific issue in your sample code.
So this might be a limitation of the render algorithm, as it needs to be in-memory.

I will mark this issue as an enhancement/performance/bug issue for now, and any further investigation is appreciated

[Ryan Daulton]

@philprime I've noticed that with judicial use of autoreleasepool, the problem improves, but does not go away

[Ryan Daulton]

Wondering if there's been any action on this issue over the last year or so. Thank you!

[Philip Niedertscheider]

Hi @rldaulton there hasn't been any work from my side.
I still believe this is memory issue with CoreGraphics which is not able to deal with the amount of data.
Regarding the mention of handling large PDF files in the README, I added that note because of the amount of elements, rather than the file size. I will consider removing that note.

If no-one else is able to help, we could consider filing a bug report with Apple, and asking CoreGraphics Engineers for feedback.

[Ryan Daulton]

@philprime I understand, thank you