Nitro Pro PDF JavaScript document.flattenPages JSStackFrame stack-based use-after-free vulnerability

We discussed this vulnerability during Episode 84 on 21 September 2021

This Talos report covers a non-trivial issue where a stack pointer is used after it went out of scope when invoking JS bindings, which are provided to document creators by Nitro Pro PDF for automating aspects of the document. When one of these bindings needs to be executed by the SpiderMonkey library, the js32u.dll!js_Invoke function is used to create stack space and push a JSStackFrame object to be used by the invoked binding. It will then link the newly created stack frame to the parent JSContext in a linked list. The main thing to note here is these stack frames are backed by stack memory allocated by js_Invoke, meaning if that function returns, these stack frames go out of scope.

The problem comes into play when the Document.flattenPages() functionality for flattening the PDF is invoked in a way that can fail (such as having annotations for pages being referenced incorrectly). The application’s registered JSFunctionSpec callback will first register a structured exception handler before allocating the necessary space for it’s stack frame. If an exception is thrown, all exception handlers that were registered during compilation will be called, the first of which will convert the exception to a different exception and throw that. This results in the next exception handler in the function being skipped, which means the exception can be caught by other C++ exception handlers.

This results in code that’s responsible for cleaning up the stack (popping the stack frames that were used to execute the binding and restoring the context) being skipped, which leads to stale JSStackFrame pointers remaining attached to the parent context. Later on after the invoke function has finished, the SpiderMonkey library will end up writing to the stale pointer in the linked list of stack frames, which can be leveraged for code execution.