The Qualys Threat Research Unit (TRU) has discovered three vulnerabilities in RenderDoc. This blog will delve into the details of these three newly discovered vulnerabilities found within RenderDoc’s implementation. As part of our ongoing commitment to safeguard digital assets and strengthen cybersecurity, it’s critical to understand these weaknesses, their potential impact, and the remedial steps organizations should take. Qualys strongly advises security teams to apply patches for these vulnerabilities as soon as possible.  

RenderDoc, a standalone graphics debugger of immense utility, has emerged as an invaluable tool for developers. As an open-source software with an MIT license, RenderDoc provides a comprehensive platform for detailed introspection and single-frame capture across various applications, including Vulkan, D3D11, OpenGL & OpenGL ES, and D3D12. It supports multiple operating systems such as Windows, Linux, Android, and even the Nintendo Switch™. However, like any software, RenderDoc isn’t immune to vulnerabilities.  

RenderDoc operates by LD_PRELOADing the shared library librenderdoc.so into the application to be debugged, and this library immediately starts a server thread. This server thread listens on TCP port 38920 across all network interfaces and stands by for client connections. Despite this function’s immense power and utility, it is within this exact implementation that the research team discovered three vulnerabilities.  

Vulnerability Analysis:

• CVE-2023-33865: This vulnerability is a symlink vulnerability exploitable by any local attacker with no privilege requirement. The attack could allow the adversary to gain the privileges of the RenderDoc user. The intriguing nature of this vulnerability lies in its details makingit an intellectually stimulating challenge to exploit.   

• CVE-2023-33864: This integer underflow leads to a heap-based buffer overflow that any remote attacker could exploit to execute arbitrary code on the host machine. The exploitation of this vulnerability utilizes a unique malloc exploitation technique that has proven reliable and efficient despite all the latest protections, such as glibc, ASLR, PIE, NX, and stack canaries.  

• CVE-2023-33863: This integer overflow culminates in a heap-based buffer overflow and may be exploitable by a remote attacker to run arbitrary code on the machine. At this point, we have not attempted to exploit this vulnerability, but its potential threat to security cannot be ignored.  

Importantly, RenderDoc swiftly addressed all three vulnerabilities upon receiving coordinated disclosure from our research team within just a few hours. RenderDoc released version 1.27, which includes the necessary fixes to mitigate these vulnerabilities.  

Technical Details of RenderDoc Vulnerabilities: 

You can find the technical details of these vulnerabilities at: 

https://www.qualys.com/2023/06/06/renderdoc/renderdoc.txt

Digging Deeper: An Examination of CVE-2023-33865, a Symlink Vulnerability in /tmp/RenderDoc  

At the heart of CVE-2023-33865 lies a symlink vulnerability closely associated with the /tmp/RenderDoc directory. As soon as the shared library, librenderdoc.so, is LD_PRELOADed into the application under debug, the library_loaded() function is triggered, initiating two distinct actions.  

First, it constructs the directory /tmp/RenderDoc. Interestingly, it also repurposes this directory if it pre-exists, regardless of whether it belongs to the user currently operating RenderDoc (for our discussion, we’ll refer to this user as ‘Alice’).  

Secondly, it opens a log file fitting the structure /tmp/RenderDoc/RenderDoc_app_YYYY.MM.DD_hh.mm.ss.log. If this log file doesn’t exist, it’s freshly created, and data gets appended in a consistent, uninterrupted flow.  

These actions harbor a significant security loophole. An adversary can preempt Alice by creating the /tmp/RenderDoc directory ahead of her. This directory can be filled with countless symlinks following the pattern /tmp/RenderDoc/RenderDoc_app_YYYY.MM.DD_hh.mm.ss.log, each pointing towards an arbitrary file within the system. Consequently, when Alice runs RenderDoc, the linked file is either created (if it doesn’t already exist) or written into, and it happens under Alice’s privileges.  

The adversary can inject arbitrary strings into the chosen file through this strategy. This can be done by transmitting these strings to RenderDoc over TCP port 38920. However, there’s a roadblock for the attacker: RenderDoc adds a non-manipulable header before every incoming string. If the transmitted string contains newline characters, RenderDoc divides it into multiple lines, each initiated with the uncontrolled header. This lack of control over the header thwarts the attacker’s attempts to escalate privileges via Alice’s standard dotfiles, such as .profile, .bashrc, .ssh/authorized_keys, etc.  

Our investigation into the uncontrolled-header issue led to a two-step solution:  

Step 1: Symlink to Directory Creation Exploit:  

First, we repurposed RenderDoc’s symlink vulnerability into an arbitrary directory creation exploit. We wrote ‘SYSTEMD=.config/systemd’ into the .config/user-dirs.defaults file in Alice’s home directory. This manipulation resulted in an auto-created directory, .config/systemd, upon Alice’s next login. Overcoming the uncontrolled-header problem involved writing a string longer than 512 bytes, causing fgets() to start a line with our data.  

 Step 2: From Symlink to Arbitrary Code Execution:  

Next, we transitioned the symlink vulnerability into arbitrary code execution by writing into Alice’s .config/systemd/user.conf file. We inserted ‘DefaultEnvironment=LD_PRELOAD=/var/tmp/shell.so’, causing systemd to execute our shared library with Alice’s privileges upon her next login. We used \r as a line delimiter to bypass the uncontrolled-header problem, given RenderDoc only appends a header after \n.  

A Closer Look at CVE-2023-33864: From Integer Underflow to Heap-Based Buffer Overflow  

When a client connects to librenderdoc.so’s server thread on TCP port 38920, the client sends a handshake packet first, containing a “client name” string. The server processes this string through a series of operations. Initially, the server allocates a 64KB intermediary buffer and reads the client’s handshake packet into it. Subsequently, the server extracts the length of the client-name string (denoted as ‘len’) from this buffer and allocates a string buffer of equivalent size.  

Depending on the size of the client name, the server follows one of two paths. For client names longer than 10MB, the server reads the name directly into the string buffer. Otherwise, it reads the client name into the intermediary buffer and then copies it into the string buffer. However, the actual reading process follows a slightly nuanced path. ReadLargeBuffer() reads all but the last 128 bytes of the client name directly into the string buffer and the previous 128 bytes into the intermediary buffer. These bytes are then copied into the string buffer.  

An error occurs with the ReadFromExternal() function, which wrongly assumes that m_InputSize (the total bytes read) will always be at most m_BufferSize (the intermediary buffer’s size). In the case of ReadLargeBuffer(), m_InputSize exceeds 10MB while m_BufferSize is 64KB, leading to an underflow in the calculation of bufSize. This error results in the size passed to recv() being far larger than the destination buffer’s size. This malfunction allows a remote attacker to overflow the string or intermediary buffer.  

In the CVE-2023-33864 exploitation process, we exploit a heap-based buffer overflow vulnerability in the librenderdoc.so’s a multi-threaded TCP server. Initially, glibc’s malloc allocates a new 64MB of memory as a “heap” upon creating the server thread. We then establish seven sequential connections, each creating a new thread with allocated stack memory.   

These stacks are leaked, effectively filling the random gap between the heap and the libraries. Subsequently, we connect to the server and send a handshake packet with a 16MB client-name string, manipulating the server’s heap layout. This results in a buffer overflow error as we overwrite the intermediary buffer and adjacent memory chunk headers.   

When the server frees the intermediary buffer and the small chunk, it triggers the munmap_chunk() function, allowing us to munmap() an arbitrary memory block relative to the chunk. This creates a hole of precisely 8MB+4KB in the server’s heap. A new client thread is then created, and its stack is allocated in the spot we made. A long-lived connection established next sends a 14MB client-name string that overwrites the client name’s end with data from the client stack. Finally, a new connection with a 9MB string sent overwrites the client stack with data that we fully control, leading to a classic “stack smashing” attack. This process represents a clever manipulation of memory allocation, buffer overflow, and munmap() functionality to exploit the server’s vulnerabilities. This is achieved even in the face of modern glibc, malloc, ASLR, PIE, NX, and stack-canary protections.  

librenderdoc.so’s Buffer Overflow Dilemma: Dissecting CVE-2023-33863  

We delved into an anomaly in librenderdoc.so’s server, operating on TCP port 38920. A client attempting to send an exact 0xffffffff byte-long string (UINT32_MAX) triggers an unexpected series of events.  

The server changes this uint32_t length into a signed int, leading to an inadvertent sign-extension of this 0xffffffff int to a 0xffffffffffffffff size_t (SIZE_MAX) within the resize() function, given that its argument is a size_t (a 64-bit integer on amd64).  

This series of transformations culminates in the reserve() function allocating a buffer for the string, only to add 1 to the string size for a null-terminator, ultimately triggering an integer overflow. Consequently, this overflows the SIZE_MAX size of the string to 0 and allocates a minimum-sized buffer. This buffer is significantly smaller than required for the client’s long string, allowing the client to overflow this heap-based buffer with up to UINT32_MAX bytes. However, we did not attempt to exploit this vulnerability.  

In the current landscape of escalating cybersecurity threats, ensuring robust protection for your digital assets has become increasingly imperative When utilizing software such as RenderDoc, it is essential to put protective measures, such as firewalls, in place for specific TCP and UDP ports the program uses to enable its operations. For RenderDoc, the applicable ports that require safeguarding include TCP and UDP ports in the range of 38920-38927 and TCP and UDP ports number 39920. By adequately securing these ports, users can substantially reduce the risk of unauthorized access or malicious cyber activity. It’s vital to recognize that in today’s interconnected world, adopting proactive defense strategies is not a choice but an absolute necessity.  

Conclusion  

In the world of graphics software development and debugging, RenderDoc plays a pivotal role. However, like any complex software, vulnerabilities can emerge. The three vulnerabilities discussed today are a sobering reminder of the constant vigilance required in our digital world. Understanding these vulnerabilities is the first step towards fortifying our defenses, and we aim to shed light on the paths to remediation and resilience. As digital integration deepens, robust security practices become increasingly crucial. Merely comprehending and addressing known vulnerabilities is insufficient; we must proactively identify potential threats. That’s where an effective vulnerability management program comes in, aiding in identifying and resolving system vulnerabilities before exploitation. Embracing this proactive approach along with knowledge of existing vulnerabilities can strengthen organizational defenses and instill confidence in navigating the digital landscape.