So far in this series, I’ve developed a fuzzer for the µC/HTTP-server. As described in the previous post, this fuzzer reads from a file to enable compatibility with AFL++. That implementation only fuzzes a single request at a time. Although that single request fuzzer uncovered a few security vulnerabilities, there are complex and interesting object interactions and internal state transitions that occur when multiple requests are received in a single session.
I wanted to be able to fuzz those interactions and transitions by providing a single test case file containing multiple requests. So, this time, I’ll discuss why this approach is more challenging than simply substituting a socket file descriptor with a typical file descriptor, and I’ll describe the feature I added to the open-source library libdesock to support multiple requests.
It’s a bit confusing thinking about solving this problem because socket communication is two-way communication with a request-response pattern, while reading from a file occurs one way. In a networking client/server model, the client will typically make a request of the server, and the server will respond before the client sends another request.
The µC/HTTP-server works in a single thread so it completely processes and responds to the first request before ever reading from the socket descriptor for a second time.
Our goal is to simulate multiple requests from the client using a single test case file. But what happens if we include two requests within the same test case file? The server code will read the available data from the file until it reaches EOF or fills the buffer.
For instance, suppose a test case file contains two complete HTTP requests. The buffer size of the µC/HTTP-server is 1,460 bytes and the length of our two requests combined are only 841 bytes. This means the entire test case fits within the buffer and is read into memory at once. The µC/HTTP-server will process the first HTTP request and then discard the remaining data in the buffer. Then, when the server code reads from the test case file again, it will have reached EOF while only actually processing the first request.
To address this, we need a way to instruct the server to stop reading the file after processing the first request. This can be achieved by using a delimiter in the test case file. The delimiter should be a unique value that is unlikely to appear anywhere else in the request.
To implement this strategy of using a delimiter to indicate where one request ends and another begins, I added a feature to an existing library called libdesock. The purpose of this library is to de-socket a network application to enable fuzzing. This is because fuzzers typically provide their test inputs via stdin while network applications expect their inputs from network connections. Typically, a network server application will call the libc functions socket, listen, accept and then read/recv.
Libdesock operates by leveraging the Linux dynamic linker feature, LD_PRELOAD. When the LD_PRELOAD environment variable is set to the path of a shared object file, it instructs the dynamic linker to load that shared object before all others. As a result, when the dynamic linker searches for a symbol being called in the executable, it will first look in the shared object specified by the LD_PRELOAD environment variable. Normally, the symbols for the functions socket, listen, accept and read/recv are found within libc.so and executed from there. However, when LD_PRELOAD is set to libdesock.so, those symbols are found within the libdesock library and executed there instead. This allows libdesock to alter the behavior of those calls. By intercepting these calls, libdesock cleverly redirects recv/read to stdin rather than a network socket.
Multiple request feature
Originally, the libdesock code would read up to the size of the buffer used from stdin. This leads to the same issue as mentioned above when attempting to read multiple requests from an input file (stdin in this case):
“For instance, suppose a test case file contains two complete HTTP requests. The buffer size of the µC/HTTP-server is 1460 bytes and the length of our two requests combined are only 841 bytes. This means the entire test case fits within the buffer and is read into memory at once. The µC/HTTP-server will process the first HTTP request and then discard the remaining data in the buffer. Then, when the server code reads from the test case file again it will have reached EOF while only actually processing the first request.”
To address this, I added a feature to the libdesock read code to look for a request delimiter while reading. If the delimiter is found, it returns all the data read before encountering the delimiter. On the next call to read, it will begin reading after the delimiter and search for another delimiter. This approach works for fuzzing multiple requests in most networked applications because these applications typically run a processing loop where read/recv is called, the data is processed, and then read/recv is called again. This loop continues until the connection is closed. As mentioned in the first blog post, it was necessary to modify the executable for fuzzing so that it would exit when read returns 0. This is crucial for fuzzing, as it indicates the executable has finished processing the test case and has exited normally. If you’d like to see the code, you can check out libdesock here.
This version of the µC/HTTP-server fuzzer, which supports multiple requests per test case, uncovered two additional vulnerabilities. In both instances, a vulnerability in the processing of the first request is exploited by the second request.
When disclosing these vulnerabilities, I discovered that much of the µC/HTTP-server codebase is shared across multiple products. The other affected products are the Silicon Labs Gecko Platform and Weston Embedded Cesium NET. All the vulnerabilities discussed here have been reported to the manufacturers in accordance with our Coordinated Disclosure Policy. Each of these vulnerabilities in the affected products has been patched by the corresponding manufacturer.
The following Snort rules will detect exploitation attempts against these vulnerabilities: 119:203:1 and 119:282:1. Additional rules may be released in the future and current rules are subject to change, pending additional vulnerability information. For the most current rule information, please refer to your Cisco Secure Firewall or Snort.org.
Below is an overview of each vulnerability.
TALOS-2023-1726
When processing the protocol version portion of an HTTP request, an integer underflow occurs, which leads to an extremely large value within the connection object’s buffer length p_conn->RxBufLenRem. In the next iteration of the processing loop, that large value is used as an argument when calling receive. This means that the next request wouldn’t be limited by the max buffer size (1,460 bytes), and an attacker could overflow the receive buffer directly with the second request.
TALOS-2023-1732
In this vulnerability, a buffer overflow of one byte occurs when handling the header fields in an HTTP request. This one-byte overflow is significant because the µC/HTTP-server heap implementation stores a pointer to a free chunk of memory in the first four bytes of an allocation. This means that the one byte overwrite can modify the free chunk address which will be used as an allocation sometime in the future. Abusing this vulnerability required four requests. The first two were innocuous and caused specific heap allocations to occur (a technique known as “heap grooming”). The one-byte overwrite is triggered by the third request, while the fourth request leads to an improper allocation in the heap.
Both vulnerabilities involved modifications to global objects between requests. These types of interactions can be difficult to understand with static code analysis because it is not always obvious the order that functions will be called under specific circumstances. There are more of these types of complex vulnerabilities out there that could be revealed with multi-request fuzzing and I hope that using this file delimiter technique will lead to more of them being found and fixed.
In Part 3, I write a TAP device driver to fuzz the µCOS TCP/IP implementation.