V3 onion services usage

Guest post by Tobias Höller

With the deprecation of V2 onion services right around the corner, it is a good time to talk about V3 onion services. This post will discuss the most important privacy improvements provided by V3 onion services as well as their limitations. Aware of those limitations, our research group at the Institute of Network and Security at JKU Linz conducted an experiment that extracts information about how V3 onion services are being used from the Tor network.

Privacy improvements in V3

The most obvious difference between V2 and V3 onion services is the different address format. V3 onion addresses have 56 characters instead of 16 (because they contain a full ed25519 public key, not just the hash of a public key), meaning that migrating from V2 to V3 requires all users to learn/remember/save a new onion address address.

The main reason behind this change is tied to a key component of all onion services, the hidden service directory. Its purpose is to provide the information needed to connect to a specified onion address (just like the DNS system does for regular domain names). To protect the privacy of onion service users, the hidden service directory is a distributed hash table formed by all Tor relays which possess the HSDir flag. For V2 onion services, the data published in the hidden service directory is uploaded in plain text, meaning that the Tor relays with the HSDir flag can learn a lot of information about a small fraction of running V2 onion services (most importantly the onion address) every day.

Note that collecting and probing V2 onion addresses via HSDir relays is considered malicious behavior and sanctioned by Tor’s bad-relays team.

V3 uses encryption and key derivation to address this issue. Since the V3 address is itself a public key, all the data uploaded to the hidden service directory can be encrypted. Clients can always decrypt that data with the key embedded in the .onion address. However, clients still need to ask the directory for information about a specific onion address, which would again allow mass collection of onion addresses. With V3 onion services, this is prevented by using key derivation to derive a daily-rotated identifier ("blinded public key").

So instead of asking the hidden service directory for facebookwkhpilnemxj7asaniu7vnjjbiltxjqhye3mhbshg7kx5tfyd.onion, the Tor client automatically calculates the current identifier from the onion address and the current date (e.g. 2021-08-19), and asks for that blinded public key (here: ek4gJEtlHmwwadLvMNG7tYx/lJuJN1zQl6pMVkGmAcM).

Collectable information

Thanks to these improvements, V3 onion services leak much less sensitive information. However, these changes do not completely stop the hidden service directory from revealing interesting metadata.

First, it is possible to estimate the number of running onion services by counting the number of uploaded blinded public keys because every onion address translates to exactly one blinded public key per day. This means that it is possible to count onion addresses by counting blinded public keys which is already used by the Tor Project to collect statistical information about the number V3 onion services since Tor version 0.4.6.1-alpha (just like they have been doing for V2 for a long time now). This is expected to increase as soon as a sufficient number of Tor relays start to report data allowing for reliable V3 statistics (hint: remember to keep your Tor relays up-to-date).

By collecting detailed logs of uploads and downloads of blinded public keys, HSDir relays can track many more statistical metrics about V3 onion services and their usage. For example, one could track how many uploaded blinded public keys are never requested or how many client requests for blinded public keys could not be answered.

To get a feeling for how interesting this data might be, we designed an experiment where we deployed 50 relays with the HSDir flag and modified them to log every upload and download of blinded public keys. We tried our best to set this up in a privacy-friendly way that does not endanger individual Tor users by reducing the precision of timestamps and randomly sorting our database to destroy order. In addition, we actively asked for feedback on our experiment design from the Tor research safety board (which we integrated in our setup) and even created a plan on what to do if law enforcement would show up with a warrant (which thankfully did not happen so far).

Some results

After walking you through all the theory it’s time to share some of the data we collected since deploying our relays on March 1st, 2021. During this period we registered more than 131 million uploads for ~4.5 million different blinded public keys and more than 225 million download attempts for ~3.4 million different blinded public keys. A nice example for utilizing information about downloads is to track how many blinded public keys uploaded to our servers were actually downloaded.

In the same fashion we can also check how many download attempts we received that could not be answered because they requested keys that had never been uploaded to us.

Our Venn diagram shows that a vast majority of uploaded blinded public keys are never downloaded, indicating that these onion services are barely used.

In case you are wondering how an onion service can be used if we do not see any downloads for it, that is due to the redundant nature of the hidden service directory. There is always more than just one relay to download from (by default 6 HSDir relays are responsible for a blinded public key). So even if we see zero downloads, other relays might have seen some, making it impossible to distinguish unused and barely used onion services in our data.

We also see that the majority of download requests asked for blinded public keys that had never been uploaded to us. Again, this does not mean that clients see this failure (because Tor tries all 6 responsible HSDir nodes before giving up) but it indicates that lots of requests for .onion addresses ask for services that are no longer operational. A particularly interesting finding was that a few blinded public keys receive a huge amount of download requests without ever being uploaded to our relays.

In our data we identified 14 blinded public keys with more than 2 million download attempts despite not having any upload attempt. This indicates that a few onion services (or even only one?) are frequently requested although they are not operational. We believe this pattern to be the product of either DDOS attacks against a specific onion service or members of a defunct botnet trying to reach their C&C server.

Let's wrap this post up with an estimate on the number of V3 onion services that have already been deployed in the Tor network:

Note that our experiment collects data from 50 Tor relays which accounts for only about 1% of the hidden service directory. Therefore, our results give a first good estimate on the total number of V3 onion services but are certainly not as reliable as you would expect for statistics at Tor metrics. Nevertheless, it is certain that the number of deployed V3 onion services has been on the rise throughout 2021. So if you still operate only a V2 onion service, now would be a great time to get on board and upgrade to V3.

Finally, if you are interested in more details on our experiment setup and results, read the full paper: On the state of V3 onion services (by T. Hoeller, M. Roland, R. Mayrhofer).

And if you happen to operate a V3 onion service that suffered a DDOS attack in 2021 or know a botnet that uses a V3 onion service and are willing to share that information with me, please get in touch with me at tobias[dot]hoeller[at]ins[dot]jku[dot]at.