Tor 0.3.2.1-alpha is released, with support for next-gen onion services and KIST scheduler

And as if all those other releases today were not enough, this is also the time for a new alpha release series!

Tor 0.3.2.1-alpha is the first release in the 0.3.2.x series. It includes support for our next-generation ("v3") onion service protocol, and adds a new circuit scheduler for more responsive forwarding decisions from relays. There are also numerous other small features and bugfixes here.

You can download the source from the usual place on the website. Binary packages should be available soon, with an alpha Tor Browser likely by the end of the month.

Remember: This is an alpha release, and it's likely to have more bugs than usual. We hope that people will try it out to find and report bugs, though.

Below are the changes since Tor 0.3.1.7.

Changes in version 0.3.2.1-alpha - 2017-09-18

• Major feature (scheduler, channel):

  • Tor now uses new schedulers to decide which circuits should deliver cells first, in order to improve congestion at relays. The first type is called "KIST" ("Kernel Informed Socket Transport"), and is only available on Linux-like systems: it uses feedback from the kernel to prevent the kernel's TCP buffers from growing too full. The second new scheduler type is called "KISTLite": it behaves the same as KIST, but runs on systems without kernel support for inspecting TCP implementation details. The old scheduler is still available, under the name "Vanilla". To change the default scheduler preference order, use the new "Schedulers" option. (The default preference order is "KIST,KISTLite,Vanilla".)

    Matt Traudt implemented KIST, based on research by Rob Jansen, John Geddes, Christ Wacek, Micah Sherr, and Paul Syverson. For more information, see the design paper at http://www.robgjansen.com/publications/kist-sec2014.pdf and the followup implementation paper at https://arxiv.org/abs/1709.01044. Closes ticket 12541.

• Major features (next-generation onion services):

  • Tor now supports the next-generation onion services protocol for clients and services! As part of this release, the core of proposal 224 has been implemented and is available for experimentation and testing by our users. This newer version of onion services ("v3") features many improvements over the legacy system, including:

    a) Better crypto (replaced SHA1/DH/RSA1024 with SHA3/ed25519/curve25519)

    b) Improved directory protocol, leaking much less information to directory servers.

    c) Improved directory protocol, with smaller surface for targeted attacks.

    d) Better onion address security against impersonation.

    e) More extensible introduction/rendezvous protocol.

    f) A cleaner and more modular codebase.

    You can identify a next-generation onion address by its length: they are 56 characters long, as in "4acth47i6kxnvkewtm6q7ib2s3ufpo5sqbsnzjpbi7utijcltosqemad.onion".

    In the future, we will release more options and features for v3 onion services, but we first need a testing period, so that the current codebase matures and becomes more robust. Planned features include: offline keys, advanced client authorization, improved guard algorithms, and statistics. For full details, see proposal 224.

    Legacy ("v2") onion services will still work for the foreseeable future, and will remain the default until this new codebase gets tested and hardened. Service operators who want to experiment with the new system can use the 'HiddenServiceVersion 3' torrc directive along with the regular onion service configuration options. We will publish a blog post about this new feature soon! Enjoy!

• Major bugfixes (usability, control port):
  • Report trusted clock skew indications as bootstrap errors, so controllers can more easily alert users when their clocks are wrong. Fixes bug 23506; bugfix on 0.1.2.6-alpha.
• Minor features (bug detection):
  • Log a warning message with a stack trace for any attempt to call get_options() during option validation. This pattern has caused subtle bugs in the past. Closes ticket 22281.
• Minor features (client):
  • You can now use Tor as a tunneled HTTP proxy: use the new HTTPTunnelPort option to open a port that accepts HTTP CONNECT requests. Closes ticket 22407.
  • Add an extra check to make sure that we always use the newer guard selection code for picking our guards. Closes ticket 22779.
  • When downloading (micro)descriptors, don't split the list into multiple requests unless we want at least 32 descriptors. Previously, we split at 4, not 32, which led to significant overhead in HTTP request size and degradation in compression performance. Closes ticket 23220.
• Minor features (command line):
  • Add a new commandline option, --key-expiration, which prints when the current signing key is going to expire. Implements ticket 17639; patch by Isis Lovecruft.
• Minor features (control port):
  • If an application tries to use the control port as an HTTP proxy, respond with a meaningful "This is the Tor control port" message, and log the event. Closes ticket 1667. Patch from Ravi Chandra Padmala.
  • Provide better error message for GETINFO desc/(id|name) when not fetching router descriptors. Closes ticket 5847. Patch by Kevin Butler.
  • Add GETINFO "{desc,md}/download-enabled", to inform the controller whether Tor will try to download router descriptors and microdescriptors respectively. Closes ticket 22684.
  • Added new GETINFO targets "ip-to-country/{ipv4,ipv6}-available", so controllers can tell whether the geoip databases are loaded. Closes ticket 23237.
  • Adds a timestamp field to the CIRC_BW and STREAM_BW bandwidth events. Closes ticket 19254. Patch by "DonnchaC".
• Minor features (development support):
  • Developers can now generate a call-graph for Tor using the "calltool" python program, which post-processes object dumps. It should work okay on many Linux and OSX platforms, and might work elsewhere too. To run it, install calltool from https://gitweb.torproject.org/user/nickm/calltool.git and run "make callgraph". Closes ticket 19307.
• Minor features (ed25519):
  • Add validation function to checks for torsion components in ed25519 public keys, used by prop224 client-side code. Closes ticket 22006. Math help by Ian Goldberg.
• Minor features (exit relay, DNS):
  • Improve the clarity and safety of the log message from evdns when receiving an apparently spoofed DNS reply. Closes ticket 3056.
• Minor features (integration, hardening):
  • Add a new NoExec option to prevent Tor from running other programs. When this option is set to 1, Tor will never try to run another program, regardless of the settings of PortForwardingHelper, ClientTransportPlugin, or ServerTransportPlugin. Once NoExec is set, it cannot be disabled without restarting Tor. Closes ticket 22976.
• Minor features (logging):
  • Improve the warning message for specifying a relay by nickname. The previous message implied that nickname registration was still part of the Tor network design, which it isn't. Closes ticket 20488.
  • If the sandbox filter fails to load, suggest to the user that their kernel might not support seccomp2. Closes ticket 23090.
• Minor features (portability):
  • Check at configure time whether uint8_t is the same type as unsigned char. Lots of existing code already makes this assumption, and there could be strict aliasing issues if the assumption is violated. Closes ticket 22410.
• Minor features (relay, configuration):
  • Reject attempts to use relative file paths when RunAsDaemon is set. Previously, Tor would accept these, but the directory- changing step of RunAsDaemon would give strange and/or confusing results. Closes ticket 22731.
• Minor features (startup, safety):
  • When configured to write a PID file, Tor now exits if it is unable to do so. Previously, it would warn and continue. Closes ticket 20119.
• Minor features (static analysis):
  • The BUG() macro has been changed slightly so that Coverity no longer complains about dead code if the bug is impossible. Closes ticket 23054.
• Minor features (testing):
  • The default chutney network tests now include tests for the v3 hidden service design. Make sure you have the latest version of chutney if you want to run these. Closes ticket 22437.
  • Add a unit test to verify that we can parse a hardcoded v2 hidden service descriptor. Closes ticket 15554.
• Minor bugfixes (certificate handling):
  • Fix a time handling bug in Tor certificates set to expire after the year 2106. Fixes bug 23055; bugfix on 0.3.0.1-alpha. Found by Coverity as CID 1415728.
• Minor bugfixes (client, usability):
  • Refrain from needlessly rejecting SOCKS5-with-hostnames and SOCKS4a requests that contain IP address strings, even when SafeSocks in enabled, as this prevents user from connecting to known IP addresses without relying on DNS for resolving. SafeSocks still rejects SOCKS connections that connect to IP addresses when those addresses are _not_ encoded as hostnames. Fixes bug 22461; bugfix on Tor 0.2.6.2-alpha.
• Minor bugfixes (code correctness):
  • Call htons() in extend_cell_format() for encoding a 16-bit value. Previously we used ntohs(), which happens to behave the same on all the platforms we support, but which isn't really correct. Fixes bug 23106; bugfix on 0.2.4.8-alpha.
  • For defense-in-depth, make the controller's write_escaped_data() function robust to extremely long inputs. Fixes bug 19281; bugfix on 0.1.1.1-alpha. Reported by Guido Vranken.
• Minor bugfixes (compilation):
  • Fix unused-variable warnings in donna's Curve25519 SSE2 code. Fixes bug 22895; bugfix on 0.2.7.2-alpha.
• Minor bugfixes (consensus expiry):
  • Check for adequate directory information correctly. Previously, Tor would reconsider whether it had sufficient directory information every 2 minutes. Fixes bug 23091; bugfix on 0.2.0.19-alpha.
• Minor bugfixes (directory protocol):
  • Directory servers now include a "Date:" http header for response codes other than 200. Clients starting with a skewed clock and a recent consensus were getting "304 Not modified" responses from directory authorities, so without the Date header, the client would never hear about a wrong clock. Fixes bug 23499; bugfix on 0.0.8rc1.
  • Make clients wait for 6 seconds before trying to download a consensus from an authority. Fixes bug 17750; bugfix on 0.2.8.1-alpha.
• Minor bugfixes (DoS-resistance):
  • If future code asks if there are any running bridges, without checking if bridges are enabled, log a BUG warning rather than crashing. Fixes bug 23524; bugfix on 0.3.0.1-alpha.
• Minor bugfixes (format strictness):
  • Restrict several data formats to decimal. Previously, the BuildTimeHistogram entries in the state file, the "bw=" entries in the bandwidth authority file, and the process IDs passed to the __OwningControllerProcess option could all be specified in hex or octal as well as in decimal. This was not an intentional feature. Fixes bug 22802; bugfixes on 0.2.2.1-alpha, 0.2.2.2-alpha, and 0.2.2.28-beta.
• Minor bugfixes (heartbeat):
  • If we fail to write a heartbeat message, schedule a retry for the minimum heartbeat interval number of seconds in the future. Fixes bug 19476; bugfix on 0.2.3.1-alpha.
• Minor bugfixes (linux seccomp2 sandbox, logging):
  • Fix some messages on unexpected errors from the seccomp2 library. Fixes bug 22750; bugfix on 0.2.5.1-alpha. Patch from "cypherpunks".
• Minor bugfixes (logging):
  • Remove duplicate log messages regarding opening non-local SocksPorts upon parsing config and opening listeners at startup. Fixes bug 4019; bugfix on 0.2.3.3-alpha.
  • Use a more comprehensible log message when telling the user they've excluded every running exit node. Fixes bug 7890; bugfix on 0.2.2.25-alpha.
  • When logging the number of descriptors we intend to download per directory request, do not log a number higher than then the number of descriptors we're fetching in total. Fixes bug 19648; bugfix on 0.1.1.8-alpha.
  • When warning about a directory owned by the wrong user, log the actual name of the user owning the directory. Previously, we'd log the name of the process owner twice. Fixes bug 23487; bugfix on 0.2.9.1-alpha.
  • The tor specification says hop counts are 1-based, so fix two log messages that mistakenly logged 0-based hop counts. Fixes bug 18982; bugfix on 0.2.6.2-alpha and 0.2.4.5-alpha. Patch by teor. Credit to Xiaofan Li for reporting this issue.
• Minor bugfixes (portability):
  • Stop using the PATH_MAX variable, which is not defined on GNU Hurd. Fixes bug 23098; bugfix on 0.3.1.1-alpha.
• Minor bugfixes (relay):
  • When uploading our descriptor for the first time after startup, report the reason for uploading as "Tor just started" rather than leaving it blank. Fixes bug 22885; bugfix on 0.2.3.4-alpha.
  • Avoid unnecessary calls to directory_fetches_from_authorities() on relays, to prevent spurious address resolutions and descriptor rebuilds. This is a mitigation for bug 21789. Fixes bug 23470; bugfix on in 0.2.8.1-alpha.
• Minor bugfixes (tests):
  • Fix a broken unit test for the OutboundAddress option: the parsing function was never returning an error on failure. Fixes bug 23366; bugfix on 0.3.0.3-alpha.
  • Fix a signed-integer overflow in the unit tests for dir/download_status_random_backoff, which was untriggered until we fixed bug 17750. Fixes bug 22924; bugfix on 0.2.9.1-alpha.
• Minor bugfixes (usability, control port):
  • Stop making an unnecessary routerlist check in NETINFO clock skew detection; this was preventing clients from reporting NETINFO clock skew to controllers. Fixes bug 23532; bugfix on 0.2.4.4-alpha.
• Code simplification and refactoring:
  • Extract the code for handling newly-open channels into a separate function from the general code to handle channel state transitions. This change simplifies our callgraph, reducing the size of the largest strongly connected component by roughly a factor of two. Closes ticket 22608.
  • Remove dead code for largely unused statistics on the number of times we've attempted various public key operations. Fixes bug 19871; bugfix on 0.1.2.4-alpha. Fix by Isis Lovecruft.
  • Remove several now-obsolete functions for asking about old variants directory authority status. Closes ticket 22311; patch from "huyvq".
  • Remove some of the code that once supported "Named" and "Unnamed" routers. Authorities no longer vote for these flags. Closes ticket 22215.
  • Rename the obsolete malleable hybrid_encrypt functions used in TAP and old hidden services, to indicate that they aren't suitable for new protocols or formats. Closes ticket 23026.
  • Replace our STRUCT_OFFSET() macro with offsetof(). Closes ticket 22521. Patch from Neel Chauhan.
  • Split the enormous circuit_send_next_onion_skin() function into multiple subfunctions. Closes ticket 22804.
  • Split the portions of the buffer.c module that handle particular protocols into separate modules. Part of ticket 23149.
  • Use our test macros more consistently, to produce more useful error messages when our unit tests fail. Add coccinelle patches to allow us to re-check for test macro uses. Closes ticket 22497.
• Deprecated features:
  • Deprecate HTTPProxy/HTTPProxyAuthenticator config options. They only applies to direct unencrypted HTTP connections to your directory server, which your Tor probably isn't using. Closes ticket 20575.
• Documentation:
  • Clarify in the manual that "Sandbox 1" is only supported on Linux kernels. Closes ticket 22677.
  • Document all values of PublishServerDescriptor in the manpage. Closes ticket 15645.
  • Improve the documentation for the directory port part of the DirAuthority line. Closes ticket 20152.
  • Restore documentation for the authorities' "approved-routers" file. Closes ticket 21148.
• Removed features:
  • The AllowDotExit option has been removed as unsafe. It has been deprecated since 0.2.9.2-alpha. Closes ticket 23426.
  • The ClientDNSRejectInternalAddresses flag can no longer be set on non-testing networks. It has been deprecated since 0.2.9.2-alpha. Closes ticket 21031.
  • The controller API no longer includes an AUTHDIR_NEWDESCS event: nobody was using it any longer. Closes ticket 22377.