New release: Tor 0.3.5.1-alpha

There's a new alpha release available for download. If you build Tor from source, you can download the source code for 0.3.5.1-alpha from the usual place on the website. Packages should be available over the coming weeks, with a new alpha Tor Browser release some time this week.

Remember, this is an alpha release: you should only run this if you'd like to find and report more bugs than usual.

Tor 0.3.5.1-alpha is the first release of the 0.3.5.x series. It adds client authorization for modern (v3) onion services, improves bootstrap reporting, begins reorganizing Tor's codebase, adds optional support for NSS in place of OpenSSL, and much more.

Changes in version 0.3.5.1-alpha - 2018-09-18

• Major features (onion services, UI change):
  • For a newly created onion service, the default version is now 3. Tor still supports existing version 2 services, but the operator now needs to set "HiddenServiceVersion 2" in order to create a new version 2 service. For existing services, Tor now learns the version by reading the key file. Closes ticket 27215.
• Major features (relay, UI change):
  • Relays no longer run as exits by default. If the "ExitRelay" option is auto (or unset), and no exit policy is specified with ExitPolicy or ReducedExitPolicy, we now treat ExitRelay as 0. Previously in this case, we allowed exit traffic and logged a warning message. Closes ticket 21530. Patch by Neel Chauhan.
  • Tor now validates that the ContactInfo config option is valid UTF- 8 when parsing torrc. Closes ticket 27428.

• Major features (bootstrap):
  • Don't report directory progress until after a connection to a relay or bridge has succeeded. Previously, we'd report 80% progress based on cached directory information when we couldn't even connect to the network. Closes ticket 27169.
• Major features (new code layout):

  • Nearly all of Tor's source code has been moved around into more logical places. The "common" directory is now divided into a set of libraries in "lib", and files in the "or" directory have been split into "core" (logic absolutely needed for onion routing), "feature" (independent modules in Tor), and "app" (to configure and invoke the rest of Tor). See doc/HACKING/CodeStructure.md for more information. Closes ticket 26481.

    This refactoring is not complete: although the libraries have been refactored to be acyclic, the main body of Tor is still too interconnected. We will attempt to improve this in the future.

• Major features (onion services v3):
  • Implement onion service client authorization at the descriptor level: only authorized clients can decrypt a service's descriptor to find out how to contact it. A new torrc option was added to control this client side: ClientOnionAuthDir <path>. On the service side, if the "authorized_clients/" directory exists in the onion service directory path, client configurations are read from the files within. See the manpage for more details. Closes ticket 27547. Patch done by Suphanat Chunhapanya (haxxpop).
  • Improve revision counter generation in next-gen onion services. Onion services can now scale by hosting multiple instances on different hosts without synchronization between them, which was previously impossible because descriptors would get rejected by HSDirs. Addresses ticket 25552.
• Major features (portability, cryptography, experimental, TLS):

  • Tor now has the option to compile with the NSS library instead of OpenSSL. This feature is experimental, and we expect that bugs may remain. It is mainly intended for environments where Tor's performance is not CPU-bound, and where NSS is already known to be installed. To try it out, configure Tor with the --enable-nss flag. Closes tickets 26631, 26815, and 26816.

    If you are experimenting with this option and using an old cached consensus, Tor may fail to start. To solve this, delete your "cached-consensus" and "cached-microdesc-consensus" files, (if present), and restart Tor.

• Major bugfixes (directory authority):
  • Actually check that the address we get from DirAuthority configuration line is valid IPv4. Explicitly disallow DirAuthority address to be a DNS hostname. Fixes bug 26488; bugfix on 0.1.2.10-rc.
• Major bugfixes (restart-in-process):
  • Fix a use-after-free error that could be caused by passing Tor an impossible set of options that would fail during options_act(). Fixes bug 27708; bugfix on 0.3.3.1-alpha.
• Minor features (admin tools):
  • Add a new --key-expiration option to print the expiration date of the signing cert in an ed25519_signing_cert file. Resolves issue 19506.
• Minor features (build):
  • If you pass the "--enable-pic" option to configure, Tor will try to tell the compiler to build position-independent code suitable to link into a dynamic library. (The default remains -fPIE, for code suitable for a relocatable executable.) Closes ticket 23846.
• Minor features (code correctness, testing):
  • Tor's build process now includes a "check-includes" make target to verify that no module of Tor relies on any headers from a higher- level module. We hope to use this feature over time to help refactor our codebase. Closes ticket 26447.
• Minor features (code layout):
  • We have a new "lowest-level" error-handling API for use by code invoked from within the logging module. With this interface, the logging code is no longer at risk of calling into itself if a failure occurs while it is trying to log something. Closes ticket 26427.
• Minor features (compilation):
  • Tor's configure script now supports a --with-malloc= option to select your malloc implementation. Supported options are "tcmalloc", "jemalloc", "openbsd" (deprecated), and "system" (the default). Addresses part of ticket 20424. Based on a patch from Alex Xu.
• Minor features (config):
  • The "auto" keyword in torrc is now case-insensitive. Closes ticket 26663.
• Minor features (continuous integration):
  • Don't do a distcheck with --disable-module-dirauth in Travis. Implements ticket 27252.
  • Install libcap-dev and libseccomp2-dev so these optional dependencies get tested on Travis CI. Closes ticket 26560.
  • Only run one online rust build in Travis, to reduce network errors. Skip offline rust builds on Travis for Linux gcc, because they're redundant. Implements ticket 27252.
  • Skip gcc on OSX in Travis CI, because it's rarely used. Skip a duplicate hardening-off build in Travis on Tor 0.2.9. Skip gcc on Linux with default settings, because all the non-default builds use gcc on Linux. Implements ticket 27252.
• Minor features (controller):
  • Emit CIRC_BW events as soon as we detect that we processed an invalid or otherwise dropped cell on a circuit. This allows vanguards and other controllers to react more quickly to dropped cells. Closes ticket 27678.
  • For purposes of CIRC_BW-based dropped cell detection, track half- closed stream ids, and allow their ENDs, SENDMEs, DATA and path bias check cells to arrive without counting it as dropped until either the END arrives, or the windows are empty. Closes ticket 25573.
  • Implement a 'GETINFO md/all' controller command to enable getting all known microdescriptors. Closes ticket 8323.
  • The GETINFO command now support an "uptime" argument, to return Tor's uptime in seconds. Closes ticket 25132.
• Minor features (denial-of-service avoidance):
  • Make our OOM handler aware of the DNS cache so that it doesn't fill up the memory. This check is important for our DoS mitigation subsystem. Closes ticket 18642. Patch by Neel Chauhan.
• Minor features (development):
  • Tor's makefile now supports running the "clippy" Rust style tool on our Rust code. Closes ticket 22156.
• Minor features (directory authority):
  • There is no longer an artificial upper limit on the length of bandwidth lines. Closes ticket 26223.
  • When a bandwidth file is used to obtain the bandwidth measurements, include this bandwidth file headers in the votes. Closes ticket 3723.
  • Improved support for networks with only a single authority or a single fallback directory. Patch from Gabriel Somlo. Closes ticket 25928.
• Minor features (embedding API):
  • The Tor controller API now supports a function to launch Tor with a preconstructed owning controller FD, so that embedding applications don't need to manage controller ports and authentication. Closes ticket 24204.
  • The Tor controller API now has a function that returns the name and version of the backend implementing the API. Closes ticket 26947.
• Minor features (geoip):
  • Update geoip and geoip6 to the September 6 2018 Maxmind GeoLite2 Country database. Closes ticket 27631.
• Minor features (memory management):
  • Get Libevent to use the same memory allocator as Tor, by calling event_set_mem_functions() during initialization. Resolves ticket 8415.
• Minor features (memory usage):
  • When not using them, store legacy TAP public onion keys in DER- encoded format, rather than as expanded public keys. This should save several megabytes on typical clients. Closes ticket 27246.
• Minor features (OpenSSL):
  • When possible, use RFC5869 HKDF implementation from OpenSSL rather than our own. Resolves ticket 19979.
• Minor features (Rust, code quality):
  • Improve rust code quality in the rust protover implementation by making it more idiomatic. Includes changing an internal API to take &str instead of &String. Closes ticket 26492.
• Minor features (testing):
  • Add scripts/test/chutney-git-bisect.sh, for bisecting using chutney. Implements ticket 27211.
• Minor features (tor-resolve):
  • The tor-resolve utility can now be used with IPv6 SOCKS proxies. Side-effect of the refactoring for ticket 26526.
• Minor features (UI):
  • Log each included configuration file or directory as we read it, to provide more visibility about where Tor is reading from. Patch from Unto Sten; closes ticket 27186.
  • Lower log level of "Scheduler type KIST has been enabled" to INFO. Closes ticket 26703.
• Minor bugfixes (bootstrap):
  • Try harder to get descriptors in non-exit test networks, by using the mid weight for the third hop when there are no exits. Fixes bug 27237; bugfix on 0.2.6.2-alpha.
• Minor bugfixes (C correctness):
  • Avoid casting smartlist index to int implicitly, as it may trigger a warning (-Wshorten-64-to-32). Fixes bug 26282; bugfix on 0.2.3.13-alpha, 0.2.7.1-alpha and 0.2.1.1-alpha.
  • Use time_t for all values in predicted_ports_prediction_time_remaining(). Rework the code that computes difference between durations/timestamps. Fixes bug 27165; bugfix on 0.3.1.1-alpha.
• Minor bugfixes (client, memory usage):
  • When not running as a directory cache, there is no need to store the text of the current consensus networkstatus in RAM. Previously, however, clients would store it anyway, at a cost of over 5 MB. Now, they do not. Fixes bug 27247; bugfix on 0.3.0.1-alpha.
• Minor bugfixes (client, reachableaddresses):
  • Instead of adding a "reject *:*" line to ReachableAddresses when loading the configuration, add one to the policy after parsing it in parse_reachable_addresses(). This prevents extra "reject *.*" lines from accumulating on reloads. Fixes bug 20874; bugfix on 0.1.1.5-alpha. Patch by Neel Chauhan.
• Minor bugfixes (code quality):
  • Rename sandbox_getaddrinfo() and other functions to no longer misleadingly suggest that they are sandbox-only. Fixes bug 26525; bugfix on 0.2.7.1-alpha.
• Minor bugfixes (configuration, Onion Services):
  • In rend_service_parse_port_config(), disallow any input to remain after address-port pair was parsed. This will catch address and port being whitespace-separated by mistake of the user. Fixes bug 27044; bugfix on 0.2.9.10.
• Minor bugfixes (continuous integration):
  • Stop reinstalling identical packages in our Windows CI. Fixes bug 27464; bugfix on 0.3.4.1-alpha.
• Minor bugfixes (controller):
  • Consider all routerinfo errors other than "not a server" to be transient for the purpose of "GETINFO exit-policy/*" controller request. Print stacktrace in the unlikely case of failing to recompute routerinfo digest. Fixes bug 27034; bugfix on 0.3.4.1-alpha.
• Minor bugfixes (directory connection shutdown):
  • Avoid a double-close when shutting down a stalled directory connection. Fixes bug 26896; bugfix on 0.3.4.1-alpha.
• Minor bugfixes (HTTP tunnel):
  • Fix a bug warning when closing an HTTP tunnel connection due to an HTTP request we couldn't handle. Fixes bug 26470; bugfix on 0.3.2.1-alpha.
• Minor bugfixes (ipv6):
  • In addrs_in_same_network_family(), we choose the subnet size based on the IP version (IPv4 or IPv6). Previously, we chose a fixed subnet size of /16 for both IPv4 and IPv6 addresses. Fixes bug 15518; bugfix on 0.2.3.1-alpha. Patch by Neel Chauhan.
• Minor bugfixes (logging):
  • As a precaution, do an early return from log_addr_has_changed() if Tor is running as client. Also, log a stack trace for debugging as this function should only be called when Tor runs as server. Fixes bug 26892; bugfix on 0.1.1.9-alpha.
  • Refrain from mentioning bug 21018 in the logs, as it is already fixed. Fixes bug 25477; bugfix on 0.2.9.8.
• Minor bugfixes (logging, documentation):
  • When SafeLogging is enabled, scrub IP address in channel_tls_process_netinfo_cell(). Also, add a note to manpage that scrubbing is not guaranteed on loglevels below Notice. Fixes bug 26882; bugfix on 0.2.4.10-alpha.
• Minor bugfixes (netflow padding):
  • Ensure circuitmux queues are empty before scheduling or sending padding. Fixes bug 25505; bugfix on 0.3.1.1-alpha.
• Minor bugfixes (onion service v2):
  • Log at level "info", not "warning", in the case that we do not have a consensus when a .onion request comes in. This can happen normally while bootstrapping. Fixes bug 27040; bugfix on 0.2.8.2-alpha.
• Minor bugfixes (onion service v3):
  • When the onion service directory can't be created or has the wrong permissions, do not log a stack trace. Fixes bug 27335; bugfix on 0.3.2.1-alpha.
• Minor bugfixes (OS compatibility):
  • Properly handle configuration changes that move a listener to/from wildcard IP address. If the first attempt to bind a socket fails, close the old listener and try binding the socket again. Fixes bug 17873; bugfix on 0.0.8pre-1.
• Minor bugfixes (performance)::
  • Rework node_is_a_configured_bridge() to no longer call node_get_all_orports(), which was performing too many memory allocations. Fixes bug 27224; bugfix on 0.2.3.9.
• Minor bugfixes (relay statistics):
  • Update relay descriptor on bandwidth changes only when the uptime is smaller than 24h, in order to reduce the efficiency of guard discovery attacks. Fixes bug 24104; bugfix on 0.1.1.6-alpha.
• Minor bugfixes (relays):
  • Consider the fact that we'll be making direct connections to our entry and guard nodes when computing the fraction of nodes that have their descriptors. Also, if we are using bridges and there is at least one bridge with a full descriptor, treat the fraction of guards available as 100%. Fixes bug 25886; bugfix on 0.2.4.10-alpha. Patch by Neel Chauhan.
  • Update the message logged on relays when DirCache is disabled. Since 0.3.3.5-rc, authorities require DirCache (V2Dir) for the Guard flag. Fixes bug 24312; bugfix on 0.3.3.5-rc.
• Minor bugfixes (rust, protover):
  • Compute protover votes correctly in the rust version of the protover code. Previously, the protover rewrite in 24031 allowed repeated votes from the same voter for the same protocol version to be counted multiple times in protover_compute_vote(). Fixes bug 27649; bugfix on 0.3.3.5-rc.
  • Reject protover names that contain invalid characters. Fixes bug 27687; bugfix on 0.3.3.1-alpha.
• Minor bugfixes (testing):
  • Fix two unit tests to work when HOME environment variable is not set. Fixes bug 27096; bugfix on 0.2.8.1-alpha.
  • If a unit test running in a subprocess exits abnormally or with a nonzero status code, treat the test as having failed, even if the test reported success. Without this fix, memory leaks don't cause the tests to fail, even with LeakSanitizer. Fixes bug 27658; bugfix on 0.2.2.4-alpha.
  • When logging a version mismatch in our openssl_version tests, report the actual offending version strings. Fixes bug 26152; bugfix on 0.2.9.1-alpha.
  • Fix forking tests on Windows when there is a space somewhere in the path. Fixes bug 26437; bugfix on 0.2.2.4-alpha.
• Code simplification and refactoring:
  • 'updateFallbackDirs.py' now ignores the blacklist file, as it's not longer needed. Closes ticket 26502.
  • Include paths to header files within Tor are now qualified by directory within the top-level src directory.
  • Many structures have been removed from the centralized "or.h" header, and moved into their own headers. This will allow us to reduce the number of places in the code that rely on each structure's contents and layout. Closes ticket 26383.
  • Remove ATTR_NONNULL macro from codebase. Resolves ticket 26527.
  • Remove GetAdaptersAddresses_fn_t. The code that used it was removed as part of the 26481 refactor. Closes ticket 27467.
  • Rework Tor SOCKS server code to use Trunnel and benefit from autogenerated functions for parsing and generating SOCKS wire format. New implementation is cleaner, more maintainable and should be less prone to heartbleed-style vulnerabilities. Implements a significant fraction of ticket 3569.
  • Split sampled_guards_update_from_consensus() and select_entry_guard_for_circuit() into subfunctions. In entry_guards_update_primary() unite three smartlist enumerations into one and move smartlist comparison code out of the function. Closes ticket 21349.
  • Tor now assumes that you have standards-conformant stdint.h and inttypes.h headers when compiling. Closes ticket 26626.
  • Unify our bloom filter logic. Previously we had two copies of this code: one for routerlist filtering, and one for address set calculations. Closes ticket 26510.
  • Use the simpler strcmpstart() helper in rend_parse_v2_service_descriptor instead of strncmp(). Closes ticket 27630.
  • Utility functions that can perform a DNS lookup are now wholly separated from those that can't, in separate headers and C modules. Closes ticket 26526.
• Documentation:
  • Copy paragraph and URL to Tor's code of conduct document from CONTRIBUTING to new CODE_OF_CONDUCT file. Resolves ticket 26638.
  • Remove old instructions from INSTALL document. Closes ticket 26588.
  • Warn users that they should not include MyFamily line(s) in their torrc when running Tor bridge. Closes ticket 26908.
• Removed features:
  • Tor no longer supports building with the dmalloc library. For debugging memory issues, we suggest using gperftools or msan instead. Closes ticket 26426.
  • Tor no longer attempts to run on Windows environments without the GetAdaptersAddresses() function. This function has existed since Windows XP, which is itself already older than we support.
  • Remove Tor2web functionality for version 2 onion services. The Tor2webMode and Tor2webRendezvousPoints options are now obsolete. (This feature was never shipped in vanilla Tor and it was only possible to use this feature by building the support at compile time. Tor2webMode is not implemented for version 3 onion services.) Closes ticket 26367.