Tor 0.3.1.2-alpha is out (with notes about 0.3.1.1-alpha)

Hello again! This post announces the second alpha in the 0.3.1.x series, which I just released today. And since the blog was down when the first alpha came out, I'm posting the changelog for 0.3.1.1-alpha below too.

Tor 0.3.1.2-alpha is the second release in the 0.3.1.x series. It fixes a few bugs found while testing 0.3.1.1-alpha, including a memory corruption bug that affected relay stability.

Since this is an alpha release, you can expect more bugs than usual.

If you build Tor from source, you can find Tor 0.3.1.2-alpha at the usual place at the Download page on our website. Otherwise, you'll probably want to wait until packages are available. The next Tor Browser alpha release with this version of Tor will likely come out in mid-June.

Changes in version 0.3.1.2-alpha - 2017-05-26

• Major bugfixes (crash, relay):
  • Fix a memory-corruption bug in relays that set MyFamily. Previously, they would double-free MyFamily elements when making the next descriptor or when changing their configuration. Fixes bug 22368; bugfix on 0.3.1.1-alpha.
• Minor bugfixes (logging):
  • Log a better message when a directory authority replies to an upload with an unexpected status code. Fixes bug 11121; bugfix on 0.1.0.1-rc.
• Minor bugfixes (memory leak, directory authority):
  • When directory authorities reject a router descriptor due to keypinning, free the router descriptor rather than leaking the memory. Fixes bug 22370; bugfix on 0.2.7.2-alpha.

Changes in version 0.3.1.1-alpha - 2017-05-22

Tor 0.3.1.1-alpha is the first release in the 0.3.1.x series. It reduces the bandwidth usage for Tor's directory protocol, adds some basic padding to resist netflow-based traffic analysis and to serve as the basis of other padding in the future, and adds rust support to the build system.

It also contains numerous other small features and improvements to security, correctness, and performance.

Below are the changes since 0.3.0.7.

• Major features (directory protocol):
  • Tor relays and authorities can now serve clients an abbreviated version of the consensus document, containing only the changes since an older consensus document that the client holds. Clients now request these documents when available. When both client and server use this new protocol, they will use far less bandwidth (up to 94% less) to keep the client's consensus up-to-date. Implements proposal 140; closes ticket 13339. Based on work by Daniel Martí.
  • Tor can now compress directory traffic with lzma or with zstd compression algorithms, which can deliver better bandwidth performance. Because lzma is computationally expensive, it's only used for documents that can be compressed once and served many times. Support for these algorithms requires that tor is built with the libzstd and/or liblzma libraries available. Implements proposal 278; closes ticket 21662.
  • Relays now perform the more expensive compression operations, and consensus diff generation, in worker threads. This separation avoids delaying the main thread when a new consensus arrives.
• Major features (experimental):
  • Tor can now build modules written in Rust. To turn this on, pass the "--enable-rust" flag to the configure script. It's not time to get excited yet: currently, there is no actual Rust functionality beyond some simple glue code, and a notice at startup to tell you that Rust is running. Still, we hope that programmers and packagers will try building Tor with Rust support, so that we can find issues and solve portability problems. Closes ticket 22106.

 

• Major features (traffic analysis resistance):

  • Connections between clients and relays now send a padding cell in each direction every 1.5 to 9.5 seconds (tunable via consensus parameters). This padding will not resist specialized eavesdroppers, but it should be enough to make many ISPs' routine network flow logging less useful in traffic analysis against Tor users.

    Padding is negotiated using Tor's link protocol, so both relays and clients must upgrade for this to take effect. Clients may still send padding despite the relay's version by setting ConnectionPadding 1 in torrc, and may disable padding by setting ConnectionPadding 0 in torrc. Padding may be minimized for mobile users with the torrc option ReducedConnectionPadding. Implements Proposal 251 and Section 2 of Proposal 254; closes ticket 16861.

  • Relays will publish 24 hour totals of padding and non-padding cell counts to their extra-info descriptors, unless PaddingStatistics 0 is set in torrc. These 24 hour totals are also rounded to multiples of 10000.
• Major bugfixes (connection usage):
  • We use NETINFO cells to try to determine if both relays involved in a connection will agree on the canonical status of that connection. We prefer the connections where this is the case for extend cells, and try to close connections where relays disagree on their canonical status early. Also, we now prefer the oldest valid connection for extend cells. These two changes should reduce the number of long-term connections that are kept open between relays. Fixes bug 17604; bugfix on 0.2.5.5-alpha.
  • Relays now log hourly statistics (look for "channel_check_for_duplicates" lines) on the total number of connections to other relays. If the number of connections per relay is unexpectedly large, this log message is at notice level. Otherwise it is at info.
• Major bugfixes (entry guards):
  • Don't block bootstrapping when a primary bridge is offline and we can't get its descriptor. Fixes bug 22325; fixes one case of bug 21969; bugfix on 0.3.0.3-alpha.
• Major bugfixes (linux TPROXY support):
  • Fix a typo that had prevented TPROXY-based transparent proxying from working under Linux. Fixes bug 18100; bugfix on 0.2.6.3-alpha. Patch from "d4fq0fQAgoJ".
• Minor features (security, windows):
  • Enable a couple of pieces of Windows hardening: one (HeapEnableTerminationOnCorruption) that has been on-by-default since Windows 8, and unavailable before Windows 7; and one (PROCESS_DEP_DISABLE_ATL_THUNK_EMULATION) which we believe doesn't affect us, but shouldn't do any harm. Closes ticket 21953.
• Minor features (config options):
  • Allow "%include" directives in torrc configuration files. These directives import the settings from other files, or from all the files in a directory. Closes ticket 1922. Code by Daniel Pinto.
  • Make SAVECONF return an error when overwriting a torrc that has includes. Using SAVECONF with the FORCE option will allow it to overwrite torrc even if includes are used. Related to ticket 1922.
  • Add "GETINFO config-can-saveconf" to tell controllers if SAVECONF will work without the FORCE option. Related to ticket 1922.
• Minor features (controller):
  • Warn the first time that a controller requests data in the long- deprecated 'GETINFO network-status' format. Closes ticket 21703.
• Minor features (defaults):
  • The default value for UseCreateFast is now 0: clients which haven't yet received a consensus document will now use a proper ntor handshake to talk to their directory servers whenever they can. Closes ticket 21407.
  • Onion key rotation and expiry intervals are now defined as a network consensus parameter, per proposal 274. The default lifetime of an onion key is increased from 7 to 28 days. Old onion keys will expire after 7 days by default. This change will make consensus diffs much smaller, and save significant bandwidth. Closes ticket 21641.
• Minor features (fallback directory list):
  • Update the fallback directory mirror whitelist and blacklist based on operator emails. Closes task 21121.
  • Replace the 177 fallbacks originally introduced in Tor 0.2.9.8 in December 2016 (of which ~126 were still functional) with a list of 151 fallbacks (32 new, 119 unchanged, 58 removed) generated in May 2017. Resolves ticket 21564.
• Minor features (hidden services, logging):
  • Log a message when a hidden service descriptor has fewer introduction points than specified in HiddenServiceNumIntroductionPoints. Closes tickets 21598.
  • Log a message when a hidden service reaches its introduction point circuit limit, and when that limit is reset. Follow up to ticket 21594; closes ticket 21622.
  • Warn user if multiple entries in EntryNodes and at least one HiddenService are used together. Pinning EntryNodes along with a hidden service can be possibly harmful; for instance see ticket 14917 or 21155. Closes ticket 21155.
• Minor features (linux seccomp2 sandbox):
  • We now have a document storage backend compatible with the Linux seccomp2 sandbox. This backend is used for consensus documents and diffs between them; in the long term, we'd like to use it for unparseable directory material too. Closes ticket 21645
  • Increase the maximum allowed size passed to mprotect(PROT_WRITE) from 1MB to 16MB. This was necessary with the glibc allocator in order to allow worker threads to allocate more memory -- which in turn is necessary because of our new use of worker threads for compression. Closes ticket 22096.
• Minor features (logging):
  • Log files are no longer created world-readable by default. (Previously, most distributors would store the logs in a non- world-readable location to prevent inappropriate access. This change is an extra precaution.) Closes ticket 21729; patch from toralf.
• Minor features (performance):
  • Our Keccak (SHA-3) implementation now accesses memory more efficiently, especially on little-endian systems. Closes ticket 21737.
  • Add an O(1) implementation of channel_find_by_global_id(), to speed some controller functions.
• Minor features (relay, configuration):
  • The MyFamily option may now be repeated as many times as desired, for relays that want to configure large families. Closes ticket 4998; patch by Daniel Pinto.
• Minor features (safety):
  • Add an explicit check to extrainfo_parse_entry_from_string() for NULL inputs. We don't believe this can actually happen, but it may help silence a warning from the Clang analyzer. Closes ticket 21496.
• Minor features (testing):
  • Add a "--disable-memory-sentinels" feature to help with fuzzing. When Tor is compiled with this option, we disable a number of redundant memory-safety failsafes that are intended to stop bugs from becoming security issues. This makes it easier to hunt for bugs that would be security issues without the failsafes turned on. Closes ticket 21439.
  • Add a general event-tracing instrumentation support to Tor. This subsystem will enable developers and researchers to add fine- grained instrumentation to their Tor instances, for use when examining Tor network performance issues. There are no trace events yet, and event-tracing is off by default unless enabled at compile time. Implements ticket 13802.
  • Improve our version parsing tests: add tests for typical version components, add tests for invalid versions, including numeric range and non-numeric prefixes. Unit tests 21278, 21450, and 21507. Partially implements 21470.
• Minor bugfixes (bandwidth accounting):
  • Roll over monthly accounting at the configured hour and minute, rather than always at 00:00. Fixes bug 22245; bugfix on 0.0.9rc1. Found by Andrey Karpov with PVS-Studio.
• Minor bugfixes (code correctness):
  • Accurately identify client connections by their lack of peer authentication. This means that we bail out earlier if asked to extend to a client. Follow-up to 21407. Fixes bug 21406; bugfix on 0.2.4.23.
• Minor bugfixes (configuration):
  • Do not crash when starting with LearnCircuitBuildTimeout 0. Fixes bug 22252; bugfix on 0.2.9.3-alpha.
• Minor bugfixes (connection lifespan):
  • Allow more control over how long TLS connections are kept open: unify CircuitIdleTimeout and PredictedPortsRelevanceTime into a single option called CircuitsAvailableTimeout. Also, allow the consensus to control the default values for both this preference and the lifespan of relay-to-relay connections. Fixes bug 17592; bugfix on 0.2.5.5-alpha.
  • Increase the initial circuit build timeout testing frequency, to help ensure that ReducedConnectionPadding clients finish learning a timeout before their orconn would expire. The initial testing rate was set back in the days of TAP and before the Tor Browser updater, when we had to be much more careful about new clients making lots of circuits. With this change, a circuit build timeout is learned in about 15-20 minutes, instead of 100-120 minutes.
• Minor bugfixes (controller):
  • GETINFO onions/current and onions/detached no longer respond with 551 on empty lists. Fixes bug 21329; bugfix on 0.2.7.1-alpha.
  • Trigger HS descriptor events on the control port when the client fails to pick a hidden service directory for a hidden service. This can happen if all the hidden service directories are in ExcludeNodes, or they have all been queried within the last 15 minutes. Fixes bug 22042; bugfix on 0.2.5.2-alpha.
• Minor bugfixes (directory authority):
  • When rejecting a router descriptor for running an obsolete version of Tor without ntor support, warn about the obsolete tor version, not the missing ntor key. Fixes bug 20270; bugfix on 0.2.9.3-alpha.
  • Prevent the shared randomness subsystem from asserting when initialized by a bridge authority with an incomplete configuration file. Fixes bug 21586; bugfix on 0.2.9.8.
• Minor bugfixes (exit-side DNS):
  • Fix an untriggerable assertion that checked the output of a libevent DNS error, so that the assertion actually behaves as expected. Fixes bug 22244; bugfix on 0.2.0.20-rc. Found by Andrey Karpov using PVS-Studio.
• Minor bugfixes (fallback directories):
  • Make the usage example in updateFallbackDirs.py actually work, and explain what it does. Fixes bug 22270; bugfix on 0.3.0.3-alpha.
  • Decrease the guard flag average required to be a fallback. This allows us to keep relays that have their guard flag removed when they restart. Fixes bug 20913; bugfix on 0.2.8.1-alpha.
  • Decrease the minimum number of fallbacks to 100. Fixes bug 20913; bugfix on 0.2.8.1-alpha.
  • Make sure fallback directory mirrors have the same address, port, and relay identity key for at least 30 days before they are selected. Fixes bug 20913; bugfix on 0.2.8.1-alpha.
• Minor bugfixes (hidden services):
  • Stop printing a cryptic warning when a hidden service gets a request to connect to a virtual port that it hasn't configured. Fixes bug 16706; bugfix on 0.2.6.3-alpha.
  • Simplify hidden service descriptor creation by using an existing flag to check if an introduction point is established. Fixes bug 21599; bugfix on 0.2.7.2-alpha.
• Minor bugfixes (memory leak):
  • Fix a small memory leak at exit from the backtrace handler code. Fixes bug 21788; bugfix on 0.2.5.2-alpha. Patch from Daniel Pinto.
• Minor bugfixes (protocol, logging):
  • Downgrade a log statement about unexpected relay cells from "bug" to "protocol warning", because there is at least one use case where it can be triggered by a buggy tor implementation. Fixes bug 21293; bugfix on 0.1.1.14-alpha.
• Minor bugfixes (testing):
  • Use unbuffered I/O for utility functions around the process_handle_t type. This fixes unit test failures reported on OpenBSD and FreeBSD. Fixes bug 21654; bugfix on 0.2.3.1-alpha.
  • Make display of captured unit test log messages consistent. Fixes bug 21510; bugfix on 0.2.9.3-alpha.
  • Make test-network.sh always call chutney's test-network.sh. Previously, this only worked on systems which had bash installed, due to some bash-specific code in the script. Fixes bug 19699; bugfix on 0.3.0.4-rc. Follow-up to ticket 21581.
• Minor bugfixes (voting consistency):
  • Reject version numbers with non-numeric prefixes (such as +, -, or whitespace). Disallowing whitespace prevents differential version parsing between POSIX-based and Windows platforms. Fixes bug 21507 and part of 21508; bugfix on 0.0.8pre1.
• Minor bugfixes (windows, relay):
  • Resolve "Failure from drain_fd: No error" warnings on Windows relays. Fixes bug 21540; bugfix on 0.2.6.3-alpha.
• Code simplification and refactoring:
  • Break up the 630-line function connection_dir_client_reached_eof() into a dozen smaller functions. This change should help maintainability and readability of the client directory code.
  • Isolate our use of the openssl headers so that they are only included from our crypto wrapper modules, and from tests that examine those modules' internals. Closes ticket 21841.
  • Simplify our API to launch directory requests, making it more extensible and less error-prone. Now it's easier to add extra headers to directory requests. Closes ticket 21646.
  • Our base64 decoding functions no longer overestimate the output space that they need when parsing unpadded inputs. Closes ticket 17868.
  • Remove unused "ROUTER_ADDED_NOTIFY_GENERATOR" internal value. Resolves ticket 22213.
  • The logic that directory caches use to spool request to clients, serving them one part at a time so as not to allocate too much memory, has been refactored for consistency. Previously there was a separate spooling implementation per type of spoolable data. Now there is one common spooling implementation, with extensible data types. Closes ticket 21651.
  • Tor's compression module now supports multiple backends. Part of the implementation for proposal 278; closes ticket 21663.
• Documentation:
  • Clarify the behavior of the KeepAliveIsolateSOCKSAuth sub-option. Closes ticket 21873.
  • Correct documentation about the default DataDirectory value. Closes ticket 21151.
  • Document the default behavior of NumEntryGuards and NumDirectoryGuards correctly. Fixes bug 21715; bugfix on 0.3.0.1-alpha.
  • Document key=value pluggable transport arguments for Bridge lines in torrc. Fixes bug 20341; bugfix on 0.2.5.1-alpha.
  • Note that bandwidth-limiting options don't affect TCP headers or DNS. Closes ticket 17170.
• Removed features (configuration options, all in ticket 22060):
  • These configuration options are now marked Obsolete, and no longer have any effect: AllowInvalidNodes, AllowSingleHopCircuits, AllowSingleHopExits, ExcludeSingleHopRelays, FastFirstHopPK, TLSECGroup, WarnUnsafeSocks. They were first marked as deprecated in 0.2.9.2-alpha and have now been removed. The previous default behavior is now always chosen; the previous (less secure) non- default behavior is now unavailable.
  • CloseHSClientCircuitsImmediatelyOnTimeout and CloseHSServiceRendCircuitsImmediatelyOnTimeout were deprecated in 0.2.9.2-alpha and now have been removed. HS circuits never close on circuit build timeout; they have a longer timeout period.
  • {Control,DNS,Dir,Socks,Trans,NATD,OR}ListenAddress were deprecated in 0.2.9.2-alpha and now have been removed. Use the ORPort option (and others) to configure listen-only and advertise-only addresses.
• Removed features (tools):
  • We've removed the tor-checkkey tool from src/tools. Long ago, we used it to help people detect RSA keys that were generated by versions of Debian affected by CVE-2008-0166. But those keys have been out of circulation for ages, and this tool is no longer required. Closes ticket 21842.