Tor has been using Trac until June 2020, when we moved to our self-hostedinstance of Gitlab administered by the Tor sysadmin team. We reached some limitations with Trac as well as were concern on some of the plugins we depended on not being mantained. The challenges on doing this migration sooner were and are related to the capacity that we have to adapt a new ticketing system to our needs.
We're hoping Gitlab will be a good fit because:
Gitlab will allow us to collect our different engineering tools into a single application: Git repository handling, Wiki, Issue tracking, Code reviews, and project management tooling.
Gitlab is well-maintained, while Trac plugins are not well maintained and Trac itself hasn't seen a release for over a year (since 2019).
Gitlab will allow us to build a more modern approach to handling Continuous Integration for our different projects. This is going to happen after the ticket and wiki migration.
We spent several months fixing and testing problemson data migration, from formatting issues to where the information of trac goes to live to in Gitlab. We tested the Gitlab instance with a few projects until we jumped into migrating all data from Trac. You can read about the use cases for a bug tracker at Tor in this ticket.
To accomplish the migration, the Gitlab migration group wrote a number of tools to make the migration happen. These tools were split into two parts: one part for fetching all the state from Trac and a number of tool from turning the Trac state into Gitlab issues and wiki content in various ways. The migration group worked with the engineering teams in the organisation on issues such as splitting up Trac "flat ticket namespace" into the different groups and projects that we wanted to have on Gitlab. This allowed the individual teams at Tor to decide the organisation they wanted to have on Gitlab and allow them to build a mapping that fits better with the project model of Gitlab, where each project have an issue tracker, a wiki, and a repository attached.
We specified a specific date for the migration to happen where all of Tor's engineering teams were asked to find different means of doing their work than using Trac while Trac was put in read-only mode. During this period the migration group worked together on the actual migration, verifying that all data was properly migrated to a point where things looked satisfying, and then finally we announced that Gitlab was what we would move to next. We did the transition period with start on Friday afternoon over the weekend to ensure that only a minimum amount of disruption would be caused by this.
The period after the migration did involve a bit of support handling from the different teams, but we ar amazed at how quickly everybody picked up the new work flows and we believe that Gitlab have made it easier for engineers to make choices around their respective projects now without needing help from the Gitlab admin team.
We are not migrating away from Gitolite and Jenkins just yet. This means those services are still fully operational and their equivalent features in GitLab are not supported (namely Git hosting and CI). Those services might eventually be migrated to GitLab.
The issues and wiki of the "Tor" project are migrated. There are no other projects in Trac.
Trac issues that remain are really legacy issues, others issues have been "moved" to the respective projects. All the tickets that were not moved to their respective projects have been closed in the first week of July 2020. Next year we will permanently shut Trac down and keep it archived in the Wayback machine.
To request a new account you have to fill the form in https://gitlab.onionize.space/ where we get the request and a few of us attend to them. Through the Outreachy Internship we are mentoring an intern that will help improve this application.
To be able to have all issues in one same board we created a main group "tpo" where all our projects live. The structure for the rest of the projects is:
Organization: host our main wiki, which links to documentation for all projects at TPO. It also hold issues that may not be related to any particular project but are organizational on TPO.
TPA: host any project related to the infrastructure administered by TPO
Gitlab : Any issue or documentation related to running the Gitlab instance.
Thank you for migrating the wiki. Please reduce the CSS padding and margins on tables so tables don't have to be scrolled as far horizontally. Please repair broken tables (example).
Please create a cypherpunks account so we, privacy-oriented Tor users, can revise wiki pages ourselves.
The issue is that to have a cypherpunk account that can revise wiki pages we would have to have it in the groups as a reporter at least. And that would give the account a lot more permissions that we are comfortable with. The wiki does not work so well in gitlab for us as we really need anonymous users to be able to contribute to it. This is something that will still be up for discussion on how to change it.
Have you explained this use-case to GitLab's developers to hear from them if it's possible in the current release? I read their documentation and haven't found any loopholes that might allow a situation like it. You can select a user role, but it appears not possible to fine-tune permissions or create custom user roles.
There are good solutions for wikis, though: MediaWiki, DokuWiki, PmWiki, ...
Do we need a valid email to sign up on Tor Project's GitLab?
Are you ever planning on making an official web forum? You must've noticed the sharp decline of your user mailing lists. People have to go places like Reddit to get answers and it's just shameful. Even worse, 4chan (shudders).
>>> We are discussing the installation of a discourse
But does "Discourse" support anonymous comments?
>>> to act as an official forum and work as comments of this blog.
I would prefer to use something like commenting at "opennet.ru" - it looks like a custom implementation (AFAIR owner discussed some details at sub of "questions to administration"). Implementation is quite democratic (as no need registration) and require no JavaScript (also there is a capture at extra screen to avoid spam messages).
May be it is reasonable to use some custom implementation? - I would vote for implementation that supports
1) anonymous comments feature (no registration)
2) No JavaScript to post messages
Just to emphasize: email is completely unsuitable for Tor users. I am sure you yourself understand why and I wish Tor Project had easy options.
The situation can be fixed, and the incoming US administration just might* be willing to pay attention to calls for creating a privacy industry in the USA (as economic stimulus, as helping tech people unemployed by expected breakup of Big Tech companies, as responding to an unmet need by empowering ordinary citizens, etc). Innovation is needed and USG can provide some neccessary seed money.
*Rumors that Biden plans to put Eric Schmidt ('privacy is dead; get over it!") in his cabinet is not encouraging, to be sure.
People don't have to go to Reddit or 4chan. People default to those because they reflexively stay inside their social knowledge bubble or don't search for specialized places or are loyal to a perception of convenience, etc. It's like why freer markets go toward oligarchs: Alphabet's Google, Amazon, Facebook, OS app stores, Microsoft, etc. But to get answers about Tor, there are options other than the mailing list:
Why the hell is Trac.torproject.org suddenly redirecting to GitLab today?! The wiki on GitLab is NOT ready for use. Migration must be considered as "in progress" until they are accessibly identical. Do NOT cut off access to Trac at this point. GitLab does NOT have a proper landing page yet. The wiki pages on GitLab are NOT easy for users to find. Various wiki pages are BROKEN on GitLab. Cutting off access should be the LAST thing you do after they are accessibly identical. See also: https://ocewjwkdco.tudasnich.de/comment/289317#comment-289317
Comments
Please note that the comment area below has been archived.
Please reduce the CSS…
Thank you for migrating the wiki. Please reduce the CSS padding and margins on tables so tables don't have to be scrolled as far horizontally. Please repair broken tables (example).
Please create a cypherpunks account so we, privacy-oriented Tor users, can revise wiki pages ourselves.
The issue is that to have a…
The issue is that to have a cypherpunk account that can revise wiki pages we would have to have it in the groups as a reporter at least. And that would give the account a lot more permissions that we are comfortable with. The wiki does not work so well in gitlab for us as we really need anonymous users to be able to contribute to it. This is something that will still be up for discussion on how to change it.
Class A - other list - S.s…
Class A - other list - S.s link was changed in 2020 after Daniel hosting problem. Please update them.
Latest URLs are listed in any readmes here.
Have you explained this use…
Have you explained this use-case to GitLab's developers to hear from them if it's possible in the current release? I read their documentation and haven't found any loopholes that might allow a situation like it. You can select a user role, but it appears not possible to fine-tune permissions or create custom user roles.
There are good solutions for wikis, though: MediaWiki, DokuWiki, PmWiki, ...
Do we need a valid email to sign up on Tor Project's GitLab?
Please replace the broken…
Please replace the broken URL in the "Proof" table cell of DuckDuckGo's onion to be these two working URLs:
https://web.archive.org/web/20150420124527/https://duck.co/blog/friends…
https://twitter.com/DuckDuckGo/status/351417664083734528
> CI What means this?
> CI
What means this?
Continuous Integration…
Continuous Integration. https://en.wikipedia.org/wiki/Continuous_integration
I will change it in the main blogpost to not have an acronym. Thanks.
Next year we will…
Are you yourselves going to make sure every Trac URL is archived, either manually or by working with archive.org / Archive Team?
We will be working to make…
We will be working to make sure that everything is archived. Please report if you find anything that is not working as it should. Thanks!
Are you ever planning on…
Are you ever planning on making an official web forum? You must've noticed the sharp decline of your user mailing lists. People have to go places like Reddit to get answers and it's just shameful. Even worse, 4chan (shudders).
I personally agree. We are…
I personally agree. We are discussing the installation of a discourse to act as an official forum and work as comments of this blog.
>>> We are discussing the…
>>> We are discussing the installation of a discourse
But does "Discourse" support anonymous comments?
>>> to act as an official forum and work as comments of this blog.
I would prefer to use something like commenting at "opennet.ru" - it looks like a custom implementation (AFAIR owner discussed some details at sub of "questions to administration"). Implementation is quite democratic (as no need registration) and require no JavaScript (also there is a capture at extra screen to avoid spam messages).
May be it is reasonable to use some custom implementation? - I would vote for implementation that supports
1) anonymous comments feature (no registration)
2) No JavaScript to post messages
Just to emphasize: email is…
Just to emphasize: email is completely unsuitable for Tor users. I am sure you yourself understand why and I wish Tor Project had easy options.
The situation can be fixed, and the incoming US administration just might* be willing to pay attention to calls for creating a privacy industry in the USA (as economic stimulus, as helping tech people unemployed by expected breakup of Big Tech companies, as responding to an unmet need by empowering ordinary citizens, etc). Innovation is needed and USG can provide some neccessary seed money.
*Rumors that Biden plans to put Eric Schmidt ('privacy is dead; get over it!") in his cabinet is not encouraging, to be sure.
I'm not going to collaborate…
I'm not going to collaborate on gitlab. It's not working without j/s. Old trac was able to edit without j/s
If I recall correctly, the…
If I recall correctly, the captcha on trac did require javascript.
Do you mean Discourse, the…
Do you mean Discourse, the software? Or Discord, the software? Or just a vague idea of something to assist commenting?
They probably didn't mean…
They probably didn't mean Discord since it's almost impossible to use with Tor Browser.
People don't have to go to…
People don't have to go to Reddit or 4chan. People default to those because they reflexively stay inside their social knowledge bubble or don't search for specialized places or are loyal to a perception of convenience, etc. It's like why freer markets go toward oligarchs: Alphabet's Google, Amazon, Facebook, OS app stores, Microsoft, etc. But to get answers about Tor, there are options other than the mailing list:
https://sedvblmbog.tudasnich.de/contact/
https://ijpaagiacu.tudasnich.de/get-in-touch/
https://ocewjwkdco.tudasnich.de/help-create-qa-site-tor (Stack Exchange)
https://trac.torproject.org/projects/tor/wiki/UserIssuesResearch
gitlab is missing onion…
gitlab is missing onion addresses. trac had it. how can i contribute and report bug through onions safety?
what is gitlab approval time…
what is gitlab approval time? problematic email domain is excluded? I wait some days
Why Gitlab? Have you ever…
Why Gitlab?
Have you ever tried Gitea (example)?
Gitlab is unusable without JavaScript. Other solutions like Gitea, Mediawiki, and Trac are usable without JS.
Others have asked about…
Others have asked about Gitea:
https://ocewjwkdco.tudasnich.de/comment/283607#comment-283607
https://ocewjwkdco.tudasnich.de/comment/287085#comment-287085
SourceHut, like Gitea, is…
Like Gitea, SourceHut is also usable without JavaScript, but SourceHut's build system targets Linux for now.
Why the hell is Trac…
Why the hell is Trac.torproject.org suddenly redirecting to GitLab today?! The wiki on GitLab is NOT ready for use. Migration must be considered as "in progress" until they are accessibly identical. Do NOT cut off access to Trac at this point. GitLab does NOT have a proper landing page yet. The wiki pages on GitLab are NOT easy for users to find. Various wiki pages are BROKEN on GitLab. Cutting off access should be the LAST thing you do after they are accessibly identical. See also: https://ocewjwkdco.tudasnich.de/comment/289317#comment-289317