FSF forge evaluation

Forge Software Evaluation

Disclaimer: All of the projects we considered were good in different ways. Our decisions are based on a long list of criteria.

https://www.fsf.org/blogs/sysadmin/the-fsf-tech-team-doing-more-for-free-software

Ongoing evaluation

• Gitea
  • Pros:
    • LibreJS is partially supported, does not look hard to add full support.
    • Someone else runs an instance open to free software: https://codeberg.org/
  • Cons:
    • Poor usability with no js from site (gnu criteria A): Can't make pull request. Can't attach file to issue. Can't create ssh key in user settings. Can't delete repo. Would need to add that functionality in plain html when no js is run.
    • No user import / export of issues and merge requests. https://github.com/go-gitea/gitea/issues/1446
    • Upstream hosted on GitHub which requires running nonfree JS for signup, https://github.com/go-gitea/gitea/issues/9303, but they seem to be very close (within a few weeks/months) to self hosting
  • Needs more research:
    • Resource requirements / performance under load
    • How featureful are is are administration interface/tools (GUI or CLI)
    • Ease of deployment, upgrading, debugging.
    • Finer points of GNU ethical repo criteria
    • import/export of data

• Pagure
  • Pros:
    • Someone else runs an instance open to free software: https://pagure.io
    • JS not aggregated, so should be easy to support LibreJS
    • User import / export of data for issues and merge requests
    • It is possible to configure Pagure to use user namespaces for projects.
  • Cons:
    • Somewhat limited usability with no JS from site (gnu criteria A): login, logout, create issue etc. Would need to investigate adding functionality without js. Could package js into an extension, but this is not ideal for users. Upstream patches are welcome: https://pagure.io/pagure/issue/3507
  • Needs more research:
    • Resource requirements / performance under load
    • How featureful are is are administration interface/tools (GUI or CLI)
    • Tools to prevent abuse. User banning is possible from Gui.
    • Ease of deployment, upgrading, debugging
    • Finer points of GNU ethical repo criteria
    • import/export of data

• Sourcehut
  • Pros:
    • Perfect usability with no js from site (gnu criteria A):
    • Only has 3 static js files. Extremely easy to support LibreJS
    • User export of data. Import for issues is in development.
    • GNU ethical repo criteria grade A seems like it requires minimal changes. https://lists.sr.ht/~sircmpwn/sr.ht-discuss/%3C87a79fifzl.fsf%40fsf.org%3E#%3CBY4SP60PHM4J.OFO3GMYYJ3OK@homura%3E
    • Bonus features: wiki, ci, supports email workflow, mercurial, agplv3
  • Cons:
    • Upstream claims it is alpha software.
    • Navigation between code & issues is difficult. In development upstream https://lists.sr.ht/~sircmpwn/sr.ht-discuss/%3C87woqgtr0s.fsf%40jb55.com%3E.
    • No traditional Web merge request UI, however a Web merge request can be done by creating a ticket and including the forked git repo url and branch name.
  • Needs more research:
    • Resource requirements / performance under load
    • How featureful are is are administration interface/tools (GUI or CLI)
    • Tools to prevent abuse.
    • Ease of deployment, upgrading, debugging
    • import/export of data

In all of the above:

• • Pros
    • Active upstream
    • 2 factor authentication support
    • Freely licensed

Evaluated and filtered out

We evaluated 15 different forge programs that we filtered out relatively quickly. Some seemed promising but were mainly designed for use by a single project/organization, not independent projects and open the public.

• GitLab
  • Pros:
    • Lots of features.
    • Very popular.
  • Cons:
    • GNU ethical repo criteria: gitlab.com listed as C, but has been operating at an F and will be reclassified soon because it sometimes requires users to run nonfree Google ReCAPTCHA code https://gitlab.com/gitlab-org/gitlab/issues/21998.
    • The forge we run should be something that would fit our own requirements to host as a project on the forge. There are major issues preventing that which could be solved through a fork, but we do not want to maintain a fork:
      • We would not want to recommend people contribute upstream as they will be randomly hit by a requirement to run nonfree ReCAPTCHA. We do not want to take contributions that upstream would normally accept.
      • The docs for nonfree features are part of gitlab CE. We don't want to host documentation for nonfree software so we would want to take it out. This looks to be a nontrivial effort.
      • GitLab has a fair bit of code and documentation we would want to remove. It integrates with a. Nonfree software, b. Services that require running nonfree JavaScript, c. Services that are substitutes for software (https://www.gnu.org/philosophy/who-does-that-server-really-serve.en.html). https://docs.gitlab.com/ce/integration/ some of them are: Google ReCAPTCHA, Sourcegraph, Trello, Bitbucket, Akismet, YouTrack, Jira, about 10 different project services https://gitlab.com/gitlab-org/gitlab-foss/tree/master/app/models/project_services. Identifying all such code and documentation, removing and maintaining that appears to be nontrivial.
      • If we were to maintain a fork, a rename would be preferable because when you say GitLab, people think to gitlab.com which is a nonfree version of GitLab. Renaming is nontrivial.
    • GitLab is a known target of spam scripts. Upstream uses Akismet, which is unethical due to being a software substitute https://www.gnu.org/philosophy/who-does-that-server-really-serve.en.html and requiring nonfree JS https://www.gnu.org/philosophy/javascript-trap.en.html. Upstream is not working on an alternative to help free software users.
    • GitLab is quite complicated. Adding LibreJS support would be nontrivial.
    • GitLab has bad licensing recommendations, we would need to remove or change. This is nontrivial. see https://gitlab.com/gitlab-org/gitlab/issues/15208 and https://www.gnu.org/licenses/identify-licenses-clearly.en.html
    • Given the myriad of nonfree/ethical problems, and the fact that upstream has spent basically no work to fix them, any fork we maintain we should expect new freedom issues and more work to correct them over time.