FSF forge evaluation

Forge Software Evaluation

Disclaimer: All of the projects we considered were good in different ways. Our decisions are based on a long list of criteria.

https://www.fsf.org/blogs/sysadmin/the-fsf-tech-team-doing-more-for-free-software

Ongoing evaluation

• Gitea
  • Pros:
    • LibreJS is partially supported, does not look hard to add full support.
    • Two-factor authentication (2FA)
    • Someone else runs an instance open to free software: https://codeberg.org/
  • Cons:
    • Poor usability with no js from site (gnu criteria A): Can't make pull request. Can't attach file to issue. Can't create ssh key in user settings. Can't delete repo. Maybe could package js into an extension.
    • No user import / export of issues and merge requests. https://github.com/go-gitea/gitea/issues/1446
    • Upstream hosted on GitHub which requires running nonfree JS for signup, https://github.com/go-gitea/gitea/issues/9303, but they seem to be very close (within a few weeks/months) to self hosting
  • Needs more research:
    • Resource requirements / performance under load
    • How featureful are is are administration interface/tools (GUI or CLI)
    • Ease of deployment, upgrading, debugging.
    • Finer points of GNU ethical repo criteria

• Pagure
  • Pros:
    • Someone else runs an instance open to free software: https://pagure.io
    • JS not aggregated, so should be easy to support LibreJS
    • User import / export of data for issues and merge requests
  • Cons:
    • Very bad usability with no js from site (gnu criteria A): login, logout, create issue etc. Maybe could package js into an extension.
    • There is only a top level namespace, so there might be some conflicts in terms of who reserves a repo name first. With the import/export functionality, at least a fork does not need to be stuck under /fork.
  • Needs more research:
    • Resource requirements / performance under load
    • How featureful are is are administration interface/tools (GUI or CLI)
    • Tools to prevent abuse. User banning is possible from Gui.
    • Ease of deployment, upgrading, debugging
    • Finer points of GNU ethical repo criteria

• Sourcehut
  • Pros:
    • Perfect bad usability with no js from site (gnu criteria A):
    • Only has 3 static js files. Extremely easy to support LibreJS
    • User export of data for issues. Import is in development.
  • Cons:
    • No web based pull request workflow
  • Needs more research:
    • Resource requirements / performance under load
    • How featureful are is are administration interface/tools (GUI or CLI)
    • Tools to prevent abuse.
    • Ease of deployment, upgrading, debugging
    • Finer points of GNU ethical repo criteria

In all of the above:

• • Pros
    • Active upstream
    • 2 factor authentication support
    • Freely licensed

Evaluated and filtered out

We evaluated 15 different forge programs that we filtered out relatively quickly. Some seemed promising but were mainly designed for use by a single project/organization, not independent projects and open the public.

• GitLab
  • Pros:
    • Lots of features.
    • Very popular.
  • Cons:
    • GNU ethical repo criteria: gitlab.com listed as C, but has been operating at an F because it sometimes requires users to run nonfree Google ReCAPTCHA code https://gitlab.com/gitlab-org/gitlab/issues/21998.
    • The forge we run should be something we are ethically fine hosting a repo of. There are major issues preventing that which could be solved through a fork, but we do not want to maintain a fork:
      • We would not want to recommend people contribute upstream as they will be randomly hit by a requirement to run nonfree ReCAPTCHA. We do not want to take contributions that upstream would normally accept.
      • The docs for nonfree features are part of the free version docs. We obviously do not want to host documentation for nonfree software so we would want to take it out. This looks to be a nontrivial effort, because it is ingrained in the interface.
      • GitLab has a fair bit of code and documentation we would want to remove. It integrates with a. Nonfree software, b. Services that require running nonfree JavaScript, c. Services that are substitutes for software (https://www.gnu.org/philosophy/who-does-that-server-really-serve.en.html). https://docs.gitlab.com/ce/integration/ some of them are: Google ReCAPTCHA, Sourcegraph, Trello, Bitbucket, Akismet, YouTrack, Jira, about 10 different project services https://gitlab.com/gitlab-org/gitlab-foss/tree/master/app/models/project_services. Identifying all such code and documentation, removing and maintaining that appears to be nontrivial.
      • If we were to maintain a fork, a rename would be preferable because when you say GitLab, people think to gitlab.com which is a nonfree version of GitLab. Renaming is nontrivial.
    • GitLab is a known target of spam scripts. Upstream uses Akismet, which is unethical due to being a software substitute and requiring nonfree JS. Upstream is not working on an alternative to help free software users.
    • GitLab is quite complicated. Adding LibreJS support would be nontrivial.
    • GitLab has bad licensing recommendations, we would need to remove or change. This is nontrivial. https://gitlab.com/gitlab-org/gitlab/issues/15208