FSF 2020 forge evaluation

From LibrePlanet
(Redirected from Fsf 2019 forge evaluation)
Jump to: navigation, search

Forge Software Evaluation

Disclaimer: All of the projects we considered were good in different ways. Our decisions are based on a long list of criteria.

https://www.fsf.org/blogs/sysadmin/the-fsf-tech-team-doing-more-for-free-software


Ongoing evaluation

  • Pagure (seems most likely, its our current focus)
    • Pros:
      • Someone else runs an instance open to free software: https://pagure.io
      • JS not aggregated, so should be easy to support LibreJS
      • User import / export of data for issues and merge requests
      • It is possible to configure Pagure to use user namespaces for projects.
    • Cons:
      • We eventually want to make our forge work without JavaScript at all, for an A rating on the GNU ethical criteria. Depending upon which user authentication module we use, or whether we create our own, we that may require some work. We also may have to make such changes for getting issue creation to work with an A rating. For the initial B rating, LibreJS support for all of the JS is required and trivial, and it's okay if the site requires some JS to work. In terms of making the site work without JS, upstream patches are welcome: https://pagure.io/pagure/issue/3507
      • Does not support Mercurial
    • Needs more research:
      • Resource requirements / performance under load
      • How featureful are is are administration interface/tools (GUI or CLI)
      • Tools to prevent abuse. User banning is possible from GUI.
      • Ease of deployment, upgrading, debugging
      • Finer points of GNU ethical repo criteria
      • import/export of data
  • Gitea
    • Pros:
      • LibreJS is partially supported, does not look hard to add full support.
      • Someone else runs an instance open to free software: https://codeberg.org/
    • Cons:
      • Poor usability with no JS from site (gnu criteria A): Can't make pull request. Can't attach file to issue. Can't create ssh key in user settings. Can't delete repo. Would need to add that functionality in plain html when no js is run.
      • No user import / export of issues and merge requests. https://github.com/go-gitea/gitea/issues/1446
      • Upstream hosted on GitHub which requires running nonfree JS for signup, https://github.com/go-gitea/gitea/issues/9303, but they seem to be very close (within a few weeks/months) to self hosting
      • Does not support Mercurial
    • Needs more research:
      • Resource requirements / performance under load
      • How featureful are is are administration interface/tools (GUI or CLI)
      • Ease of deployment, upgrading, debugging.
      • Finer points of GNU ethical repo criteria
      • import/export of data
  • sourcehut
    • Pros:
    • Cons:
      • Upstream claims it is alpha software.
      • Navigation between code & issues is difficult. In development upstream https://lists.sr.ht/~sircmpwn/sr.ht-discuss/%3C87woqgtr0s.fsf%40jb55.com%3E.
      • Email + web for patches (aka pull requests). Completely web based pull requests are popular, however sr.ht has web features that make it much easier than plain GNU Mailman. For example it has mailto links, no need for email to accept pull requests, very simply guide for sending email patches, etc.
    • Needs more research:
      • Resource requirements / performance under load
      • How featureful are is are administration interface/tools (GUI or CLI)
      • Tools to prevent abuse.
      • Ease of deployment, upgrading, debugging
      • import/export of data

In all of the above:

    • Pros
      • Active upstream
      • 2 factor authentication support
      • Freely licensed


Evaluated and not currently focused on

We evaluated 15 different forge programs that we decided not to go with. Some of the projects not listed below also seemed promising but were mainly designed for use by a single project/organization, not independent projects and open the public.


  • GitLab
    • Pros:
      • Lots of features.
      • Very popular.
    • Cons:
      • Someone else runs an instance open to any free software: Doesn't exist for GitLab. All community run instances only accept projects/repos related to their community. Many use the nonfree ReCAPTCHA as well. gitlab.com is a nonfree version.
      • GNU ethical repo criteria: gitlab.com listed as C, but has been operating at an F and will be reclassified soon because it sometimes requires users to run nonfree Google ReCAPTCHA code they have been very slowly working on moving away from for almost 2 years now, https://gitlab.com/gitlab-org/gitlab-foss/issues/46548 https://gitlab.com/gitlab-org/gitlab/issues/21998 (https://www.gnu.org/philosophy/javascript-trap.en.html), which also has problems like it helps google track users and train their AI models.
      • The forge we run should be something that would fit our own requirements to host as a project on the forge. There are major issues preventing that which could be solved through a fork, but we do not want to maintain a fork:
        • We would not want to recommend people contribute upstream as they will be required to run nonfree software, but we don't want to maintain a fork with code that would naturally belong upstream.
          • They will likely be required to run nonfree ReCAPTCHA code.
          • Upstream only accepts contributions to a repo that contains the nonfree parts, so you need to download nonfree software.
          • Running the test suite implies running nonfree software.
        • The docs for nonfree features are part of GitLab CE. We don't want to host documentation for nonfree software so we would want to take it out. This looks to be a nontrivial effort.
        • GitLab has a fair bit of code and documentation we would want to remove: It integrates with a. Nonfree software, b. Services that require running nonfree JavaScript, c. Services that are substitutes for software (https://www.gnu.org/philosophy/who-does-that-server-really-serve.en.html). https://docs.gitlab.com/ce/integration/ some of them are: Google ReCAPTCHA, Sourcegraph, Trello, Bitbucket, Akismet, YouTrack, Jira, about 10 different project services https://gitlab.com/gitlab-org/gitlab-foss/tree/master/app/models/project_services. Gravatar is somewhere else. Identifying all such code and documentation, removing and maintaining that appears to be nontrivial.
        • If we were to maintain a fork, a rename would be important because when you say GitLab, people think of gitlab.com which is a nonfree version of GitLab. Even GitLab CE, most instances run nonfree software or SaaSS. Renaming is nontrivial.
      • GitLab is a known target of spam scripts. Upstream uses ReCAPTCHA (see above), but also Akismet, which is unethical due to being a software substitute and requiring nonfree JS. Upstream is not working on an alternative to help free software users. We would be stuck developing our own anti-spam solutions.
      • GitLab is quite complicated. Adding LibreJS support would be nontrivial.
      • GitLab has licensing recommendations that leads thousands of developers to leave their code's licensing unclear. We would need to remove or change them. This is nontrivial. see https://gitlab.com/gitlab-org/gitlab/issues/15208 and https://www.gnu.org/licenses/identify-licenses-clearly.en.html
      • Given the current presence of nonfree/ethical problems, any fork we maintain we should expect new freedom issues and more work to correct them over time. Related: in 2019 they put on hold a plan to add more nonfree JS tracking to gitlab.com and we are grateful for that. However, nonfree JS tracking already existed and still does on gitlab.com.
      • Does not support Mercurial