Group: Copilot Watch Group/Research

Copilot Research

• Note: This is a work-in-progress. Please add your research and findings below. We are just trying to put things in a public-facing location, so that others can participate. Thank you!*

Objective/Purpose

To find out what is bad about copilot, and to what extent. Please add any research you have done below, as we are trying to understand better what Copilot is doing and in what ways it is bad for free software and its users.

Research Findings

Scholarly Research

• Five papers on the implications of copilot were published by the Free Software Foundation and are available at: https://www.fsf.org/news/publication-of-the-fsf-funded-white-papers-on-questions-around-copilot
• Educational Opportunities and Challenges of AI Code Generation

Videos

• This video shows a person typing comments: <https://yewtu.be/watch?v=DeO7xLXORpY>
• "Do not use" (discusses copyright): <https://yewtu.be/watch?v=b9u3ZAGQmT0>
• Recent and discusses copyright and licensing: <https://yewtu.be/watch?v=CHvIOgSFp9I>
• Year old, but shows large portions of verbatim copying, including the license: <https://yewtu.be/watch?v=xxX7dpYSClQ&t>
• Another video of large portions of code being copied, including the license: <https://nitter.it/moyix/status/1432085687365513225>

Photos

• See tweet by Tim Davis (Oct 16, 2022):
• Tim's code is on left; copilot on right -- csparse
• "Sparse matrix addition, from github.com/DrTimothyAldenDav… but also reguritated by copilot with "// sparse matrix add, cs_". My code on left, github on right."
• "Other codes are slightly morphed. I could probably reproduce my entire sparse matrix libraries from simple prompts. My code on left (in the Terminal). Prompt is "sparse matrix elimination tree, cs_". Github on right."

Mentions in Publications or Talks

Richard Stallman discusses the implications of copilot briefly in his talk "The state of the free software movement." See from timestamp [0:38:24] on LibrePlanet:Conference/2022/Transcripts/RMS-state-of-free-software

Relevant articles

• Do GitHub's updated terms of service conflict with copyleft?
• Ethics in, ethics out -- promote user-respecting software development platforms
• What Is GitHub Copilot And Why Are Developers Hating On It?
• GitHub Copilot AI Improved, Offered as API: 'A Taste of the Future'
• Evaluating Large Language Models Trained on Code
• Minecraft’s Code-Writing AI Points to the Future of Computers
• GitHub Copilot and open source laundering
• https://devclass.com/2022/10/17/github-copilot-under-fire-as-dev-claims-it-emits-large-chunks-of-my-copyrighted-code/
• githubcopilotinvestigation.com
• 2022-10-19 Register article: "On Monday, Matthew Butterick, a lawyer, designer, and developer, announced he is working with Joseph Saveri Law Firm to investigate the possibility of filing a copyright claim against GitHub."

Notable from photo/video

• Users may just type comments of functions they want, and the code appears for them below
• (May be out of date but...) license text can be "summoned" verbatim (but line by line). This could be problematic even if it wasn't verbatim because what if a user just copied most, but "tweaked" a few things in, for example, the GPL.
• https://nitter.net/DocSparse/status/1581461734665367554 has three examples of non-trivial code that is between 16-22 lines long structured very similarly.

Points brought up by general public

• "The thing is that anybody can put my code on github even if all my hobby projects are on a self hosted gitlab community editing instance." (and, of course, this can include projects which may not neessarily be "hobby")

Not yet known

• How many lines of code can be copied verbatim? (Certainly, the license copy example is too much) And, how many lines can be copied? *Update: So far, we have between 16-22 lines of code documented in photos (see above tweet by Tim Davis).
• To what extent are the keystrokes etc. of copilot users being tracked and analyzed by Microsoft?

Preliminary conclusions

(These need to be verified and backed up with research. Please provide links, if you have them.)

• People say, (and even GitHub's own Web site says,) "0.1% of the time, the code is verbatim"
• Certainly, copilot never tells you what code any of this is being copied from (nor its license) no matter how many lines of code.
• Using copilot requires running nonfree software (e.g. visual composer, GitHub's javascript), which in itself is an injustice.
• Copilot is a SaaSS