Title:Jupyter Notebooks on GitHub: Characteristics and Code Clones

Abstract:Jupyter notebooks have emerged as a standard tool for data science programming. Programs in Jupiter notebooks are different from typical programs by dividing code into snippets interleaved with text and visualisation and allowing interactive exploration and execution of snippets in different order. Previous studies have shown the presence of considerable code duplication in sources of traditional programs, in both so-called systems programming languages and so-called scripting languages.
In this paper we present the first large-scale study of code cloning in Jupyter notebooks. We analyse a corpus of 2.7 million Jupyter notebooks hosted on GitHJub, representing over 36974714 individual snippets and 226744094 lines of code. We study clones at the level of individual snippets, and study the extent to which snippets are recurring across multiple notebooks. We study both identical clones clones (with possible differences in whitespaces) and approximate clones and conduct a small-scale ocular inspection of the most common clones.
We find that that code cloning is common in Jupyter notebooks -- more than 70% of all code snippets are exact copies of other snippets, and around 50% of all notebooks do not have a unique snippet, but consists solely of snippets that are also found elsewhere. In notebooks written in Python, around 80% of all snippets are near-miss clones and the prevalence of code cloning is higher in Python than in other languages.
We further find that clones between different repositories are far more common than clones within the same repository. However, the most common individual repository from which a Jupyter notebook contains clones is the repository in which itself resides.