Tools
Contents
Organisational Tools
There are a few general purpose tools which enabled us to collaborate successfully with so many people on a project of this scale during the course of a year. Most tools used were used on a project-wide scale, meaning that every team made use of them. Sometimes, the specific team structures required extra tools which were then brought in as needed. The tools used were selected based on a few fundamental requirements. These will be explained using our standard collaborative writing environment, Etherpad (Fig. 1). For our project, we mostly selected general tools which provided a small but functional feature set which could be used in multiple situations. This allowed us to have a clear understanding of the functionality of any one given tool, even given the amount of different tools we used during the project year. Another important point was the scalability of our tools. This was an important criterion from the beginning as at least 25 people needed to be able to use the tools at any given time. Since the choices for most tools were made in the beginning, when the possibility to have a growing team was still in place, we opted for tools that would have allowed us to work with an even larger team than the initial starting group.
Fig. 1: Etherpad including a default description.
Etherpad was a key tool used throughout a project. It is a great choice since it is one of the best options for having live online collaboration on a document. The Etherpads used were mostly hosted by the University of Osnabrück, which provided the performance we needed. When having such a big and diverse team, it is important to consider the on-boarding process for each service or tool. For Etherpad, the onboarding process is one of the easiest possible. The access is not restricted since there is no account or login required. Furthermore, the Etherpad lite software hosted on the university server is compatible with all modern web browsers including Firefox, Safari, and Chromium-based browsers. The learning curve for Etherpad is very shallow since almost all options are displayed to the user and only very few options are kept in a separate menu, which is itself only one layer. This shallow learning curve made a tutorial on Etherpad unnecessary and every team member could use it easily. The example of Etherpad reflects a gold standard with regard to utility, scalability, on-boarding, and access. However, a lot of this is achieved by the limited feature set of the service and to have a reliable collaboration, more specialized tools were need. They will be showcased in the following sections.
We made use of a lot of tools hosted by the University of Osnabrück as well as the Gesellschaft für Wissenschaftliche Datenverarbeitung mbH Göttingen (Society for Scientific Data Processing Göttingen, Limited). These two providers allow login using the account provided to every student of the University of Osnabrück, thus reducing the effort of onboarding for the team. Whenever a service mentioned in the following sections was hosted by either of the two above organizations, the team’s university credentials were used to access it. Since the project took place as a study project, there were multiple university courses for it, each with a corresponding page on StudIP (the online course management system at the University of Osnabrück). Students could participate in either the full study project or in an interdisciplinary seminar with reduced hours. To streamline the communication, all the activity was put into one course on StudIP for the whole duration of the project.
The tools used can be roughly separated into two groups - communication services, and collaborative services. The communication services allowed the team to stay in touch synchronously during meetings and asynchronously otherwise. Furthermore, some tools were used to capture the opinion of the team when key decisions needed to be made. The collaborative services helped to bring the collective work together in a remote setting by helping to structure the tasks and combine completed work.
Communication Tools
For our everyday communication, we used the Matrix chat protocol, of which the University of Osnabrück hosts an instance. As a default client, we opted for Element (introduced above; Fig. 2), which is developed by the same core team as the Matrix protocol and provides all the necessary features of modern team chats like Microsoft Teams, Slack, or Rocket Chat. Although the matrix protocol can in principle be used with any one of a vast selection of clients and still be at its core interoperable, having a default client ensured that every team member had access to the same features. In Element, each chat is structured as one room which can have a varying amount of members. The rooms can be grouped into units called “spaces.” Spaces can contain rooms as well as further additional spaces. This allowed us to split the communication into organizational or functional units. The exact layout of the Element space will be discussed in the section Communication Between the Teams.
Fig. 2: Overview over the main Element rooms.
As an additional written communication channel, e-mails were also used. Here, the team’s e-mail accounts with the University of Osnabrück were used. The most common use case for e-mails was as an announcement tool for messages by the project management team. This is also further discussed in the Communication Between the Teams section.
Since this project was a remote-first project, all mandatory meetings needed to be available online. For this, the BigBlueButton video conferencing instance hosted by the university was used. This tool was familiar to most of the group since it is employed by the University of Osnabrück for online lectures. Besides the basic meeting functionality, the poll function could also be used to quickly assess the sentiment of the team regarding specific topics. The other crucial feature was the ability to record meetings and automatically upload those recordings into the Opencast section of the StudIP course. This allowed anyone to watch the meetings after they took place, which was important for a better integration of the team across time zones.
The StudIP course also provided the possibility to create polls, which was used every time an important decision required the whole team to give their opinion. The poll links were shared via mail and stayed open for multiple days to accommodate different schedules and time zones.
Collaboration Tools
Some early planning for mapping out project ideas and a skills map of the team was done using the whiteboard program Mural. Here, individual text blocks can be created and freely moved on a canvas. Ideas and notes could be captured, grouped, and rearranged. This proved beneficial in the creative planning. The SmartPlanner team also continued to use a similar service called Miro throughout the duration of the project to collect their issues and collaborate on their agile workflow Kanban board.
One of the most important and most used tools aside from Element was GitLab. GitLab is a web application for version control based on Git. In addition to providing the general functionalities a Git platform needs, the application offers a broad variety of helpful tools for software development in larger teams, including an issue tracking system, management of projects consisting of more than one Git repository, a Kanban board, and the infrastructure for continuous integration and continuous delivery. The GitLab instance used in this project was hosted be the Gesellschaft für Wissenschaftliche Datenverarbeitung mbH and thus gave us the added benefit of an easy onboarding for the team.
Out of the additional features of GitLab which we used heavily was the issue tracking system. An “issue” in this context can be described as a forum entry regarding a problem or feature request for the software project. It consists of a title and a description, can be tagged with so-called “labels” and thereby grouped, can be assigned to a milestone, can have a deadline, can be referenced in commit messages and other texts within the system, and can be discussed in comments towards that issue on its corresponding page in the application. There are still many more functionalities regarding issues, but these are the most important ones. Issues can be understood as encapsulated information hubs dedicated to a certain work topic.
Further, there are so-called “merge requests”. A merge request is an issue that is dedicated to the specific task of merging one branch into another. It basically represents the demand “please look at my changes on branch A, approve them, and merge them into Branch B.” Single code changes can be commented and discussed in forum-like threads, merge requests can be approved by other developers, and the merges can be interactively performed by clicking on a single button (see also Reviews and Merge Requests).
The last important thing that GitLab provides is the ability to easily host static webpages (such as this documentation website; see also Git).
Design Tools
The term “Design Tools” refers to all software and non-software that was used to create the design and user experience of our project. We primarily used the design software Figma, a vector-based design tool that is built for online collaboration and is commonly used for user interface and user experience design. The advantage of Figma over similar services is that Figma’s basic version is free to use and allows design projects to be shared with unlimited collaborators. Its collaborative features include real-time collaborative editing, commenting, and sharing of the contents of a design project.
Fig. 3: Figma logo.
The Core team used Figma to collaboratively create the general design guide as well as to design many pages created by the Core. The SmartPlanner team also used Figma to create most of their components. They also used the commenting function to improve their designs in a series of design review phases. The StudyBuddyMatch team used Figma to create an overview of their questionnaires and matching process.
Development Tools
By the term “Development Tools” we mean all tools that can be used in order to support coding and code management, such as code editors, integrated development environments (IDE), or version control systems (VCS). While IDEs like PyCharm are quite well known by most people who have coded at least once in their lives, VCSs only get more useful once you get to teams larger than four or more actively working developers. Further, in such a large project, the amounts of code exceed what we have known by now from most of our university courses, so the many features of complex IDEs became more and more useful to us. In the following we will describe the development tools we have used in the project and what features came in most handy to us.
What is an IDE?
An integrated development environment (IDE) is a software application that provides comprehensive facilities to computer programmers for software development. An IDE normally consists of at least a source code editor, build automation tools and a debugger. - Wikipedia
In modern software development, at least simple IDEs are so heavily integrated in the development process that most developers couldn’t live without them. They make many simple, but time- and nerve-consuming tasks you face as a developer much easier and way more time-efficient than doing all these things by hand. Imagine, for instance, renaming a variable in a large software project. You would have to search each and every file by hand for usages of that variable and replace it with the new name. Automated code refactoring does that for you in milliseconds. If something is still broken afterwards, you would have to scroll through large error messages in your terminal. In a modern IDE you have your debugger attached that tells you (in most cases) exactly what is going on and where the error lies. Nowadays, it can be said that only real purists strictly rely on their VI editor and Unix terminal only.
PyCharm
PyCharm is a rather complex IDE developed by the Czech company JetBrains for the programming language Python. In this project we were able to use the professional version thanks to the university’s license. The professional version offers additional support for languages used in web development like HTML or JavaScript, the database language SQL, and most importantly for the Python-based web server framework Django, which we used to develop SmartUni. PyCharm is quite popular in the Python community and probably the most widely spread IDE for that language.
Fig. 4: PyCharm logo.
Besides the quite common features the IDE offers like smart code navigation, or automated refactoring, it offers a graphical user interface (GUI) for managing different software versions retrieved from the VCS Git (next section). Oftentimes in such a large project, you encounter complicated merge conflicts between versions that have been developed by different people. It would probably be quite a pain to resolve these without any graphical support. In PyCharm’s graphical merge tool you can easily compare the different versions side by side and accept or reject a change with just one click. Further, PyCharm comes with a huge set of plugins you can install for optimizing your personal coding experience even more. These plugins vary from graphical shenanigans, over language support, or the integration of the controls of classical editors like VI, to the integration of the intelligent code completion tool GitHub Copilot (if you have access).
Most of our developers used PyCharm, and it made the whole development process much easier and more comfortable.
Git
Git is a software tool for distributed version control. That makes Git a version control system (VCS). In distributed version control, a codebase includes its entire history of changes and that history is copied alongside each copy of the codebase on each developer’s machine. That means you can switch back and forth between all different versions the codebase ever had. You can also split the codebase into different branches which can be worked on independently alongside each other. Nowadays, such tools are essential for efficient software development in larger or even in smaller teams, as everyone can work asynchronously on their own work packages, and once they are done, everything can be more or less easily merged and deployed.
Fig. 5: Git logo.
Git is currently the most popular VCS. Being able to use it is an essential skill in modern software development, because so many projects rely on it. What is so special about Git compared to other VCSs is simply its outstanding speed in which the operations are being executed, while having rather small memory usage. Git performs almost every operation locally. As a result, only the initial clone operation takes much time, because the entire history is downloaded. Afterwards, most operations don’t go through the bottleneck, the internet connection, anymore. Further, Git stores just sets of changes, not vast amounts of copies of the entire codebase. Doing so drastically reduces the amount of memory used by Git.
Git by itself is used from a Unix command line, but there are many applications that provide a GUI for Git including visualizations of the branches and how they are connected, or graphical merge tools. One of those is PyCharm which we used anyway.
For being able to manage the different versions between different machines that are not locally connected, Git has to be hosted on a server. For that two platforms are most common: GitHub and GitLab. We used GitLab that comes with many more features which we talk about in a separate section. Git is so essential that we built our entire development workflow around it. Therefore, everyone who wanted to provide code had to learn how to use it. While the main commands git add, git commit, and git push are quite easy to understand, there are more and more quirks you run into when using it more intensely. But because it is so famous, we could rely on some experts in our own team and on Git’s huge online community feeding the forums. Using Git helped us a lot, and we can’t imagine how the project would have gone without it.
Open Source Licensing
A license is an official permit to do, use, or own something (as well as the document of that permit). For software, there are 5 types of licenses that can be classified into 2 general categories: free and open licenses or non-free licenses. For non-free licenses, the source code of the software must not be provided, while for the free and open licenses, the source code must always be provided.
Since SmartUni is an open-source project, we will concentrate on free and open types of licenses, also known as open-source licenses. Open-source licenses allow the source code, blueprint, or design of a project or product to be used, modified, and shared under defined terms and conditions. These types of licenses are usually free of charge.
For our documentation site, we chose a Creative Commons BY-NC-SA 4.0 license. This license allows the licensee to share and adapt the code and contents of the documentation site. In detail, the licensee is allowed to copy and redistribute the material in any medium or format and to remix, transform, and build upon the material. This is allowed as long as the licensee gives appropriate credit by providing a link to the license and indicating if changes were made and not suggesting that the licensor endorses the licensee or their use of the material. In addition, the material may not be used for commercial purposes and in case of changes of the material (remix, transform, or extension of the material), these contributions must be distributed under the same license as the original.
The source code of our project is licensed under a GPLv3 license. This license is a so-called copyleft license. Copyleft licenses enforce the freedom of use and grant subsequent users the right to modify, distribute, and use the software as they please. GPLv3 allows licensees to use the code for commercial and non-commercial purposes and therefore imposes no conditions on the internal use of the software. It also allows changes of the code. Licensees can change or rework the code and even distribute copies or modifications of the code if these modifications are also released under the GPLv3 license. All of this is permitted as long as the modifications and redistributions of the code are open-sourced as well. A person using our code in their own application cannot then close those changes off from the public. This also means that they cannot use our code in closed-off proprietary systems. If you include source code from a GPLv3 licensed project in your code base, this code base has to be open-sourced under a GPLv3 license as well.