Last week moved quickly for me in ListenBrainz. I submitted multiple pull requests and participated in the weekly developer’s meeting on Monday. I was also invited to take part as a mentor for ListenBrainz for the upcoming round of Google Code-In! In addition to my changes and new role as a mentor, I’m researching libraries like D3.js to help build visualizations for music data.  Suddenly, everything started moving fast!

Last week: Recap 🔗

The ListenBrainz team accepted my development environment improvements and documentation. This gave me an opportunity to better explore project documentation tools. I experimented with Sphinx and Read the Docs. Sphinx introduced me to reStructuredText for documentation formats. I’ve avoided it in favor of Markdown for a long time, but I see where reStructuredText is stronger for advanced documentation.

Since ListenBrainz is a new project, I plan to contribute documentation for any of my work and improve documentation for pre-existing work. One of the goals for this independent study is to make ListenBrainz a viable candidate for a future data analysis course. To make it easy to use and understand, ListenBrainz needs excellent documentation. Since one of my strengths is technical writing, I plan to contribute more documentation this semester.

You can see some of the new documentation already!

Google Code-In mentor 🔗

The MetaBrainz community manager, Freso Olesen, approached me to mentor for Google Code-In. Google Code-In is an opportunity for teenagers to meaningfully contribute to open source projects. Google describes Google Code-In as…

Pre-university students ages 13 to 17 are invited to take part in Google Code-in: Our global, online contest introducing teenagers to the world of open source development. With a wide variety of bite-sized tasks, it’s easy for beginners to jump in and get started no matter what skills they have.

Mentors from our participating organizations lend a helping hand as participants learn what it’s like to work on an open source project. Participants get to work on real software and win prizes from t-shirts to a trip to Google HQ!

MetaBrainz is a participating organization of Google Code-In this cycle. Because of my work with ListenBrainz, I will contribute a few hours a week to help mentor participating students with ListenBrainz. Beginner problems should be easy to help with since I’m still beginning too, and as I spend more time with ListenBrainz, I can help with harder problems.

I’m excited to give back to one of my favorite open source projects in this way! I’m grateful to have this chance to help out during Google Code-In.

Choosing easyfix bugs 🔗

After I figured out the development environment issues, I went through open tickets filed against ListenBrainz to find some to work on. I made a preliminary pass through all open tickets and left some comments for more information, when needed. The tickets I highlighted to look into next were

• LB-85: Username in the profile URL should be case insensitive
• LB-124: Install messybrainz as a a python library from requirements
• LB-176: Add stats module and begin calculating some user stats from BigQuery
• LB-206: “playing_now” submissions not showing on profile
• LB-212: Show the MetaBrainz logo on the listenbrainz footer.

Of these five, LB-124 and LB-212 are already closed. While drafting this article, I completed LB-124 in PR #266. This was part of a test to get the documentation building again because of odd import errors. Later, a new student also learning the project for the first time asked to work on LB-212. Since it was a good first task to explore the project code, I passed the ticket to him.

I want to do one more “easyfix” bug before going into the main part of my independent study timeline. I don’t yet feel comfortable with the code and one more bug solved will help. After this, I plan to pursue the heavier lifting of the independent study to explore data operations and queries to make.

Researching D3.js 🔗

Prof. Roberts introduced D3.js as a library to build interactive, dynamic charts and visual representations of data. I haven’t yet looked into much front-end work, but this was a cool project that I wanted to highlight in my weekly report. This feels like it could be a powerful match for ListenBrainz, especially since the data has high detail.

Upcoming activity 🔗

This next week, I won’t have as much time to contribute to ListenBrainz. On October 21, I’m traveling to Raleigh, NC for All Things Open. On October 24, I present my talk, “What open source and J.K. Rowling have in common”. Since I’ll be out of Rochester and missing other classwork, I expect less time on my ListenBrainz work.

This next week will be slower than the last two weeks. Hopefully I’ll learn something at the conference too to bring back for ListenBrainz.

Until then… keep the FOSS flag high.

One of the first rites of passage when working on a new project is creating your development environment. It always seems simple, but sometimes there are bumps along the way. The first activity I did to begin contributing to ListenBrainz was create my development environment. I wasn’t successful with the documentation in the README, so I had to play around and work with the project before I was even running it.

The first part of this post details how to set up your own development environment. Then, the second half talks about the solution I came up with and my first contribution back to the project.

Install dependencies: Docker 🔗

This tutorial assumes you are using a Linux distribution. If you’re using a different operating system, install the necessary dependencies or packages with your preferred method.

ListenBrainz ships in Docker containers, which helps create your development environment and later deploy the application. Therefore, to work on the project, you need to install Docker and use containers for building the project. Containers save you from installing all of this on your own workstation! Since I’m using Fedora, I run this command.

sudo dnf install docker docker-compose

Register a MusicBrainz application 🔗

Next, you need to register your application and get a OAuth token from MusicBrainz. Using the OAuth token lets you sign into your development environment with your MusicBrainz account. Then, you can import your plays from somewhere else.

To register, visit the MusicBrainz applications page. There, look for the option to register your application. Fill out the form with these three options.

• Name: (any name you want and will recognize, I used listenbrainz-server-devel)
• Type: Web Application
• Callback URL: http://localhost/login/musicbrainz/post

After entering this information, you’ll have a OAuth client ID and OAuth client secret. You’ll use these for configuring ListenBrainz.

Update config.py 🔗

With your new client ID and secret, update the ListenBrainz configuration file. If this is your first time configuring ListenBrainz, copy the sample to a live configuration.

cp listenbrainz/config.py.sample listenbrainz/config.py

Next, open the file with your favorite text editor and look for this section.

# MusicBrainz OAuth
MUSICBRAINZ_CLIENT_ID = "CLIENT_ID"
MUSICBRAINZ_CLIENT_SECRET = "CLIENT_SECRET"

Update the strings with your client ID and secret. After doing this, your ListenBrainz development environment is able to authenticate and log in from your MusicBrainz login.

Initialize ListenBrainz databases 🔗

Your development environment needs some databases present to work. Before proceeding, run these three commands to initialize the databases.

docker-compose -f docker/docker-compose.yml -p listenbrainz run --rm web python3 manage.py init_db --create-db
docker-compose -f docker/docker-compose.yml -p listenbrainz run --rm web python3 manage.py init_msb_db --create-db
docker-compose -f docker/docker-compose.yml -p listenbrainz run --rm web python3 manage.py init_influx

Your development environment is now ready. Now, let’s actually see ListenBrainz load locally!

Run the magic script 🔗

Once you have done this, run the develop.sh script in the root of the repository. Using docker-compose, the script creates multiple Docker containers for the different services and parts of the ListenBrainz server. Running this script will start Redis, PostgreSQL, InfluxDB, and web server containers, to name a few. But this also makes it easy to stop them all later.

./develop.sh

You will see the containers build and eventually run. Leave the script running to see your development environment. Later, you can shut it down by pressing CTRL^C. Once everything is running, visit your new site from your browser!

http://localhost/

Now, you are all set to begin making changes and testing them in your development environment!

Making my first pull request 🔗

As mentioned earlier, my first attempt at a development environment was unsuccessful. My system kept denying permission to the processes in the containers. After looking at system audit logs and running a temporary setenforce 0, I tried the script one more time. Everything suddenly worked! So the issue was mostly with SELinux.

With my goal to get my environment set up, I figured out a few issues with the configuration offered by the project developers. I eventually made PR #257 against listenbrainz-server with my improvements.

Labeling SELinux volume mounts 🔗

To diagnose the issue, I started with a quick search and found a StackOverflow question with my same problem. There, the question was about Docker containers and denied permissions in the container. The answers explained it was an SELinux error and the context for the containers was not set. However, temporarily changing context for a directory didn’t seem too effective and doesn’t persist across reboots.

Continuing the search, I found an issue filed against docker-compose about the :z and :Z flags for volume mounts. These flags set SELinux context for containers, with the best explanation I found coming from this StackOverflow answer.

Two suffixes :z or :Z can be added to the volume mount. These suffixes tell Docker to relabel file objects on the shared volumes. The ‘z’ option tells Docker that the volume content will be shared between containers. Docker will label the content with a shared content label. Shared volumes labels allow all containers to read/write content. The ‘Z’ option tells Docker to label the content with a private unshared label.

Therefore, I added the :z flag to all the volume mounts in the docker-compose.yml file. I submitted a fix upstream for this in listenbrainz-server#257!

Correct the startup port 🔗

In the README, it says the server will start on port 8000, but the docker-compose.yml file actually started the server on port 80. I included a fix for this in my pull request as well.

git push! 🔗

This post makes a debugging experience that actually took hours look like it happened in minutes. But after getting over this hurdle, it was awesome to finally see ListenBrainz running locally on my workstation. It was an even better feeling when I could take my improvements and send them back in a pull request to ListenBrainz. Hopefully this will make it easier for others to create their own development environments and start hacking!