Making graphs is easy. Making intuitive, easy-to-understand graphs? It’s harder than most people think. At the Rochester Institute of Technology, the ISTE-260 (Designing the User Experience) course teaches the language of design to IT students. For an introductory exercise in the class, students are tasked to visualize any set of data they desire. Students David Kim, Jathan Anandham, Justin Wheeler, and Scott Tinker used the MusicBrainz database to look at how five different Queen songs went mainstream in different ways.

Five factors of Queen 🔗

Our mini data science experiment decided to look at five unique data points available to us via MusicBrainz Works:

• Number of recorded covers
• Number of artists who covered a song
• Release year
• Year of last recorded cover
• Time elapsed between release year and year of last recorded cover

Originally, we looked at songs from different artists, but decided to look at five recordings from the same artist. With Queen being a notoriously famous band, there were several data points to work with in terms of how often a song was covered.

Making sense of the data 🔗

A few explanations are necessary for some of the data, especially the difference in number of covers and number of artists. Don’t Stop Me Now, Fat Bottomed Girls, and We Will Rock You had the same number of recorded covers as number of artists who have covered the song. Why were Another One Bites the Dust and Bohemian Rhapsody different?

As it turns out, Another One Bites the Dust had more covers than the number of artists who have covered the song. This happens because some artists have covered the song twice (e.g. once on a studio release and another on a live recording release). On the other hand, Bohemian Rhapsody had more artists covering it than number of covers because some recordings featured multiple artists on the same cover (e.g. the 1992 live performance with Elton John and Axl Rose).

The data opens many interesting questions. Why have some songs persisted longer than others (in terms of recent covers)? Have these songs impacted culture and society in different ways? How have they permeated culture? Is there geographical bias in the data?

This exercise was an exploratory assignment, but we had fun visualizing it and ended up learning an interesting pattern in music data.

Check out the presentation and paper 🔗

If you’re interested for the full details, the slides and a short paper about the presentation are available online. They provide deeper context for the research and the visualization details based on different design concepts.

You can see what else David Kim, Jathan Anandham, Justin Wheeler, and Scott Tinker are up to on LinkedIn. Thanks for tuning in to this adventure into music data analysis, powered by MusicBrainz!

Photo by Matthias Wagner on Unsplash.

Home is a suitcase 🔗

I began the third year of my degree and moved for the fifth time in two years when I made it back to Rochester in August. This time, I found somewhere to ideally live longer than only a few months of the year. I moved into a house with a few other roommates with more space than I’ve had before. For the first time in a while, it’s somewhere I’ve made to feel like home.

This move came months after I ended a semester of a study abroad program and lived in a city for an internship. Most of 2017 made my suitcase feel like a home, but it afforded many unique experiences.

Croatia: Study abroad 🔗

From January to May 2017, I participated in a study abroad program with my university to Dubrovnik, Croatia. RIT has full campuses in both Zagreb and Dubrovnik. This made planning the semester abroad easy, but also encouraged me to go somewhere I might not have gone otherwise.

My choice to study in Croatia was well-rewarded. On paper, I earned 12 credit hours, but I took away more than what I learned in class. My most important lessons came in the form of midnight bus rides to Albania, photograph exhibits capturing genocide in Sarajevo, and hugs from normally faraway friends in Czechia. My time abroad began a process in finding myself that has continued since my time in Europe.

DevConf 2017 / Fedora Diversity FAD 🔗

At the beginning of the year, the Fedora Diversity Team held a “Fedora Activity Day” (FAD) event in Brno, Czechia. If you’re outside of the Fedora community, think of a FAD as a focused, in-person team sprint. Together with our team in-person and remote, we mapped out our goals and plans for 2017 and set out to continue the work we began nearly three years ago.

In addition to the work we accomplished together, it was fulfilling for me to see my teammates that span three continents. I spent a week with not only my teammates but also my friends. The days we get to spend together are a privileged few in the year, and it was fulfilling and motivating for me to spend some of our time together in a way that wasn’t Pagure tickets or IRC meetings.

Read more about our team sprint in this event report:

https://communityblog.fedoraproject.org/fedora-diversity-fad-2017/

FOSDEM 2017 🔗

In February, I attended the Free and Open Source Software Developers European Meeting (FOSDEM) for the first time. FOSDEM is the largest open source conference in Europe, bringing together over 8,000 open source enthusiasts, contributors, and leaders from around the globe.

I had the privilege to attend as a member of the Fedora community, so my time was between the Fedora booth to meet the community and catching interesting talks. I also gave a talk of my own on the main track, What open source and J.K. Rowling have in common! I gave this talk to a smaller audience at DevConf, but the FOSDEM audience was considerably larger.

In retrospect, my original talk topic is relevant but I have ideas on how I could have delivered my message more effectively. Regardless, it was a learning experience for me to present in front of a new audience. Public speaking opportunities filled my youth, both in theater and in presentations, but I had never presented to a technical audience before (let alone on a non-technical topic). The experience at FOSDEM helped build my understanding and I hope to return with a new topic someday in the future.

Exploring the Balkans 🔗

Outside of open source and Fedora, my time in Croatia included a lot of time outside of Croatia. When many of my roommates went to explore the wonders of Western Europe, I lost my heart in the shadows of the Balkan mountains. My spring break was a solo trip split between Sarajevo, Bosnia and Hercegovina and Tirana, Albania.

Sarajevo 🔗

The three days I spent in Sarajevo were short but significant. I was truly alone on this visit and it was up to me to make the most of it. Originally, I was skeptical to go alone, but I knew that I would never have a better opportunity to go. My fascination with Sarajevo stemmed from a year of studying European history in high school, and knowing the cultural significance of Sarajevo as a meeting point of western and eastern cultures. In the end, I decided to go, and I was rewarded for it.

Most of my trip in Sarajevo consisted of museums. I visited various museums, ranging from eighteenth to twentieth century history. The most rewarding for me were the Galerija 11/07/95 and the Tunnel of Sarajevo. The gallery documented the Srebrenica genocide in July 1995 by the Serbian armed forces. The exhibit was eye-opening and perspective-shifting. The Tunnel of Sarajevo, sometimes called the Tunnel of Hope, is another perspective-shattering experience. The museum introduces the tunnel used during the siege of Sarajevo during the 1990s, when Serbian forces surrounded the city for an almost four-year siege. The tunnel was the only way for citizens and resistance forces to contact the outside world and keep the resistance alive. A small part of the tunnel is preserved, and the other artifacts make it a gripping experience (not to mention it’s a short drive out of the city, so you also have a chance to mentally prepare and later unpack the experience).

Tirana 🔗

I visited Tirana, Albania four times on my trip abroad. In Tirana, my heart was captured by the people there. For years, I read about the Open Labs Hackerspace community based in Tirana and I always imagined an opportunity to see it in person. I actually remember my first encounter with their community was an Ada Lovelace Day event report. And somehow, the circumstances shifted where I was able to meet their community and immerse myself in the culture, if only for a short time.

During my times in Tirana, I participated in the first-ever 48 hour hackathon to support the UN’s Sustainable Development Goals (SDGs), the first edition of Linux Weekend, and the annual Open Source Conference Albania (OSCAL).

https://fedoramagazine.org/students-fedora-linux-weekend-2017/

India 🔗

At the end of my study abroad experience in Croatia, a unique opportunity presented itself to me. I did not buy my return airfare back to the US before I left for Croatia. When price-checking for my trip back, I noticed it was a few hundred dollars extra if I decided to spend a week in India before flying back to the US.

I booked the tickets.

The last day of my classes finally came, and the next day, I was traveling further east, to Mumbai (or Bombay, if you prefer). I had the great fortune of having two great friends who invited me to the homes of their families during my trip. I visited Bee in Mumbai and Amita in Pune, all split across a single week!

My trip to India was eye-opening. For years, I’ve had a fascination with Eastern culture and philosophy, but it was something completely different to experience. Bee and her family took me to the Global Vipassana Pagoda, a personally fulfilling experience for me. We visited the Bandra–Worli Sea Link, Shree Mahalakshmi Temple, and several other places in Mumbai. I remember walking through the streets more than anything.

In Pune, Amita and her family showed me their favorite places. I had a chance to meet many other Fedora friends in Pune too. One of my favorite memories of Pune was a historic gurdwara. Amita took me and the others in our group to visit. For a moment, I finally got to see something I’ve only read about right in front of my eyes. The history and reverence in these places was absorbed into my mind.

I never expected an Indian visa stamp in my passport in 2017, yet it happened. I’m equally filled with wonder at how the circumstances unfolded as I am grateful this experience sneaked into my year.

Chicago: Urban experience 🔗

After my semester abroad and visiting India, I was whisked back to the United States, only to pack up once again for another new experience. From June to August, I lived in Chicago, Illinois to work an internship at Jump Trading. Chicago had a feeling of nostalgia for me because much of my father’s family has origins tracing back to Chicago. But I would find myself losing more of my heart in Chicago than I realized.

The internship 🔗

I worked with a fantastic team of people on exciting projects. Professionally, my time in Chicago was motivating and empowering. I was provided the opportunity to learn and also contribute. I walked in with a dreadful feeling of imposter syndrome and left feeling more confident in my own learning abilities. Docker, Kubernetes, and time-series data became a part of my daily work life, when I had little to no knowledge before then.

By the time my internship finished, I helped contribute to our team’s goal of standing up Kubernetes and contributing a few patches in Kubernetes projects like Minikube. I have great mentors to thank for not only direct, technical assistance but also motivational mentorship and empowerment too.

Everything else 🔗

There was more to Chicago than only the work too. Before long, I felt like a true Chicagoan, traveling the subways into the Loop, catching free concerts in Millennium Park, and indulging in the Chicago tradition of deep-dish pizza.

Unlike other cities I’ve visited, like New York City, Chicago felt easier to integrate into. The culture was notably “slower” than the fast-pace life of NYC, London, or Washington DC. I discovered Middle Eastern markets that became a regular part of my weekends, made friends with the baristas at a local coffeehouse, and had the privilege of hosting friends from three continents for short stays.

I left Chicago and was offered a new contract for the following summer in 2018. I’m looking forward to be back in June again.

Year of Fedora 🔗

2017 was full of time and effort spent in the Fedora community. In addition to the Diversity FAD, I was elected to the Fedora Council (on my third attempt), attended the annual Fedora contributor conference, Flock, and also narrowed my scope for contributions.

When I began contributing to Fedora, I was contributing to many things. Marketing, community operations, Fedora Badges, Fedora Magazine, Ambassadors, Games SIG, Join SIG, the Diversity Team, and maybe a few more things. After a while, I realized my contributions carried great width but poor depth. In 2017, I “reconfigured” my time in Fedora to focus in on the areas where I felt my time yielded the highest impact. This is Fedora CommOps and the Diversity Team.

I resigned as Fedora Magazine editor-in-chief and also formally stepped down from other teams. It made me sad, but I knew it was the right decision for me. I’m happy to spend more time working in fewer projects at a greater depth and focus than I had before.

Flock 2017 🔗

Flock, Fedora’s annual contributor conference, was held from Aug. 29 to Sep. 1 in Cape Cod, Massachusetts. Every year, Flock is an empowering experience for me because of the face-time I get with the people I spend much of my year working with remotely. This year was no different, and many new faces were mixed in with the old ones.

The highlights for me were in three forms: the CommOps session, the Diversity Team session, and the Fedora Magazine session. Together with Sachin Kamath, we led the CommOps session. You can read more about our session here:

https://communityblog.fedoraproject.org/metrics-docs-flock-2017/

The Diversity Team and Magazine sessions were also valuable for both teams to get feedback from the rest of the community. In the Diversity Team session, we had many active participants outside of our team that reminded us the importance of narrowing our focus for higher impact. I also attended other interesting sessions held by the community, like the future of fedmsg by Jeremy Cline.

CommOps FAD 🔗

Towards the end of 2017, I worked together with our team in CommOps to organize our own team sprint, or FAD, in 2018. We successfully planned the event and organized it in Brno, Czechia, similar to last year’s Diversity FAD.

More details on this will be found in its own event report!

ListenBrainz indie study 🔗

In my fall semester of 2017, I took on an independent study to further explore the ListenBrainz project. ListenBrainz is an open source social platform to document the music you listen to over time. If you’re familiar with Last.fm or Libre.fm, it’s a similar concept, but the focus is more on the data than the social features. ListenBrainz is supported by the MetaBrainz Foundation, also the guiding body for the more well-known MusicBrainz project.

In my independent study, I had a chance to contribute documentation and community tools (like issue / PR templates), as well as explore how the project gathers and builds metrics. I didn’t make my original milestone of major code contributions to the project, but I better understood the community and tried to help in the areas of low coverage, like documentation.

The experience was insightful for me and provided me an excuse to work on something that I am genuinely passionate about. Music is a powerful part of human culture, and the MetaBrainz Foundation takes a serious approach to documenting music, especially in a technical sense. ListenBrainz represents an opportunity for us to better explore and understand ourselves through our music listening habits. I hope someday that ListenBrainz will be a platform for data journalism and research about music. That’s my dream.

Opensource.com community moderator 🔗

At the beginning of 2017, I was brought on board as an Opensource.com community moderator. Together with other community moderators and site staff, I help contribute new content and source new writers to the site. My invitation to the community moderator team came shortly after the announcement that I received the 2017 People’s Choice Award. When Rikki Endsley invited me to the team, it felt like a natural alignment to my passion for storytelling.

All Things Open 2017 🔗

I was invited to All Things Open, an annual open source conference in Raleigh, by the Opensource.com team. The day before the conference, I met the rest of the team and other community moderators at the Red Hat HQ in Raleigh. We spent the day locked into a room together to hash out plans and goals for the next year. It was a productive opportunity for the team to work together and also a great opportunity to meet the other members of the community.

Some of my best takeaways from this experience were catching coffee with other community moderators, meeting Jim Whitehurst to talk about Opensource.com, and giving my talk, What open source and J.K. Rowling have in common, for the final time.

I hope I have the opportunity to go again next year to meet the awesome team behind Opensource.com. (If you haven’t considered before, come and write for us too!)

Happiness Packet Challenge 🔗

Another unusual milestone for my 2017 was the first rendition of the Happiness Packet Challenge. I was introduced to the Happiness Packets website in 2016. Happiness Packets are an easy way to say thank you to someone who has had a positive impact on you. I came up with a challenge to my friends and network to write one Happiness Packet a day, every day, for a week.

I followed up with the team behind the project to evaluate the impact of this idea, and I was pleasantly surprised. Here’s the number of messages sent for the two weeks prior to the Happiness Packet Challenge, followed by the week of the challenge:

• Week starting 2017-03-27: 2 sent
• Week starting 2017-04-03: 35 sent
• Week starting 2017-04-10 (challenge week): 72 sent

You can read more about the challenge in my original blog post. Keep an eye out for it again in 2018.

Living openly 🔗

Earlier in this post, I alluded to how I felt like I began to find myself when I was abroad. My study abroad experience was the beginning of a longer process that leads into present day.

In April, I went public with my depression, both to help take a weight off my shoulder and to be a voice for others who are afraid to speak up. I was always concerned of the reaction from publishing something like that, but I was met with nothing but loving-kindness from friends and strangers. It gave me new confidence to live more openly and wear my values in the open.

The story continued in October, when I decided to delete my Facebook and Instagram accounts.

https://medium.com/@jflory7/cut-the-plug-deleting-facebook-and-instagram-6cbe7c86d9c9

I considered this for a couple of years before, but I pulled the trigger in October. Like many others, it felt almost too much of a task to disconnect myself from this huge network of people and friends. But the negative impacts of it were draining me and trapping me. Since I deleted my accounts, I’ve noticed a positive impact in overall levels of happiness and awareness. However, I don’t think the social media accounts alone are the reason for this.

In the near future, I hope to do a follow-up post to my decision to cut away from the Facebook and Instagram machines. Keep an eye out for more.

2018 🔗

It’s already February in 2018 when I finished this post. This year, I thought it would be the year when I get the post out closer to the new year, but somehow I always slip. In either case, it gives me a chance to take in some of the new opportunities and excitement of the new year before reflecting and looking back.

This year, I’m working an internship with UNICEF to help lead on open source community engagement and supporting the non-technical areas of their MagicBox platform. In the one month I’ve been doing this, I feel like I have tens of articles I could write about, but the experience is still maturing for me.

I also have another round in Chicago to look forward to over the summer. I’ll get to work with the same team as last year on similar projects, and I’m looking forward to going back.

As for the rest, who knows what’s to come? So many things that made 2017 what it was were the things I didn’t expect. The surprises in life are the salt to the regiment of daily life, and add flavor and spice in unexpected ways. I have no idea what my 2018 Year in Review will look like, and that’s okay. I’m looking forward to seeing what will make it in.

Thank you 🔗

Above all, every year, I think back on the people who positively impacted my life and contributed to the “flavor” of my year. A close friend reminded me recently that we all stand on the shoulders of giants. And isn’t it true? We all have our great mentors, great friends, and unexpected sages that help us find our own footing on this great path of life. We become ourselves from the various pieces impacted on us by others.

I’m thankful for all of the people who have made my year into the experience it was. The list is too long to write and I fear I would leave someone out – even significant impacts were made by people who had a short-term role in this last year.

A long time ago, my open source experience was jump-started by someone who did something kind and exceptional for me. It was a continuing trend since that moment. My only aspiration is to pay forward the good will that so many have bestowed unto me.

Thanks for making it this far down, and I hope to see you in 2018. Or who knows – maybe it will just be me reading this far down for next year, when I go to write my next year in review. Hi future me!

This week is the last week of the fall 2017 semester at RIT. This semester, I spent time with the MetaBrainz community working on ListenBrainz for an independent study. This post explains what I was working on in the last month and reflects back on my original objectives for the independent study.

Running my own ListenBrainz 🔗

The RIT Linux Users Group hosts various virtual machines for our projects. I requested one to set up and host a “production” ListenBrainz site. The purpose of doing this was to…

1. Test my changes in a “production” environment
2. Offer a service for the RIT Linux Users Group to poke around with

I spent most of this time working with our system administrator to set up the machine and adjust hardware specs for ListenBrainz. Once we fixed storage space and memory issues, it was easy to set it up and get ListenBrainz running. My experience writing the development guide made it easy to get set up and get working. On the first run, it worked!

Now, listen.ritlug.com is live.

Figuring out HTTPS 🔗

My next challenge for the site is to set up HTTPS. I tried using a reverse proxy in nginx to set up HTTPS, but I received 502 Bad Gateway errors. I realized I spent too much time figuring this out on my own and decided to ask for help in the MetaBrainz community forums.

Proposing new statistics 🔗

Halfway through the independent study, I realized I would fall short of my original objective of implementing basic statistics in ListenBrainz. To compromise, I wrote a proposal for new statistics to start in the project. My proposal looked at other proprietary platforms that compete with ListenBrainz to see some of their statistics. I also came up with some of my own.

I proposed this to the MetaBrainz community on the community forums. I’m awaiting feedback on my ideas. Once I get feedback, I plan to file new tickets for each statistic to track their implementation over time.

I don’t expect statistics being at the forefront of ListenBrainz for some time. A lot of work is going towards other areas of the project. But later in 2018, I expect more focus on the user-facing side of the project.

My statistic and Google BigQuery 🔗

My biggest blocker over the last month was Google BigQuery. I wrote a statistic to calculate play counts over a time period, but was asked to test my statistic. To test my statistic, I needed real data to work with.

Originally, I tried using the Simple Last.fm Scrobbler to submit listens to the local IP address for my development environment, but I wasn’t able to get the app to reach my ListenBrainz server. To get the data, I had to set up Google BigQuery credentials so I could make queries against data on the production site, listenbrainz.org.

I tried working through the Google BigQuery documentation. There’s a lot of documentation for using BigQuery as a developer, but it was confusing where to find the information I needed to set it up in my development environment. I tried creating a new project in the Google Cloud Platform, but I was confused because it prompted me to upload my own data instead of accessing data already in BigQuery.

Too late, I realized I spent too much time on my own and not asking for help. I submitted a pull request with the statistic I made and asked for help in the MetaBrainz community. I also offered to write documentation for setting this up once I learn how to do it.

Reflecting back 🔗

I looked back on my original objectives for the independent study, and I was satisfied and dissatisfied.

Not enough programming 🔗

I wanted this independent study to enhance my programming knowledge. I especially wanted to focus on Python because I wanted to become more familiar with the language. However, I actually didn’t do much programming during the independent study, to my own fault.

My biggest challenge was I bit off more than I could chew. I wanted to write code, and made a big goal before I knew the code base of the project. Even now, I still am not completely comfortable with the code yet. It’s a big project with a lot of things going on. I was able to understand the things I did work on, but there’s still a lot.

I realized that next time, I need to spend more time evaluating the code base of a project before writing out my milestones. I wish I set more realistic, smaller milestones for myself. My milestone of implementing basic reports was lofty given my existing programming knowledge.

Successes 🔗

One of my other objectives was to write documentation for the project. I felt I succeeded in this milestone, and actually found it enjoyable and interesting to do! I helped separate out documentation from the README into the dedicated ReadTheDocs site. I wrote the development environment guide and helped fix some build issues with the docs site. I also plan to write more for some of the other pain points I found, like Google BigQuery.

My last milestone was to create a use case for a data visualization course at RIT. While I didn’t implement my basic reports, I did create the proposal and make an effort to write new statistics. There’s a lot of potential now to work with the data in Google BigQuery and do front-end work with tools like D3.js and Plotly.js. I believe there’s significant potential to use ListenBrainz as a hands-on project for students to explore data visualization with real data. I hope to support my independent study professor, Prof. Roberts, with questions and logistics of using it as a tool for learning in the future.

Unexpected success 🔗

I also think I had an unplanned success too. I immersed myself in the community for ListenBrainz too. Over the last few months, I realized that many of my strengths are in community management and tooling. During my time in the community, I did the following:

• Fixed SELinux labels in Docker
• Contributed a pull request template
• Drafted contributing guidelines
• Fixed a PostgreSQL bug
• And more…

To the future! 🔗

This ends my independent study with ListenBrainz, but it doesn’t end my time contributing! I chose ListenBrainz because it’s a project I’m passionate about. An independent study allowed me to justify more time on it than a side project in my free time. I’m happy to have that opportunity, but I don’t want to end here!

I want to follow through on the statistics because I’m passionate about understanding music listening trends. I think there’s a lot of power for psychological research through music data. To this point, I filed a ticket to request tagging listens with “emotion” words that are synced back to MusicBrainz entities.

I won’t have as much time to work on the project without the course credit, but I hope to stay involved for the future. I love the project and I love the community. I’m thankful for the opportunity to work on this project as an independent study, and learn some things along the way.

My progress with ListenBrainz slowed, but I am resuming the pace of contributing and advancing on my independent study timeline. This past week, I finished out assigned tasks to discuss contributor-related documentation, like a Code of Conduct, contributor guidelines, and a pull request template. I began research on user statistics and found some already created. I wrote one of my own, but need to learn more about Google BigQuery to advance further.

Paving the contributor pathway 🔗

Earlier, I identified weaknesses for the ListenBrainz contributor pathway and found ways we could improve the pathway. This started with the development environment documentation. Now, I helped draft first revisions of our contributor guidelines, Code of Conduct reference, and pull request templates. Together, these three documents have two goals.

1. Make it easier to contribute to ListenBrainz
2. Have a better experience and have fun contributing!

Adding these documents addresses these goals. Additionally, the GitHub community profile also highlights these deliverables as ways to meet these goals. After getting feedback and seeing what others think, we make more revisions later (with some trial runs).

Back to SELinux context flags 🔗

Recently, I set my desktop back up and installed Docker for the first time on this machine; however, the development environment still failed to start. When I ran the script, it would eventually error out because of a permission denial. The web server image for ListenBrainz was failing.

After debugging, I noticed that I missed the SELinux volume tags for the ListenBrainz web server images in my original pull request, #257. When I created the pull request, I might have had cached data that let my laptop run the development environment without a problem. In either case, it was an easy fix and I knew what the issue was when it happened. Therefore, I submitted a new fix in #290.

Writing new user statistics 🔗

The most interesting part of my independent study is working with the music data to build and generate interesting statistics. I finally began exploring the existing statistics in ListenBrainz. The statistic queries use BigQuery standard SQL. BigQuery helps rapidly scan and scale data queries to help with performance (I have a lot to learn about BigQuery).

Two types of statistics 🔗

Additionally, ListenBrainz generates two types of statistics:

1. Site-wide statistics
2. User statistics

Site-wide statistics are metrics non-specific to a single user. There is only one site-wide query now. It counts how many artists were ever submitted to this ListenBrainz instance and returns an integer. There’s room for expansion in site-wide statistics.

On the other hand, user statistics are metrics specific to a single user. There’s a fair number already, like the top artists and songs in a time period and the number of artists you’ve listened to. These are a little more complete and offer more expansion for doing cool front-end work with something like D3.js.

Writing user statistics 🔗

Of course, I had to try writing my own. One helpful query I thought of was getting a count of the songs you listened to over a time period (e.g. “you listened to 500 songs this week!”). I haven’t tested it yet, but I have this in a local branch and hope to test it with real data soon.

def get_play_count(musicbrainz_id, time_interval=None): 
 
 filter_clause = "" 
 if time_interval: 
     filter_clause = "AND listened_at >=
     TIMESTAMP_SUB(CURRENT_TIME(), 
     INTERVAL {})".format(time_interval) 
 
 query = """SELECT COUNT(release_msid) as listen_count 
            FROM {dataset_id}.{table_id} 
            WHERE user_name = @musicbrainz_id 
            {time_filter_clause} 
            LIMIT {limit} 
         """.format( 
                 dataset_id=config.BIGQUERY_DATASET_ID, 
                 table_id=config.BIGQUERY_TABLE_ID, 
                 time_filter_clause=filter_clause, 
                 limit=config.STATS_ENTITY_LIMIT, 
            ) 
 
 parameters = [ 
     { 
         'type': 'STRING', 
         'name': 'musicbrainz_id', 
         'value': musicbrainz_id 
     } 
 ] 
 
 return stats.run_query(query, parameters)

Researching Google BigQuery 🔗

My next steps for the independent study are researching Google BigQuery. After going through the existing statistics and understanding how ListenBrainz generates them, an understanding of Google BigQuery is essential to writing effective queries. When I become more comfortable with the tooling and how it works, I want to map out a plan of statistics to generate and measure.

Until then, the hacking continues! As always, keep the FOSS flag high…

Last week moved quickly for me in ListenBrainz. I submitted multiple pull requests and participated in the weekly developer’s meeting on Monday. I was also invited to take part as a mentor for ListenBrainz for the upcoming round of Google Code-In! In addition to my changes and new role as a mentor, I’m researching libraries like D3.js to help build visualizations for music data.  Suddenly, everything started moving fast!

Last week: Recap 🔗

The ListenBrainz team accepted my development environment improvements and documentation. This gave me an opportunity to better explore project documentation tools. I experimented with Sphinx and Read the Docs. Sphinx introduced me to reStructuredText for documentation formats. I’ve avoided it in favor of Markdown for a long time, but I see where reStructuredText is stronger for advanced documentation.

Since ListenBrainz is a new project, I plan to contribute documentation for any of my work and improve documentation for pre-existing work. One of the goals for this independent study is to make ListenBrainz a viable candidate for a future data analysis course. To make it easy to use and understand, ListenBrainz needs excellent documentation. Since one of my strengths is technical writing, I plan to contribute more documentation this semester.

You can see some of the new documentation already!

Google Code-In mentor 🔗

The MetaBrainz community manager, Freso Olesen, approached me to mentor for Google Code-In. Google Code-In is an opportunity for teenagers to meaningfully contribute to open source projects. Google describes Google Code-In as…

Pre-university students ages 13 to 17 are invited to take part in Google Code-in: Our global, online contest introducing teenagers to the world of open source development. With a wide variety of bite-sized tasks, it’s easy for beginners to jump in and get started no matter what skills they have.

Mentors from our participating organizations lend a helping hand as participants learn what it’s like to work on an open source project. Participants get to work on real software and win prizes from t-shirts to a trip to Google HQ!

MetaBrainz is a participating organization of Google Code-In this cycle. Because of my work with ListenBrainz, I will contribute a few hours a week to help mentor participating students with ListenBrainz. Beginner problems should be easy to help with since I’m still beginning too, and as I spend more time with ListenBrainz, I can help with harder problems.

I’m excited to give back to one of my favorite open source projects in this way! I’m grateful to have this chance to help out during Google Code-In.

Choosing easyfix bugs 🔗

After I figured out the development environment issues, I went through open tickets filed against ListenBrainz to find some to work on. I made a preliminary pass through all open tickets and left some comments for more information, when needed. The tickets I highlighted to look into next were

• LB-85: Username in the profile URL should be case insensitive
• LB-124: Install messybrainz as a a python library from requirements
• LB-176: Add stats module and begin calculating some user stats from BigQuery
• LB-206: “playing_now” submissions not showing on profile
• LB-212: Show the MetaBrainz logo on the listenbrainz footer.

Of these five, LB-124 and LB-212 are already closed. While drafting this article, I completed LB-124 in PR #266. This was part of a test to get the documentation building again because of odd import errors. Later, a new student also learning the project for the first time asked to work on LB-212. Since it was a good first task to explore the project code, I passed the ticket to him.

I want to do one more “easyfix” bug before going into the main part of my independent study timeline. I don’t yet feel comfortable with the code and one more bug solved will help. After this, I plan to pursue the heavier lifting of the independent study to explore data operations and queries to make.

Researching D3.js 🔗

Prof. Roberts introduced D3.js as a library to build interactive, dynamic charts and visual representations of data. I haven’t yet looked into much front-end work, but this was a cool project that I wanted to highlight in my weekly report. This feels like it could be a powerful match for ListenBrainz, especially since the data has high detail.

Upcoming activity 🔗

This next week, I won’t have as much time to contribute to ListenBrainz. On October 21, I’m traveling to Raleigh, NC for All Things Open. On October 24, I present my talk, “What open source and J.K. Rowling have in common”. Since I’ll be out of Rochester and missing other classwork, I expect less time on my ListenBrainz work.

This next week will be slower than the last two weeks. Hopefully I’ll learn something at the conference too to bring back for ListenBrainz.

Until then… keep the FOSS flag high.

One of the first rites of passage when working on a new project is creating your development environment. It always seems simple, but sometimes there are bumps along the way. The first activity I did to begin contributing to ListenBrainz was create my development environment. I wasn’t successful with the documentation in the README, so I had to play around and work with the project before I was even running it.

The first part of this post details how to set up your own development environment. Then, the second half talks about the solution I came up with and my first contribution back to the project.

Install dependencies: Docker 🔗

This tutorial assumes you are using a Linux distribution. If you’re using a different operating system, install the necessary dependencies or packages with your preferred method.

ListenBrainz ships in Docker containers, which helps create your development environment and later deploy the application. Therefore, to work on the project, you need to install Docker and use containers for building the project. Containers save you from installing all of this on your own workstation! Since I’m using Fedora, I run this command.

sudo dnf install docker docker-compose

Register a MusicBrainz application 🔗

Next, you need to register your application and get a OAuth token from MusicBrainz. Using the OAuth token lets you sign into your development environment with your MusicBrainz account. Then, you can import your plays from somewhere else.

To register, visit the MusicBrainz applications page. There, look for the option to register your application. Fill out the form with these three options.

• Name: (any name you want and will recognize, I used listenbrainz-server-devel)
• Type: Web Application
• Callback URL: http://localhost/login/musicbrainz/post

After entering this information, you’ll have a OAuth client ID and OAuth client secret. You’ll use these for configuring ListenBrainz.

Update config.py 🔗

With your new client ID and secret, update the ListenBrainz configuration file. If this is your first time configuring ListenBrainz, copy the sample to a live configuration.

cp listenbrainz/config.py.sample listenbrainz/config.py

Next, open the file with your favorite text editor and look for this section.

# MusicBrainz OAuth
MUSICBRAINZ_CLIENT_ID = "CLIENT_ID"
MUSICBRAINZ_CLIENT_SECRET = "CLIENT_SECRET"

Update the strings with your client ID and secret. After doing this, your ListenBrainz development environment is able to authenticate and log in from your MusicBrainz login.

Initialize ListenBrainz databases 🔗

Your development environment needs some databases present to work. Before proceeding, run these three commands to initialize the databases.

docker-compose -f docker/docker-compose.yml -p listenbrainz run --rm web python3 manage.py init_db --create-db
docker-compose -f docker/docker-compose.yml -p listenbrainz run --rm web python3 manage.py init_msb_db --create-db
docker-compose -f docker/docker-compose.yml -p listenbrainz run --rm web python3 manage.py init_influx

Your development environment is now ready. Now, let’s actually see ListenBrainz load locally!

Run the magic script 🔗

Once you have done this, run the develop.sh script in the root of the repository. Using docker-compose, the script creates multiple Docker containers for the different services and parts of the ListenBrainz server. Running this script will start Redis, PostgreSQL, InfluxDB, and web server containers, to name a few. But this also makes it easy to stop them all later.

./develop.sh

You will see the containers build and eventually run. Leave the script running to see your development environment. Later, you can shut it down by pressing CTRL^C. Once everything is running, visit your new site from your browser!

http://localhost/

Now, you are all set to begin making changes and testing them in your development environment!

Making my first pull request 🔗

As mentioned earlier, my first attempt at a development environment was unsuccessful. My system kept denying permission to the processes in the containers. After looking at system audit logs and running a temporary setenforce 0, I tried the script one more time. Everything suddenly worked! So the issue was mostly with SELinux.

With my goal to get my environment set up, I figured out a few issues with the configuration offered by the project developers. I eventually made PR #257 against listenbrainz-server with my improvements.

Labeling SELinux volume mounts 🔗

To diagnose the issue, I started with a quick search and found a StackOverflow question with my same problem. There, the question was about Docker containers and denied permissions in the container. The answers explained it was an SELinux error and the context for the containers was not set. However, temporarily changing context for a directory didn’t seem too effective and doesn’t persist across reboots.

Continuing the search, I found an issue filed against docker-compose about the :z and :Z flags for volume mounts. These flags set SELinux context for containers, with the best explanation I found coming from this StackOverflow answer.

Two suffixes :z or :Z can be added to the volume mount. These suffixes tell Docker to relabel file objects on the shared volumes. The ‘z’ option tells Docker that the volume content will be shared between containers. Docker will label the content with a shared content label. Shared volumes labels allow all containers to read/write content. The ‘Z’ option tells Docker to label the content with a private unshared label.

Therefore, I added the :z flag to all the volume mounts in the docker-compose.yml file. I submitted a fix upstream for this in listenbrainz-server#257!

Correct the startup port 🔗

In the README, it says the server will start on port 8000, but the docker-compose.yml file actually started the server on port 80. I included a fix for this in my pull request as well.

git push! 🔗

This post makes a debugging experience that actually took hours look like it happened in minutes. But after getting over this hurdle, it was awesome to finally see ListenBrainz running locally on my workstation. It was an even better feeling when I could take my improvements and send them back in a pull request to ListenBrainz. Hopefully this will make it easier for others to create their own development environments and start hacking!

Many students take part in independent studies where they work on their own projects. However, in the spirit of open source collaboration, I wanted to contribute to a project that already existed. That way, my work would be helpful to a real-world project where it would have a value even after the end of the semester. Additionally, I wanted  a project to help me sharpen my Python skill. And ListenBrainz was a fun, exciting candidate for this.

Objectives 🔗

The independent study proposal included three primary goals I hoped to meet during this independent study:

1. Add basic reports to ListenBrainz from listening history data for top songs / artists / albums of week, month, year, etc.
2. Create documentation to improve ease to use and develop for the ListenBrainz project
3. Offer as a use case for the data visualization course in fall 2018 with instructions on how to use the data

Add basic reports 🔗

Methods for generating basic reports, charts, and statistics about listening history are important. They help make ListenBrainz a more interesting platform for a casual music listener, not just a developer. Therefore, my goal was to add a way to add basic reports or specific metrics for presenting to the user in the front-end.

As a stretch goal, if I have extra time, I would work on generating content (e.g. charts / graphs / statistics) to show the user in the front-end.

Documentation 🔗

Documentation is something near and dear to me. I enjoy making it easier for other people to use a project or get started with contributing. Therefore, I will contribute some time as a technical writer and help improve documentation on the project. This includes improving existing documentation, like how to set up a development environment, or creating new content.

As an end deliverable, it would be nice to have someone who has never worked with the project run get a development environment set up, import some data, and see something presented to them. Good documentation is key to making something like this possible.

Use case for data course 🔗

RIT will offer a data visualization course in future semesters and it would be helpful if ListenBrainz could be a use case or even tool for the course. Then, students could work with ListenBrainz for creating different visualizations for the music data. And maybe contribute some of their visualizations back upstream! For this to happen, we need comprehensive documentation and complete features.

A focus includes making ListenBrainz a good fit for this course.

Learn more about ListenBrainz 🔗

For the next few months, until December, I will blog regularly about contributing to the ListenBrainz project and my progress. Additionally, more posts about MusicBrainz, other MetaBrainz projects, or music data may follow. I’m hoping to either create new or improve old documentation as well, so I plan to write often anyways!

For now, you can learn a bit more about ListenBrainz and other projects in the MetaBrainz family, like MusicBrainz.

• ListenBrainz
• GitHub: metabrainz/listenbrainz-server
• About MusicBrainz
• About MetaBrainz