We're really proud to be residents at the Digital Catapult Centre in Brighton - it's a great environment for us as an innovative start-up working with big data, digital and Internet of Things technologies, and one of the things it's enabled us to do is collaborate with people and groups we probably wouldn't normally have met. As an example we've been doing some really interesting work on data with Brighton Music Office (BMO).
If you live and work in Brighton like we do, or if you've ever visited, you'll be aware that there is a vibrant and creative music scene from festivals like the incredible Great Escape to the brilliant bloke dressed as Rowlf pushing a piano around town. The Brighton Music Office's role is to help develop Brighton's music economy and take a lead on defining and delivering the support and environment required to grow that economy.
Brighton Music Office's plan for doing this is to firstly map out all of the key components involved in building a great musical economy. Once those are mapped, the goal is to build and publish a comprehensive online directory of all the bands, venues, gigs, festivals, producers, engineers, agents and several others, enabling people to easily find what they need to progress.
Then the aim is to define the optimal environment and roadmap for growth of the musical economy in Brighton, and to provide clear pathways for people involved in the scene to become successful.
We thought that this would be a really interesting project to demonstrate the value of using data and analytics solutions to drive efficiency and effectiveness, and hopefully some transformational outcomes, and decided to collaborate with BMO.
What did we do to help?
The first thing we did was to quickly go through our Making Data Work Assessment to understand what BMO were trying to achieve and how, what data assets BMO had access to and what data they might need to acquire, and to understand what kind of technical platform and capability they would need to support them.
We also spent some time understanding the likely applications of data and analytics, and the users that might be interacting with them - there's a potentially wide range of stakeholders that could be users for BMO's platform.
We then looked at the kinds of value we could deliver, identifying areas we could make more efficient and effective through using data and analytics.
The Opportunities We Identified
So far the team had done a great job manually capturing a wide amount of data, but it was taking too long and not covering enough ground.
The team at BMO faced a number of challenges when trying to build the current picture of Brighton's music scene, and we identified opportunities to use data to address these:
- Capturing the Data: How to capture a large amount of data across a wide range of diverse sources efficiently
- Storing and Visualising the Data: How to store, model and consolidate these data sources and to identify the key players in the various networks effectively
- Speed, repeatability and Low TCO: How to rapidly build a capability that could be reused for other cities, without high overheads on maintenance or technology costs
Capturing the data
The first challenge to address was automating the capture of the data, to reduce the amount of time it was taking for the BMO team.
To do this we took the data that Chelsea and Laura from BMO had manually recorded as a starter, then built a series of scripts in R to start sampling further data from a variety of sources and website APIs.
We started by using a depth-first search method to build out a list of bands in the Brighton area. From an initial sample set we expanded out our sample by looking for connections between the bands - for example bands who had liked each other's social media pages. We managed to build out a really comprehensive list (helped by a few well connected or well known ones like Royal Blood, Bat for Lashes and Tigercub) in just half a morning, something that would have taken weeks manually.
From there we were able to expand our sample to include gigs and venues. This will allow BMO to publish a comprehensive gig listing for Brighton in the directory. Next we'll be looking at expanding this into other areas like promoters and PR.
Storing and Visualizing the Data
The next challenges were where could we store the data and start to model it out for analysis as required. We used an R package to automatically load the data into Google Sheets initially as a staging area so we could validate and merge the data quickly with the Google Sheets BMO had been using.
We wanted to use a database that was flexible, scalable, cheap and required no maintenance, and as we were staging some of the data in Google Sheets we decided to use Google Cloud SQL. It's really simple to set up and use and fits the bill perfectly. There's tons of alternatives out there, but this one is going well for us so far.
We then wanted to visualise the data to enable the BMO team to quickly identify key nodes in the networks based on a range of attributes so they could focus their approach for
To do this quickly without too much effort (5-10 mins!) we brought the data into Google Fusion Tables and built a quick network visualisation of the connections between groups so the BMO team could review which ones they wanted to focus on. Here's a simple example showing bands and their social media connectivity and popularity:
Speed, repeatability and low TCO
Through using the R scripts, we were able to automate much of the data capture process, saving the BMO team hundreds of hours of data entry. Also, through the depth-first searching approach we were able to build a more comprehensive view of the music economy than would have been possible manually.
Through the ability to quickly visualise the data in simple ways, we were able to help the BMO team quickly review and identify key aspects to focus on rather than either a random approach that might miss things or a comprehensive approach that would take way too long.
The fact that all this was done for no cost other than having the Google accounts and relatively little coding and development time shows that data and analytics can be leveraged for all kinds of organisations to help drive efficiency and effectiveness.
As BMO move forward with their work we're planning to look at how we measure the success and growth of the music economy for Brighton and to do some deeper analysis on the data sets we've been able to capture to start correlating the factors behind that success.
As the online directory gets up and running we'd like to make the visualisations available to the users so they can engage with the data themselves and start planning their own path to success - it was incredibly easy just to publish using Google Fusion Tables.
In summary this collaboration has shown that through using simple, quick data and analytics tools and our approaches, we're able to identify and deliver real value to companies of all kinds without huge overheads and lomg timescales.
If you're interested to hear more we'll publish the next steps soon, and you can subscribe to the blog to get them delivered right to you.