Data science hacka-what? Hackathon!

The organization of a hackathon is increasingly common and is involved in business innovation. Don’t worry: it is not about hacking our customer servers! (By the way, if this ever happens to you, our security and Identity Access Management teams are there to help you)

Hacka-what?

So, what is it about? A combination of hacking and a marathon, hackathon (also known as hackfest) is an event where teams of people involved in development, collaborate intensively on software projects [source: wikipedia]. Taking the form of a race against the clock competition, the objective is to develop a prototype program (algorithms, web interfaces, API, hardware…). Though it is a competition, the motivation for the participants is more often about the recognition of their skills than about the prize itself.

Usually organized in a (very) short time such as during nights or week-ends, it is about: collaboration, experimentation, agility and serendipity.

1. Collaboration

After the hackathon challenge presentation, teams are created around a list of problem solving ideas/pitch to be developed. Although friends/colleagues often intend to participate together, participants usually end up getting mixed up in different groups, in an attempt to distribute key resources. During the event, participants will have to closely collaborate under high level of (positive) stress. The high degree of required collaboration leads to fruitful exchanges and knowledge sharing.

2. Experimentation and agility

Hackathon topics are announced prior to the event. Everyone therefore receives the opportunity to prepare for the hackathon, as he deems most appropriate, through reading, investigating promising ideas or playing with potential candidate solutions. However, ultimate specifications, requirements or pre-existing components are only given at a later stage, once the project is officially kicked-off on the first meet-up. This implies the necessity to be agile and deploy an iterative trial and error approach.

3. Serendipity

The key objective of hackathons: outcomes, either being a prototype (in one or two days!) or unexpected ideas. For example, the company GroupMe began by a hackathon project and was acquired by Skype less than one year after for… 85 million dollars. The Facebook Like button is an outcome of a hackathon. Hackathons are also sources of disruption…
It is no accident that big companies such as Google or Facebook organize internal hackathons on a regular basis.

The specifics of a hackathon in Data Science

Data science hackathons are the direct transpositions to… data science applications (as you might have guessed). It may cover any data science activity: data acquisition, enrichment, visualization including real time dash boarding, predictions, patterns extraction and so on. Most of the time, a set of data is provided at the beginning of the event as well as business information and the objective is about modeling an event of interest.

As a key Big Data player, Business & Decision encourages its data science teams to participate to hackathons and is investing the time of its consultant for it. Benefits are multiple: team building, on-the-job knowledge increase (worth several training courses!), visibility, development of consulting skills & networking with other participants (and as such a good channel for talents attraction).

A positive strategy

This also happened to be a winning strategy for Business & Decision in general. In March 2016, Business & Decision Belgium participated in a 4 weeks hackathon organized by Procter & Gamble and won the data visualization challenge. During the challenge, more than 200 Gb of compressed signal data was manipulated, coming from 2.200 sensors (time lapse between sensor signals: 5 seconds), as well as motors electricity consumption. The data was as new for P&G , as it was for B&D and other hackathons participants.

Three tracks in parallel: sustainability (is there a way to reduce consumption), dash boarding (real time visualization) and creativity (any insight that may be useful to Procter & Gamble). The Business & Decision Belgium team was composed of 5 consultants working 1 day per week and including two external data scientists. A great experience, with very positive outcomes: after data enrichment with weather information (retrieved thanks to a web API), Business & Decision proposed an approach based on Bayesian Network and trees that may lead to a gain of energy consumption equivalent to annual consumption of more than 300 households under given conditions. On the dashboarding track, two mockups have been proposed: historical visualization based on most recent Microsoft Power BI technology (including natural language data querying) and real time dashboarding implemented with a Spark stack.

To be honest, we do have only one criticism on hackathons: aside some unhealthy sleeping, they often encourage strange eating habits: french fries and beer, but maybe this is more related to Belgium!