Highways in the Sky

A data visualisation by easypeasy-analytics

Highways in the Sky

From small talk to 3D Airspace video

I guess it all started right before our lunch break. As always, we were asking ourselves where to go for lunch and what not to have for the sake of diversity. Our discussion lead somehow to planes and how they take off and land (by the way I believe at least one of us has ADHD) and how air traffic control manages to sequences hundreds of aircraft without them bumping into each other. Alistair as a former air force pilot has a pretty good understanding how things work up in the air. However, I definitely lack in 3 dimensional imagination and an ability to comprehend all his hand gestures waving around my face.

For me a visualization of the actual flight data would have been a good start in understanding how things work up there. So what I needed was a simple visualization of a typical day. And so it had begun.

Well, one thing lead to another and here we are; Ali did a little research and found out that there is an API that provides real time location of planes, it sounded quite simple just to plot it using Visual Python for a quick demo. When he did, we discovered the beauty hidden in data, like the messy air space above London and the almost perfect Atlanta corridors. At that point we thought that it would be worth to go one step further and share it.

From then on, it was collecting and filtering data, plotting and rendering … and exactly here ends my technical knowledge how Alistair and Tatiana did this amazing video.

How To:

First things first, we needed data. We used python to call the flightstat realtime flight position API to get back various data on aircraft in predetermined locations at a given time. The data came in the form of a JSON file which we parsed to get the parts that were required to plot a 3-dimentional scene. Initially we stored this on a local database (my mac!) to get things working before installing everything on our server to enable uninterrupted round the clock gathering of data.

Google provides a static maps API where http queries can be made to retrieve a specific location and zoom map fragment. We also used python to generate these queries so we could retrieve any map segment across the globe for which we had also collected data. Cartography is a precise business and we had to derive some relations to fit the maps together and combine pixel locations on an image with real world latitudes and longitudes. This means knowing how to work with the Mercator Projection. Not for the light hearted!

So we had the maps and the data, the next step was to combine them all in a single scene and run the clock so we could create the time-lapse of what is going on in the sky above us. For this we use a module of python specifically designed to create visualisations. With some careful ordering and filtering of the data we can plot the flights with trails so we can see 1000,s of flights and the directions they move in.

Highway lanes at Atlanta

Visual python is great for making animations of almost anything you can think of, but we were looking for a final product with a unique quality and distinctive style. For that we needed to render each frame of the time-lapse from a file that defines all of the components (lights, objects, reflectivity etc) of the scene. That meant creating thousands of HD frames for each clip of the video. We turned again to our server for the round the clock processing power. This was by far the longest process of the entire project (maybe comprehending the Mercator projections comes in a close second!). Then QuickTime and iMovie was used to glue it all together.

Try For Yourself:

Raw data:

Highways_in_the_Sky_Raw_data.zip – raw data covering 30 mins of traffic over London (Original JSON format)

Useful References:

Learn Python through public data hacking – youtube – great intro to python with a couple of fun cases

Google developers static maps api documentation – all you need to know about static maps

Out of intense complexities intense simplicities emerge.

Winston Churchill

Recent Videos

Hottest skills of 2014 that will get you hired

Hot Skills Statistical Analysis and Data mining universally top the board of the skills that will get you hired in a study by LinkedIn. Having recently made a move myself towards a data focused position I can personally vouch for these results! See the...

Companies Want MBAs Who Can Code

Companies are increasingly reliant on data for the answers to questions and are less influenced by the blah blah blah and gut feelings of "that person who is louder and uses big fancy business words". The people looking into the data are most likely to start...

Highways in the Sky – Destination London

An Airport Under Pressure The National Air Traffic Control Services (NATS) in the United Kingdom announced recently that they now have a new system to squeeze more flights into London’s Heathrow Airport, which currently operates at...