Big Data, Big Numbers and Elusive 4D Temporal Geography
The presentation by Young Hahn at FOSS4G NAlast year, on his work with the MapBox team to churn out updated map tiles for the whole planet in a day got me thinking once again about an idea I’ve had for some time that makes sense, but perhaps only to me. I thought I would toss this idea out for general consumption and peer review, scrutiny, critique and hopefully, limited bruising to my ego. The concept of trying to map things (physical or cultural) on the earth over time is challenging at best. There have been excellent examples of doing this in limited areas over slices of time but I’ve yet to see a truly global effort to provide a continuum of data as a surface. The closest thing I have seen that even approaches doing this at a global scale isMapStory.
I often use the analogy in conversation, when trying to describe this to colleagues, of the moment when you break through the clouds on an airplane and you happen to have a window seat on a clear day, and you are at eye-level with the top of the clouds. Looking out, you can see an undulating, irregular and dynamic surface punctuated by pockets (no clouds) or perhaps a boiling thunderstorm rising up in an isolated vertical column.
The turbulence you experience is the impact of the plane slicing through this surface or skidding across patches of it at cruising altitude. This provides a healthy analogy for continuous mapping of spatio-temporal data across the entire planet.
To do this, we need a way to uniquely index the planet in 4D (x, y, z & t for those of you scoring at home). My idea (and the latent flaws in it) draws from a combination of these variables, so humor me and let me try to articulate this without additional context or history.
If we look at the maximum practical values for each variable above, we’d have something like this:
Latitude (x) = xx.xxx (ie. 89.999 in DD)Longitude (y) = xxx.xxx (ie. 142.034 in DD)Elevation (z) = xxxxx (ie. in meters, yes, it’s metric, get over it.)Date (d) = xxxxxxxx (ie. 31122012, sorry StarWars is cool, but no Star Dates here)Time (t) = xxxxxx (ie. 23:59:59)
If we fishnet grid the entire planet, or perhaps just the land area, at 5 meter intervals we get 20,402,880,000,000 points in the grid (give or take). That’s nearly 20.5 Trillion points. By any definition, that qualifies as “big data”. It gets better.
If we concatenate (that word always sounds funny to say out loud) the numeric values of the variables above into a single integer (and we stay consistent about it) we get something like:
899,991,420,341,000,031,122,012,245,959 which is more than enough digits to conquer the ~ 20.5 trillion point fishnet across the land surface of the earth.
‘Why bother with all this big math?’ someone countered me when I brought this up two years ago at WhereCampDC, since we already have UTM to do this? (I admit, I chose the wrong forum at the wrong time to introduce an imperfect idea and was promptly & properly shot down) The question is still valid however and my response at this point would be, this number (whatever I get around to calling it) provides a truly unique temporal index to just about everything that has ever or would ever be important to us as humans (and the non-human species around us). It would allow us to reference everything (to the nearest 5 meter coordinate) over time and if that were to happen consistently and uniformly, we could then perform analysis that is only possible in the realm of science-fiction or DARPA (sort of the same thing at times)
Being able to investigate, interrogate or query time & space (true 4D) and have all results returned on-demand and presented in a rich UI (ie. map) via a browser would be, well, pretty awesome. It leads back to the notion of the top of the cloud deck - that active filtering of spatial-temporal data at 5 m resolution could provide true “art of the possible” capabilities and insight that might just make a difference in trying to understand the world around us and our impacts on it and upon each other, even get crazy and start anticipating if not predicting strategic events. Or perhaps my idea is just a more modern adventure fashioned after Quixote.
If you thought that was off the reservation, cycle through that math with a 1 meter fishnet and we’re into the quadrillions (510,072,000,000,000,000 to be precise, give or take). Those numbers venture into the realm of Jeff Jonas and Terry Buschand that gets truly interesting real quick.