It’s true, data lets us see around corners
When I was a kid, I had a book about how to be a spy. In it there were ciphers, instructions on using lemons to write in invisible ink, and my favorite, how to make a periscope. It just seemed super cool to be able to see around corners.
Fast forward to today, and in an era of big data, this same ability to see around corners is still the one that excites me. We can use data to reconstruct things that we can’t observe with the naked eye. We may not be able to observe something happening, but we can measure related phenomena and make a calculated estimate of the original event. Let me give you an example.
Whenever you use an electrical device in your house, it has a distinctive signature in the way it uses power. Devices such as Sense can use that information, combined with its models, to determine which devices you’re using at any point in time.
It is absurd to contemplate collecting this information directly. Nobody will ever log every device use, nor could you persuade every electrical manufacturer to include logging functionality. By using a combination of passive data collection and modeling, you can circumvent the lack of physical access.
Data can fix the physical world
The power of data and modeling is good news for anybody with a large investment in real things. Let’s look at an extreme example, the Hubble Space Telescope. The initial investment and the cost of repairs are, quite literally, sky high. The sensors on the telescope operate in the most brutal environment possible, exposed to cosmic rays. Over time this causes degradation, making the images less useful to scientists. Fortunately, scientists discovered that they can model the degraded sensors, and use this information to restore the original image. Even in its imperfect state, more value can be realized from the telescope.
What else can we fix with data?
Much has been made of the travails of retailers in the age of Amazon. Jeff Bezos built Amazon from the ground up as an online retailer. Its distribution logistics are specifically geared to its business model. Companies such as Target or Macy’s are at risk of being left behind, because their logistics are oriented to a nationwide network of physical stores. They can’t reorganize overnight to look like Amazon. But what they can do is use software and data to present an online experience that looks like Amazon to the customer, making smart use of their existing stores and distribution networks. It won’t be the same, but it keeps them in the hunt.
In the same vein, think about consumer brands. Traditionally, their products are sold through distribution channels, bought and sold by brick and mortar stores. The brands don’t have a strong idea exactly who is buying their product, and they tend to iterate on their product line very slowly. Online retailers present strong competition: brands such as Harry’s have a one-to-one relationship with their customer and can move much quicker to tailor the product to meet market needs. Established brands are falling behind as consumer expectations are recalibrating to the level of service they get from apps and online services.
How can data help the brands? They have no direct customer connection. However there are many indirect signals available they can measure: talk on social media, creating web sites and apps that encourage further interaction with the brand, reviews on consumer sites. Using these, you can construct a probabilistic model of consumer behavior. A brand might not get a 1:1 connection to the customer, but it can build a “good enough” model to let it move faster and serve its market better.
Not just seeing through walls, but time travel too
One of my favorite stories about the power of data comes from my friend and colleague John Akred. Some years ago, he worked a project on oil rigs, looking at data from the sensors on the oil platform to determine when a key component was going to fail. It didn’t seem like there was a way of finding out how to get an early warning. What John did was broaden the search, and he discovered that a pressure sensor showed abnormal readings two full weeks before the failure occurred. Add this data into the mix, and you can get to the problem in time. But to me what’s most striking about this story, even years later, is the satisfaction it gives John. An oil platform is a very big thing, made from steel and concrete, but he could fix it with math!
Movies portray a future where we increasingly live inside machines. The evolution of cellphones and virtual reality certainly points that way. But I still remain excited about the reverse notion, of how much we can improve the real world through computation. Big data has the ability to be a “macro-microscope”, helping us understand our environment more deeply and broadly than we ever could before.
