Push to exploit an ocean of information
Richard Waters Source: The Financial Times. (Dec. 10, 2012): News: p19 Richard Waters Source: The Financial Times. (Dec. 10, 2012): News: p19 Richard Waters Source: The Financial Times. (Dec. 10, 2012): News: p19 It took atomic physicists Ari Tuchman and John Stockton four years to realise that there was no future for the business they had set up to make gyroscopes for nuclear submarines and missiles.
Last year, however, they hit on a new use for the complex algorithms they had written: to predict what kind of people are most likely to buy tickets for Hollywood's latest blockbuster films.
"The math is kind of the same," says Mr Tuchman.
Their technology was geared to "measuring tiny spin signals in a background of noise" - pretty much, he adds, like trying to pick out Hollywood's best prospects from the mass of people talking about a new film on Facebook.
Quantifind, the company they founded, has joined one of the hottest, and potentially most disruptive, tech trends in business.
The application of hardcore science to the ocean of digital data in which many businesses and organisations are awash promises tantalising new insights in fields far beyond Hollywood.
But like all disruptive technologies, it threatens those that fail to keep up.
Companies that do not have data in their DNA will be the losers in this upheaval, says David Friedberg, a former Google executive and founder of Climate Corporation. His company sets prices for crop insurance by running big daily simulations of weather patterns to predict their impact on crop yields across the US - an exercise in parallel processing that means calling on 4,000 servers at once.
"It's very hard for an insurance company to do what we do," he says.
Behind this wholesale effort to bring applied science to bear on business decision-making lies a force that is quickly becoming one of the most overused buzz words in business: "big data".
There is nothing new in the idea of gathering and crunching large amounts of information to glean insights about customers. Data warehousing, a term long applied to the practice, became a staple for companies in industries such as credit cards and retailing in the 1990s.
But soaring online activity, falling costs for gathering and processing information and a technology infrastructure designed to make it easier to combine and analyse different streams of information have pushed the practice into more industries and many different forms of decision-making. The gusher of data coming out of social media sites and generated by smartphones and other mobile devices has added the latest spin.
Unlikely-sounding attempts to turn these new forms of data to business use are breaking out all over.
Euclid Analytics, a Silicon Valley start-up, sells WiFi "sniffing" sensors to retailers so that they can detect people with smartphones who walk past their stores.
The technology tries to identify how many stop to look at a window display - as well as how many come into the store, how long they stay, and how often they come back.
It is already used by several of the largest US apparel retailers, says William Smith, chief executive. "We're creating data where there wasn't any, and in the process answering questions that couldn't be answered before."
Like anticipating film demand and judging the effectiveness of window displays, much of the effort in the field of big data analytics is aimed at making existing companies more effective. Designing products, setting optimal prices and reaching the best prospects among potential customers are turning into data-driven exercises.
But it is also throwing up disruptive new businesses. Companies set up from scratch have the chance to draw on public streams of digital data to enter markets that were once closed to incumbents with long-established customer relationships and proprietary information. And such businesses come without the legacy technology platforms, entrenched business processes and cultural norms that make it hard for big groups to change.
"Even if you're not a bank or a healthcare company, you can play in banking or healthcare," says James Manyika, director at McKinsey's research arm.
The public availability of streams of data from different sources, in a form that can be ingested, combined and analysed to yield new insights, has become one of the keys to this big data science.
Climate Corporation taps into information on historic weather patterns, details of soil types down to the level of individual fields and years of data types of seed that have been planted.
Behind this stand companies such as Factual, a start-up that sells voluminous feeds of information to data-driven businesses. Its initial services include location and other information about 63m businesses and a catalogue covering 650,000 consumer products.
Founder Gil Elbaz, another former Google executive, forecasts that a small number of companies such as his will end up becoming the "canonical" sources of digital information on which the big data world turns.
Information that would once have been dismissed as a waste byproduct of other activities is creating an opening for companies to break into new markets.
Kabbage, a US company that lends to small businesses over the internet, has been collecting large amounts of data about its customers, and plans to use it to offer them tailored products like property-casualty insurance from companies it partners with, says chief operating officer Kathryn Petralia.
Echoing the mantra of many new companies that see their data-crunching skills as their competitive advantage, she adds: "What we're building is a data business."
Such rallying cries are not new in the business world. Over the past two decades, a small number of big groups have won recognition for their success at turning data into gold. They include US credit card company Capital One - a start-up from the early 1990s that used superior data analysis skills to take on the credit card industry - and casino operator Harrah's, as well as retailers such as Walmart and Tesco.
Many others are trying to learn the big data ropes and will eventually catch up with the pioneers, says Mr Elbaz at Factual.
Echoing other experts, though, he warns that this will produce many losers as well as winners.
"The question is, who has the skillset inside their organisation to do something with [the data]? It's a cultural question."