Big data is poised to be the next big market for vendors of all stripes. Thanks to the proliferation of cheaper storage and cloud services, everyone from small time hackers to the government are looking for ways to leverage big data for discovery. The proof-of-concept for big data comes from Google, Amazon and Facebook which all power their businesses on big data. Now, governments of all sizes are realizing their own big data potential and contractors are lining up.
Government has always been a vestige of big data, it has also been a leader in inefficient storage, poor organization, and outdated technology. That began to change with the influx of more tech-savvy public officials ushered in through the Obama Administration at the federal level. State and local government is also realizing the need for better understanding of its data and technology improvements as revenue starved desperation has defined post 2008 budget cycles.
Proponents of big data bill it as the answer to everything from cost savings to better government, better health care and better crime fighting. Cities are launching open data policies, releasing data sets and asking local hackers to make sense of them largely for free – creating a budget director’s dream scenario – shiny new government services, no pay-for required.
Big Data flush with VC cash
Not everything happening with Big Data is free however, this morning we saw the announcement of two big funding rounds for big data hopefuls – Junar and 10gen. Junar provides a Software-as-a-Service (SaaS) open data platform for governments and small businesses. According to the company, the cloud-based SaaS platform is designed to help users create and manage big open data projects without having to create their own in-house solution. Visitors to the Junar website can create their own free account.
“Open Data is fast becoming the next defining theme for business and government,” says Diego Ma, CEO and co-founder of Junar. Some angel investors are inclined to agree, the company raised $1.2 million in initial funding from Aurus, Austal Capital and angels in both Latin America and the US.
The story is much the same for 10gen, the company behind MongoDB a database used by government and other industries such as financial services that require hefty but agile infrastructures. The company announced this morning that it has secured $42 million in financing in a round led by New Enterprise Associates with participation from venture capital firms Sequoia Capital, Flybridge Capital Partners and Union Square Ventures.
10gen has grown its team 400% since last year – with most new hires happening in the technology group. Harry Weller, general partner at New Enterprise Associates the venture capital firm that led the fundraising effort calls the company a “disrupter” in the database race.
The federal government recently announced that it will be supporting a wide rage of big data research and development as well. Several government agencies will be working in concert with private sector IT providers to find ways of “unlocking the potential” of big data for government. The government clearly needs help to manage its data, a recent report from MeriTalk shows that government agencies will add a petabyte of new data in the next two years adding to a growing data storage problem.
You never get fired for hiring IBM
Even while the hype around big and open data for government grows, questions remain about how much of an impact these upstarts will really have. Open data policies and app contests may improve civic engagement within a certain sector of a local populace but it seems that big data has an equal chance of simply becoming another part of big IT’s enterprise stack.
Data education and ownership are also big questions. According to the Meritalk study, 60% of civilian agencies and 42% of Department of Defense/intelligence agencies say they are just now learning about big data; even fewer agencies (40%) are using big data to make strategic decisions. Numbers are lower at the state and local level, where data could arguably be used faster and to greater effect.
May of the big announcements in big data are coming from the usual suspects – IBM announced this morning that it has acquired Vivismo, a provider of federated discovery and navigation software that helps users organize and understand big data. The acquisition adds to IBM’s growing data and analytics portfolio which already includes law enforcement and health care initiatives.
Vivismo’s existing client base includes the US Air Force, US Navy, and the Social Security Administration, in addition to other government entities.
This acquisition is notable as it focuses on federated discovery and navigation – key issues as much of the data housed within government is poorly structured, may include proprietary third party information, and has unclear ownership. 64% of respondents in the Meritalk study said the amount of unstructured data they store has increased in the past two years.