Big data: opportunity and challenge
Over the past two decades, America has experienced a societal revolution led by the Internet and the availability of “big data,” – defined by the Wall Street Journal as “the ability to collect, process and interpret massive amounts of information.”
Although big data has transformed modern culture, massive information sharing and analysis has yet to generate significant benefits within healthcare.
The promise of big data in healthcare is revolutionary. Use of big data will ease the transition to authentic data-driven healthcare, allowing healthcare professionals to improve the standard of care based on millions of cases, define needs for subpopulations, and identify and intervene for population groups at risk for poor outcomes. To date, few healthcare professionals would claim that the promise of big data has been fulfilled.
To understand the growth and usage of big data, it makes sense to look at an industry with advanced use of big data: consumer information technology.
Big data in consumer IT
Consumer IT uses big data daily. Accessing the Internet via Google or reaching out to friends via Facebook relies on the massive collection and transformation of information. How did consumer IT gain the ability to deliver massive value from massive data? Three trends supported this transformation.
1. Data availability: Information availability grew in the 1990s as the Internet offered a source of content with formalized protocols and broad access.
2. Metadata and grouping: Collecting the information wasn’t enough; applications had to understand the information. Information coding and grouping accelerated in the 2000s thanks to algorithms and systems that incorporated keyword matching, social grouping, natural language processing (NLP) and search algorithms. Simple text became marked up content, indexed and annotated for use.
3. Applications to leverage big data: Once information was available and annotated, companies like Google, Facebook and LinkedIn capitalized on a wealth of usable data and enabling technologies to meet consumers’ emerging information needs.
Big data in healthcare
If consumer IT accomplished so much over the last two decades, can healthcare derive similar high value from its big data? Given the high stakes and serious concerns, the answer in the short term is “maybe.” Industry commitment is shaped by multiple concerns, including data security, de-identification, patient versus societal benefit, stakeholder profits and political swings.
To balance these concerns and questions, healthcare professionals must understand their position in the trends of enabling big data usage.