Big Data Means New Big Challenges
Data is now considered to be the most valuable commodity on Earth, which is why enterprises and government agencies look to leverage it to their advantage. And yet, warehousing massive amounts of data requires modern technologies and new ways of thinking.
On today’s podcast, Judson Graves, director of analytics solutions at ViON, and Christian Shrauder, director of federal systems at Yellowbrick, offer their insights into the challenges of storing big data.
“There’s too much data, and it’s always growing exponentially,” Graves explained, but storing tons of data at an unlimited scale isn’t a problem. The challenge is storing it so that it’s readily accessible in a useful way, and in the demand of time required.
The world has transitioned to a real-time workload with people wanting answers in seconds or minutes, not hours or days. Arranging data so that it’s available for immediate analytics and evaluation is of top priority.
Shrauder went on to say that he’s occupied with developing systems that can support real-time ingest and ad hoc queries at scale, and can store data so that it’s optimized and makes sense. The way to do that is by implementing the latest and greatest in hardware and software. This also includes rethinking the physical architecture of data centers.
“Anything that is going to work in the future will have to scale incrementally without causing massive downtime,” Shrauder said.
When asked about his opinion on public versus private cloud solutions, Graves maintained that all the coolest technological advancements, from self-driving cars to artificial intelligence, required hardware and software combinations that weren’t available at the time in a public cloud environment.
“If you have a real workload, I don’t ever see when a private cloud or a personal infrastructure would not be needed,” Graves said.
He went on to relay the essential keys to data warehousing: speed, scale, simplicity, security. Solutions that address those four factors will drive successful storage operations in today’s business environment.