We’ve been saying for years that the only way for the legal system to deal with the volumes of information available for discovery is legal cloud computing. If you are a Nextpoint user or just friendly follower, we are preaching to the choir. It seems like such an obvious proposition that perhaps we take it for granted, one that no one could possibly question. But today, let’s unpack that idea.
When we say that the volume of digital evidence is exploding, what does that mean? When we say that traditional IT infrastructure cannot hope to handle the volumes of data available for litigation, why is that? And why is cloud computing the only technology that can keep up?
How big is Big Data?
Organizations today love data. And modern businesses are finding new and interesting ways to generate and use it. The growth of data is clobbering business IT environments, our federal government, federal court system, and pretty much any data-driven business.
It’s hard to quantify, but EMC, one of world’s leading storage technology service providers has a ticker running on their website on the growth of digital information.
The numbers are huge. For example, consider that Amazon Web Services now hosts over a trillion objects. Certain industries, like medical or financial services, are finding new and innovative ways to create terabytes of data– whether it be brain scans or high-frequency trading data that may need to be preserved for decades. In the 1970’s Bill Gates was telling people, “No one will need more than 637 kb of memory for a personal computer.” Today, personal computers ship with 2 terabyte hard drives.
The Committee Against The Misuse Of The Word ‘Exponentially’
When computer scientists talk about the kind of growth seen in most corporate environments, it’s called exponential growth. That word gets used a lot, and may be one of of the most misused terms in vogue today. To understand why traditional IT technology cannot be expected to process high volumes of data, it’s important to understand that term.
I’m no snoot, but as people who should know better insist on overusing the word to describe anything that’s growing fast, it’s quickly losing all meaning. Exponential is supposed to mean something grows by multiples of some factor in a rapidly accelerating fashion. It does not refer to ordinary rapid, but steady, growth.
The distinction between exponential and ordinary, linear growth is important. It is the same as the difference between simple interest and compound interest.
Think about it this way – if you start with a population of 100, and grow that population by 1 percent a year, that figure will double in 100 years and after 500 years will grow to 600. However, if a population grows exponentially at 1 percent, it doubles in only 70 years, and in five hundred years will be well over 1.6 million. (Check my math here.)
For a truly nerdy example, think about the difference in population growth rates between the Tribbles and the crew of the U.S.S. Enterprise.
Adapting to Change Can Be Counterintuitive
The most famous philosopher of exponential growth was also one of the most wrongheaded philosophers in the history of the world, Thomas Malthus.
Malthus realized that even at modest growth rates, human populations could double in only a matter of decades (Exponential growth). He assumed that agricultural technology, which grew linearly, could not keep up. This would lead to mass starvation, war, and death. Fortunately for the human race, Malthus was pretty much completely wrong. Mostly because he did not anticipate the ability of modern agriculture – powered by fossil fuels – to support more than 6 billion people.
A law firm Malthusian might look at the explosion in the volume of data that is discoverable for litigation and then look at the IT infrastructure in their law firm with despair. They would do some rough calculations and realize that they were quickly heading for a complete breakdown.
But like Malthus, they are not anticipating the ability of the larger society to innovate its way out of disaster. Specifically, the ability of cloud providers to pool resources to meet demands for storage and application resources.
Nextpoint embraced Amazon Web Services’ cloud platform years ago because it allows us to bring as much or as little computing power online as needed for any matter our clients may throw at us. Given the growing universe of data, this is the only viable, long-term solution.
Linear Solutions to Exponential Growth
When faced with an exponentially growing challenge, people often respond with linear solutions. When a law firm that is used to dealing with gigabytes of evidence in litigation starts to get cases involving a terabyte of data, their instinct is to upgrade their technology. That means giving their IT guy a budget to buy more computers.
When a law firm with one data storage computer buys a new computer to address this need, it is effectively doubling its capacity. When a law firm has 100 computers and buys a few new file servers every couple of years, even with the exponential growth in hard drive capacity, it is only increasing its IT capacity linearly, especially as old hardware is likely taken offline.
Computers are a bottleneck. You can always throw hardware at the problem, but when clients that used to provide several gigabytes of data for discovery are now delivering terabytes of structured and unstructured data for review, a few new computers is not going to address the problem. That’s a linear solution for exponential growth.
Why the Cloud is the Answer to eDiscovery
Computer scientists who understand the limits of computing power know that cloud networks are the only mature and viable solution for the growing universe of data. (At least until quantum computers come into existence.) Law firms are reluctant to abandon the technology they have relied on for decades in favor of something that seems overhyped.
We tend to overestimate the effect of a technology in the short run and underestimate the effect in the long run.
But even back-of-the-envelope calculations would probably open their eyes to the long-term benefits. As the late American futurist Roy Amara said, “We tend to overestimate the effect of a technology in the short run and underestimate the effect in the long run.”
Answering the challenge of big data with current Internet and cloud-based technologies is the only answer. The volume of data expansion is directly the result of the Internet. Building out your own data center is putting both the buyers and vendors in a nether world. In addition, there are other costs associated with running your own data center- buying the servers, maintaining them, networking them, depreciating them, replacing them, troubleshooting them, staffing them – and the list goes on.
Everyone, even traditional tech vendors, are coming to understand that no one is willing to invest long term in a space that Amazon has commoditized. Just like no one buys electrical generators for their infrastructure, there is no need or reason to build out a data center.
For this reason, an answer rooted in Internet technologies is the only feasible answer going forward as a cost-effective, systematic method. Throwing additional funds at outdated, legacy technology will simply not solve the problem – and most litigators we work with have often already learned this lesson the hard way.
Interested in more about eDiscovery in the Cloud? Download our free eBook, Managing eDiscovery in the Cloud.
