Everyone in oil and gas is talking Big Data, but is anyone doing anything about it? After all, what has produced more oil and gas in the last half decade: greater sand volume, better lateral placement and closer stage spacing—or Big Data?

If one views Big Data as automating business processes, generating economic and performance gains through data analytics, or developing insight via machine learning and predictive analytics, then give the devil his due: The oil and gas industry is in the early innings of a Big Data revolution.

And speaking of the devil, Big Data originated in oil and gas through the advent of large-scale geo-seismic programs in the 1960s. Big Data has since migrated to Silicon Valley. That said, cheaper, more powerful computing horsepower is democratizing Big Data so that the smallest E&P firms today, in theory, have access to the same technology as the largest firms.

Still, there is need for a devil’s advocate when it comes to Big Data solutions in tight formation oil and gas.

For example, will Big Data supplant basic math? In other words, if three E&P companies employ artificial intelligence and machine learning using archived data from the same play, will a Big Data approach produce the same solution for all three?

Similarly, is Big Data an internal initiative or a service best contracted out to whiz-bang specialists? A staff of three young and talented data scientists will not outdo Google, Amazon, Apple or Facebook in creating new artificial intelligence solutions.

Finally, can smaller E&Ps benefit to the same extent as their larger peers?

For now, Big Data initiatives in oil and gas are primarily internal efforts dedicated to standardizing archived well data and creating an accessible, user-friendly, interdepartmental platform across an organization. Think of it as a convoluted way to get the ultimate recovery-focused production engineer to communicate with the IP-focused completions engineer or IRR-focused investor relations officer.

Large E&Ps have a good foundation that incorporates information from thousands of archived wells addressing geomechanics, formation and reservoir characteristics and completion techniques. Some are mining that data to address how to complete individual stages or build a program for full field development. For others, Big Data simply involves scrubbing and standardizing archived information.

Examples of machine learning and automation at work include directional drilling, a technique that supports the most important advance in recent oil and gas productivity, which is threading the lateral through the very best rock. Elsewhere, several E&Ps are studying ways to automate various well processes to capture efficiency gains.

The greatest industry benefit longer term will be found in applying Big Data analytics to production optimization.

Shorter term, how do smaller E&Ps participate? Many lack the well data archive found among their larger peers. Furthermore, private-equity-backed firms are on a short investment cycle that precludes in-house initiatives.

Could regional consortia accelerate adoption along the lines of what Core Laboratories NV has done in creating peer studies to decipher reservoir characteristics in a new play? Maybe, but E&Ps that have spent time and money on back office Big Data efforts remain understandably reluctant to share proprietary solutions with competitors—even before the attorneys get involved.

One promising avenue for smaller firms is in real-time data acquisition from the drilling rig, where proper instrumentation can generate up to 7,000 measurements per second. The devil in disguise revolves around who owns that data, creating a user friendly interface to inform the drilling and completion process, and collating disparate data from well logs and multiple service providers, who often don’t speak to one another.

Big Data is disruptive. In the question of frack sand or data, the answer to which has produced more oil and gas is nuanced. The move away from 25 years of light sand slickwater completions to loading large volumes of finer mesh proppant in 2012 originated in outsourced Big Data. So, too, did the emphasis on precise lateral placement in the best rock, coupled with targeted fracks and an emphasis on near wellbore harvest. Both innovations were opposite of existing industry best practices.

For Big Data, the devil is in the details. Rather than pursing a moonshot, think more along the lines of incremental gains. E&Ps that adapt will prosper. For the others, the devil will take the hindmost.