By Felix Fallon

In 2006, Clive Humby, then the architect of a supermarket loyalty scheme, expressed the now-popular belief that “data is the new oil”. He was talking about the potential value in using the massive amounts of customer data generated by the Tesco Clubcard system, but the axiom rings just as true for the oil and gas industry.

Analysis of big data is currently utilized mostly in the upstream sector, giving engineers and geologists a viable way to use the astronomical amounts of data generated by seismic imaging and other drilling and exploration technologies. However, the practice is yet to be implemented on a large scale in the midstream and downstream sectors, despite having the potential to streamline logistics, provide targeted sales models, and even uproot the business structures of the most established companies to make them more efficient, streamlined, and to generate value.

There are three principal restrictions in traditional data analysis: volume, velocity and variety.  As data generation has evolved, conventional computing methods have struggled to keep up with the amount of data being received, the speed at which it is generated, and the growing spectrum of sources used for data collection.

We no longer live in the times of simple documents, personnel files and financial transactions. Improved technology means that data can be extracted from images, video, audio, 3D models, simulations, location and time-specific information. Vast databases are now required to store such volumes of data, the sizes of which exceed the capacity of traditional IT systems. As an example, a modern offshore drilling platform has about 80,000 sensors, which are expected to generate 15 petabytes of data in various forms during the lifespan of the facility.

The advent of readily-available big data technology earlier this decade presented the opportunity of both better understanding systems and better allocating resources; traditional computing techniques have to ignore and delete large quantities of potentially useful information which is outside of its capabilities. Big data analytics is able to categorize and process this information through machine learning and analysis software; revealing trends, correlations, and outcome probabilities otherwise impossible to reach.

In addition to the benefits it can bring to operational efficiency, big data can also make positive improvements to a company’s balance sheet. Anders Brun, partner at McKinsey Oil & Gas Practice, told Egypt Oil & Gas that “large savings in operations improve cash flows, enhance production, and improve logistics ensuring growth.”

“Recent analyses across 12 operators show that the total potential, across upstream and downstream for an integrated player, may amount to a cash flow improvement of as much as $11 per barrel of oil equivalent by 2025… Specifically, for refiners we see potential in the range of 5% of gross margin improvements.”

Big Data in the Upstream Sector

Oil and gas companies conduct advanced geophysical modeling and simulation to support upstream operations, using 2D, 3D, and 4D seismic imaging that generates significant amounts of data (a single 3D seismic image can exceed one petabyte in size). Companies use tens of thousands of data-collecting sensors to provide real-time monitoring of both subsurface wells and surface facilities. Yet, the disparate and increasingly complex forms of information received, make it a challenge to collect, interpret, and leverage the data.

For exploration, the use of big data makes all this information accessible and interpretable; navigation, visualization, and discovery become quicker and easier. Aided by machine learning and advanced analysis software, companies can use integrated asset models to assess the viability of drilling a certain well, or exploring a certain concession, especially for unconventional resources.

Unconventional fields have many wells on a limited acreage, each with its own specific production-type curves, cost of drilling, geological formations, leasing costs, and completions optimizations. Each of these factors changes from well to well, location to location, or from one period of time to another. These variances make it difficult to compare assets using traditional data analysis.

Oil and gas operator ConocoPhillips began development on the Eagle Ford shale field in Texas in 2010 and drilled 1,000 wells in their first seven years of production. In the period from 2015 to 2017, the company utilized advanced data analytics and reduced their drilling days by 50%.

The operator had incoming data on factors such as the rates of penetration, the rotations per minute (RPM) on the drill floor, and pump flow rates; comparing these data points to operations on other rigs provided a large reference database for the Eagle Ford concession. Through the use of analytics, ConocoPhillips was able to identify and replicate optimal conditions and techniques across all of its drilling sites.

“It did not stop there as we are continually measuring, adjusting, and experimenting through several different cycles to dramatically improve drilling performance over time,” ConocoPhillips Chief Information Officer, Mike Pfister, stated in 2017.

Due to ConocoPhillips’s use of big data enabled, in 2017 the company pumped 15.5 million lb/well of proppant, compared with 7.5 million lb/well and 3.8 million lb/well in 2014 and 2012 respectively. “Our completion design continues to evolve, but analyzing the data over time has maximized return from our investments in the Eagle Ford,” ConocoPhillips Chief Technology Officer Greg Leveille stated.

Big data is also used in upstream operations to ensure machinery is working effectively and to minimize disruptions. Machinery used in drilling has to operate in harsh conditions for prolonged periods of time and can be susceptible to wear and damage. The machinery is fitted with sensors collecting information on performance, which is then compared with aggregated data. This data, livestreamed to engineers, means that parts can be replaced efficiently and downtime is minimized, reducing overhead production costs and saving time in identifying problematic equipment. On average, the oil and gas industry experiences up to 10% operational downtime due to unanticipated equipment failure, three times the US industrial average. The cost of an offshore well going out of commission is roughly $7 million per day.

The live streaming of complex real-time sensor data – only possible through big data technology – can help predict the success rate and environmental risks of drilling operations. Real-time weather data, combined with drilling operation data, mitigates the risks of dangerous conditions for workers. In addition, data from continuous pipeline monitoring can warn of oncoming earthquakes and help in making the decision to shut down operations.

Big Data in the Midstream and Downstream Sectors

Big data analysis can bring positive benefits to the midstream sector; particularly regarding transportation, maintenance, and refining. Transporting oil and gas can pose several problems to companies; chief among them is ensuring that equipment used is in suitable condition and can transport oil and gas at the lowest possible risk. Big data can provide ways for companies to actively monitor pipelines and other infrastructure used for transportation. For example, sensors placed on pipelines can detect abnormal stress levels, allowing the company to pre-emptively take them offline, preventing accidents and other disruptions caused by damaged machinery.

The process of refining also benefits from big data analysis. Logilube, a technology development company, uses predictive data analytic solutions for oil and gas compression. Its SmartOil system can track any change in the manually adjusted lubrication rate of refineries, ensuring compressor efficiency. “A drop of just 0.5% in efficiency can cost a natural gas compression operation $180,000 per year in lost revenue,” Logilube chief executive, Bill Gillette, stated in a 2017 press release.

Despite the range of uses for big data in the downstream sector, it is currently used much less in downstream than in upstream and midstream operations.

Brun says that the prospect of its implementation in the downstream sector garners both “strong interest and strong skepticism.”

Downstream data analysis can serve two functions; in marketing, and in trading. For marketing, big data draws off retail and marketing information to better understand the customer. Electronic payment systems provide large data banks from which to extrapolate customer trends. There are opportunities in running analytics on retail websites, and through aggregating and analyzing social media posts to improve sales and end-consumer retail operations.

However, the oil and gas industry has to look to the banking and retail industries for successful use cases of consumer based big data – the potential is in optimizing the last stage of the hydrocarbon value chain, leveraging consumer data to optimize pricing and product visibility.

Through modeling commodities markets with data analytics, oil and gas companies can forecast trends, predict regulatory changes in trading, and gain market insights rivaling those of dedicated financial institutions. Greater market insight can also affect operations higher up the supply chain; market analysis can help determine the best sale strategy for fuel vessels – whether to change the selling point of a cargo or maintain possession waiting for a better opportunity given the market state at the time.

Big Data and Business Models

The implementation of big data predictive analysis has ramifications for the structure of operations for entire companies. “Big data enables new ways of managing projects and new ways of operating leading to agile operations,” Brun stated.

“Gone are the days when information and technology was a centralized headquarter function. Today, it permeates across the organization and decision-making has moved from a hierarchical top-down flow into fluid mechanisms through lean teams and methodology,” he continued.

Big data analytics offers companies the opportunity to integrate all their disparate systems and departments through data consolidation, and make informed top-down decisions to improve performance. Having a centralized database for all sectors including production, revenue, transportation, marketing, and human resources, provides the platform for statistical analysis, modeling and insight into company-wide performance. The ability to extract trends and make predictions across different operations throughout the industry can give a company the edge needed to generate the most value out of tight profit margins.

“Some executives believe that the oil and gas operating model is about to change fundamentally,” Brun asserted. “It will take several years to get there and it is likely to be more of a linear development than abrupt and dramatic,” he added.

Drawbacks and Challenges in Implementation

According to McKinsey research, the oil and gas industry only generates value from 1% of its data. A paper from Orbis Research states that only 36% of oil and gas companies have invested in big data and analytics, only 13% of which use the insights generated by the technology to enhance business intelligence. Partial use of big data technology and analysis, coupled with ignorance of the new technologies hitting the market are holding the industry back from capitalizing on the full potential of big data analytics.

Brun told Egypt Oil & Gas that the “application of advanced analytics is not even across all segments, and overall it is fair to say the oil and gas industry has been relatively slow in adopting the newest digital capabilities.”

Brun detailed four factors that discourage industry wide implementation of big data: first, building the “sufficient capabilities” to house the technology and drive transformative action; second, establishing an “aligned road map of priority change themes” to pinpoint the resources and areas which would benefit most from big data, companies often “fall into the ‘proof of concept trap’ in testing many digital use-cases, but not really scaling them”; third, ensuring “business led-transformation” where change come from the top down, not following the lead of IT departments; and fourth, “managing the trade-offs” of changing operating models in such a way, reallocating tasks to data analysis software while ensuring job creation on the administrative end.

Cybersecurity represents another concern for operators. “The dark side gathers more and more analytical skills” in contrast to its legal counterparts, Brun warned. “Connecting critical equipment and processes to networks, and storing critical company data in the cloud imposes new and potentially very large security risks.”.

Cooperation is Necessary

The successful exploitation of big data requires cooperation among areas of a business. For example, management and IT departments need to collaborate so that a consistent company-wide digital infrastructure can be developed. From the specifics of drilling and exploration operations to trends driving the upstream, midstream and downstream industries can all be extrapolated with advanced analysis to create value and map business strategies.

“Big data can plot trends and probabilities on infinite parameters,” Brun stated. “From geological precision to extracting more out of existing assets, data had always been the key to decision-making, both for larger investments and operational decisions.”

“Big data can put companies on the path to sustainable growth, and in turn lead to greater return on investment for capital, creating a virtuous circle of positive investment environments and continued innovation.”

However, it is important to understand that digital transformation is not only about technology; people and governance are critical success factors. Transformation requires people to work very differently and be much more agile to reap the benefits of big data and advanced analysis.