Investment banks, many of which finance new projects, along with oil majors such as Total and Eni, have warned that huge spending cuts caused by a plunge in oil prices since 2014 would lead to a supply crunch in the next two years.

Yet Goldman Sachs, the only bank to make more than $1 billion a year from commodities trading, believes a looming recovery in U.S. output on the back of higher oil prices combined with an avalanche of new conventional projects will create a substantial surplus by 2019.

Prior to the shale revolution, conventional oil was the only game in town. Estimating future supply essentially involved calculating the project pipeline and factoring in the "unknown knowns" such as political risk in oil-producing nations.

The ability of the shale sector to adapt quickly and nimbly to a lower-price environment means production cycles have shortened as fields can be switched on and off in a matter of weeks.

Most forecasters including OPEC and the International Energy Agency underestimated shale's decline during the oil price collapse and its production increases as prices recovered.

Goldman predicts the coming two years will see a huge burst of development, complicating OPEC's efforts to rebalance the market and ease a global glut with the help of output cuts.

"This long lead-time wave of projects and a short-cycle revival, led by U.S. shales, could create a material oversupply in 2018-19," Goldman's equity research team said last month.

"As OPEC prepares for its May 25 meeting, it is likely to weigh the relative benefit of stability (extend cut) versus the risk of long-term share loss."

Goldman estimates that new projects and rising shale output could add 1 million barrels per day (bpd) to global supply by 2018-2019.

The forecast contrasts with those of consultancy Wood Mackenzie, which foresees a supply gap of 20 million bpd by 2025, and Goldman's rival Morgan Stanley, which believes a surge in U.S. production this year will not derail the rebalancing.

"OPEC has successfully constrained output, and although drilling activity in U.S. shale is picking up rapidly, this will probably not come quick enough to prevent a period of sizeable inventory draws late this year," Morgan Stanley said.

"By 2020, we estimate that (around) 1.5 million bpd of demand will need to come from projects that have not been sanctioned yet, but that have break-even oil prices of $70-75 a barrel," the bank said.

BLACK HOLES

Goldman advises anyone from institutional investors such as pension funds, to oil producers and it seems the oil market is listening.

Brent crude futures show prices for oil deliverable up to 2019 trading below those for prompt delivery, before reverting to the contango structure of low prompt prices and higher futures prices that is typical of an oversupplied market.

Goldman stands by its prediction that supply and demand will fall into line this year, even though global crude inventories in developed economies alone top 3 billion barrels, some 300 million barrels above the five-year average that OPEC is targeting with its supply cuts.

The Organization of the Petroleum Exporting Countries and some of its biggest rivals including Russia, agreed in late2016 to cut output jointly by 1.8 million bpd for the first half of this year to tackle the overhang.

UBS, meanwhile, sees a potential 4 million bpd hole by 2020, even though a higher crude price this year has prompted some companies to bring forward their exploration and development plans.

"Beyond 2017, the impact of a collapse in longer-cycle conventional investment over 2014-16 begins to be felt. 2015 saw just six major upstream projects totaling (some) 0.6 million bpd ... versus the 3-4 million bpd average, and 2016 has seen just one major liquids project sanctioned," UBS strategist Jon Rigby said.

Bank of America-Merrill Lynch points out that along with the collapse in spending, the global rig count, a measure of production activity, shows no sign of picking up outside the United States.

According to oil services company Baker Hughes, the number of non-U.S. oil rigs has risen by just 29 since hitting an 11-year low of 666 in November last year, compared with a rise of 346 in U.S. rigs in just 10 months.