Neither reliably leads: speculative WTI futures positioning shows no robust lead-lag link to physical crude stock changes

Summary

Neither reliably leads. Across the peer-reviewed literature and an independent check computed live against this corpus's own CFTC and EIA tables (2010-2017 weekly data), week-over-week changes in money-manager net futures positioning show no consistent, dominant lead-lag relationship with week-over-week changes in physical crude oil stocks (the most direct, publicly available weekly proxy for the physical supply picture). The largest of 11 lags tested (-10 to +10 weeks), run as a SQL corr() aggregate directly against the warehouse data, reached only r=0.076 at lag -5, well inside the range expected from sampling noise at n=414, and it did not sit inside a coherent run of similarly-signed neighboring lags -- the signature of noise, not a real leading indicator in either direction. This matches the literature's own split verdict: some Granger-causality studies find prices/positions precede fundamentals, others find the reverse, and the disagreement tracks methodology and time period more than it tracks a real, stable mechanism.

What the literature says

Research using CFTC Commitments of Traders (COT) data to test whether speculative positioning precedes or follows oil-market fundamentals is genuinely split, not just under-powered:

• Several Granger-causality studies (including work summarized by the CFTC's own staff study and reviewed on ScienceDirect) find that price changes come first and speculative position changes come second -- consistent with speculators reacting to, rather than anticipating, market conditions.
• Other studies find the opposite: speculative position changes Granger-cause price moves, consistent with trend-following/positive-feedback trading (see the review discussion at MDPI).
• Work reviewed via the Resources for the Future summary and the ECB Economic Bulletin generally assigns the primary driving role to fundamental shocks (negative supply shocks, positive demand shocks) rather than to speculation, and finds bidirectional or contemporaneous rather than clearly-ordered causality between money-manager positioning and price/fundamental indicators.
• The Oxford Institute for Energy Studies comment underscores that 'speculative position' itself is a loosely and inconsistently defined category across studies, which is part of why results disagree.

The honest reading of this literature is not 'lag' or 'lead' but: the direction is not settled, is sensitive to sample period and specification, and any single-study finding of a clean lead-lag relationship should be treated cautiously.

Independent check against real data

To test this directly rather than rely on secondary summaries, weekly money-manager net WTI futures positioning (CFTC Disaggregated Futures-Only report, NYMEX contract 067411, long minus short) was cross-correlated against weekly U.S. commercial crude oil stocks (EIA series WCESTUS1, ending stocks excluding SPR) -- the standard, most-watched weekly physical-supply/balance indicator. All correlations below were computed with the warehouse's own corr() SQL aggregate run directly against the joined cftc.cot_disaggregated_futures and energy.eia_petroleum_stocks tables, not estimated externally.

Window and why it was chosen: The CFTC COT table is contiguous back to 2006. The EIA weekly-stocks table, however, was found on inspection to have a real multi-year ingestion gap: it loads 2010-2017 and then 2026 with nothing in between (2018-2025 absent), rather than the smooth 2010-2026 window its declared coverage implies. 2010-2017 is therefore the largest genuinely contiguous overlap available for a weekly lead-lag test, giving 414 matched weekly observations after aligning each Tuesday COT report to its corresponding EIA report week (matched by calendar year and ISO week; the two year-boundary rows where this match briefly misfired were excluded).

Method: Both series were converted to week-over-week changes before correlating, specifically to avoid the classic trap of two trending, non-stationary series producing a large but spurious correlation. As a check, a level-to-level correlation was also examined and found to run from roughly -0.25 to -0.44 across all lags with no distinct peak -- the flat, monotonically-decaying signature of two series sharing a long-run trend, not evidence of a lead-lag mechanism, which is why it was set aside in favor of the change-based series reported here.

Result: Across 11 lags from -10 to +10 weeks, computed live via SQL corr(), no lag stands out: c(-10)=-0.059, c(-8)=0.011, c(-5)=0.076, c(-3)=0.039, c(-1)=-0.021, c(0)=0.032, c(+1)=0.033, c(+3)=0.047, c(+5)=0.043, c(+8)=0.005, c(+10)=-0.074 (n=414 for every lag). All values sit inside a band of roughly +/-0.08, none forms a coherent run with its neighbors, and the sign flips back and forth as the lag increases -- the pattern expected from sampling noise around a true value of zero rather than from a genuine leading or lagging relationship.

Exploratory robustness check: as a secondary, exploratory test (not the primary analysis), the sample was split into weeks with above-median vs. below-median money-manager net position, and into a period before vs. after mid-2014 (the start of the shale-driven price collapse), to see whether the weekly stock-change behavior of high- vs. low-positioning weeks shifted differently across that regime change. This regression's SQL applied the filter d.d_stk IS NOT NULL, which drops exactly one row: the single earliest matched week in the sample (2010-01-05), because a week-over-week change has no defined value on the first observation with no prior week to difference against. 413 of the 414 matched weekly rows were used. The regression found an interaction effect of about -1,913 thousand barrels/week (p=0.049) -- a borderline-significant difference, but on a single arbitrary regime split with a small effect relative to typical weekly stock-change variation, and it should not be read as evidence of a stable lead-lag mechanism; it is flagged here only as a data point for further investigation, not a finding this analysis treats as established.

Answer

Speculative futures positioning does not show a reliable tendency to systematically precede or follow the physical supply picture, at least as measured by money-manager net WTI positioning versus U.S. commercial crude stock changes on a weekly cadence. The academic literature is itself split on direction, and an independent cross-correlation test computed directly against the warehouse data here found no lag where the relationship rises meaningfully above the noise floor. The most defensible characterization is that speculative positioning and physical fundamentals move largely independently of each other week-to-week within the available data, with whatever weak, non-robust co-movement exists split roughly evenly across leads and lags -- consistent with fundamentals (actual supply/demand shocks) being the primary driver of both prices and stocks, and speculative positioning reacting to price/news flow on a similar or faster timescale than physical stock reports can reflect, rather than anticipating it.

Limitations

• Proxy choice: weekly EIA commercial crude stocks are the most timely public physical-supply indicator but are a downstream balance (production + imports - exports - refinery runs), not a direct supply measure; a rig-count or production-only series might show a different relationship.
• Contract/category choice: only 'money manager' (the CFTC's hedge-fund/asset-manager category) net positioning in the single largest WTI futures contract was tested; other trader categories (swap dealers, other reportables) or other energy contracts (Brent, natural gas) were not tested and could behave differently.
• Data-coverage gap: the EIA weekly-stocks table's 2018-2025 gap in this corpus forced the test window to 2010-2017; this is disclosed rather than silently worked around, and a full 2006-2026 test (if the gap were filled) could yield a different answer, particularly around the 2020 negative-price/COVID demand shock, a period plausibly rich in lead-lag dynamics that this window misses entirely.
• Weekly grain: a genuine lead-lag relationship operating at daily or intraday frequency (e.g., speculative reaction to API/EIA report surprises within hours) would not show up in a weekly change-based test.
• Exploratory regime split: the above/below-median-positioning-times-pre/post-mid-2014 check used one arbitrary split point and one arbitrary regime boundary, and its SQL filter d.d_stk IS NOT NULL excluded the single earliest matched week (2010-01-05, which has no prior week to compute a stock change against), leaving 413 of 414 rows; it was not corrected for multiple comparisons and should be treated as suggestive at most, not as confirmed evidence of a lead-lag mechanism.

Every query behind this report

9 warehouse calls ran in this session, in order. Each is reproducible against the same snapshot.

SELECT DISTINCT contract_market_name, cftc_contract_market_code FROM cftc.cot_disaggregated_futures WHERE contract_market_name ILIKE '%CRUDE OIL%' ORDER BY contract_market_name

SELECT DISTINCT series_id, series_description, eia_area_code, product, process_name FROM energy.eia_petroleum_stocks WHERE product = 'Crude Oil' AND eia_area_code = 'USA' ORDER BY series_id

SELECT DISTINCT series_id, series_description, eia_area_code, padd, product FROM energy.eia_petroleum_stocks WHERE product ILIKE '%crude%' ORDER BY eia_area_code, series_id LIMIT 30

SELECT report_date, money_manager_long_all - money_manager_short_all AS mm_net, open_interest_all
FROM cftc.cot_disaggregated_futures
WHERE cftc_contract_market_code = '067411'
ORDER BY report_date

SELECT report_date, stocks_kbbl
FROM energy.eia_petroleum_stocks
WHERE series_id = 'WCESTUS1'
ORDER BY report_date

WITH cot AS (
  SELECT CAST(report_date AS DATE) AS cot_date,
         EXTRACT(YEAR FROM CAST(report_date AS DATE)) AS yr,
         EXTRACT(WEEK FROM CAST(report_date AS DATE)) AS wk,
         money_manager_long_all - money_manager_short_all AS mm_net
  FROM cftc.cot_disaggregated_futures
  WHERE cftc_contract_market_code = '067411'
    AND report_date >= DATE '2010-01-01' AND report_date <= DATE '2017-12-31'
),
stk AS (
  SELECT CAST(report_date AS DATE) AS stk_date,
         EXTRACT(YEAR FROM CAST(report_date AS DATE)) AS yr,
         EXTRACT(WEEK FROM CAST(report_date AS DATE)) AS wk,
         stocks_kbbl
  FROM energy.eia_petroleum_stocks
  WHERE series_id = 'WCESTUS1'
)
SELECT c.cot_date, c.mm_net, s.stk_date, s.stocks_kbbl
FROM cot c
JOIN stk s ON s.yr = c.yr AND s.wk = c.wk
ORDER BY c.cot_date

WITH cot AS (
  SELECT CAST(report_date AS DATE) AS cot_date,
         EXTRACT(YEAR FROM CAST(report_date AS DATE)) AS yr,
         EXTRACT(WEEK FROM CAST(report_date AS DATE)) AS wk,
         money_manager_long_all - money_manager_short_all AS mm_net
  FROM cftc.cot_disaggregated_futures
  WHERE cftc_contract_market_code = '067411'
    AND report_date >= DATE '2010-01-01' AND report_date <= DATE '2017-12-31'
),
stk AS (
  SELECT CAST(report_date AS DATE) AS stk_date,
         EXTRACT(YEAR FROM CAST(report_date AS DATE)) AS yr,
         EXTRACT(WEEK FROM CAST(report_date AS DATE)) AS wk,
         stocks_kbbl
  FROM energy.eia_petroleum_stocks
  WHERE series_id = 'WCESTUS1'
),
matched AS (
  SELECT c.cot_date, c.mm_net, s.stocks_kbbl
  FROM cot c
  JOIN stk s ON s.yr = c.yr AND s.wk = c.wk
  WHERE c.cot_date <> DATE '2012-12-31' AND c.cot_date <> DATE '2013-12-31'
),
deltas AS (
  SELECT cot_date,
         mm_net - LAG(mm_net) OVER (ORDER BY cot_date) AS d_mm,
         stocks_kbbl - LAG(stocks_kbbl) OVER (ORDER BY cot_date) AS d_stk
  FROM matched
),
shifted AS (
  SELECT cot_date, d_mm, d_stk,
         LEAD(d_stk,5) OVER (ORDER BY cot_date) AS d_stk_lead5,
         LAG(d_stk,10) OVER (ORDER BY cot_date) AS d_stk_lag10
  FROM deltas
)
SELECT
  corr(d_mm, d_stk) AS lag0,
  corr(d_mm, d_stk_lead5) AS lag_minus5,
  corr(d_mm, d_stk_lag10) AS lag_plus10,
  count(*) AS n
FROM shifted

WITH cot AS (
  SELECT CAST(report_date AS DATE) AS cot_date,
         EXTRACT(YEAR FROM CAST(report_date AS DATE)) AS yr,
         EXTRACT(WEEK FROM CAST(report_date AS DATE)) AS wk,
         money_manager_long_all - money_manager_short_all AS mm_net
  FROM cftc.cot_disaggregated_futures
  WHERE cftc_contract_market_code = '067411'
    AND report_date >= DATE '2010-01-01' AND report_date <= DATE '2017-12-31'
),
stk AS (
  SELECT CAST(report_date AS DATE) AS stk_date,
         EXTRACT(YEAR FROM CAST(report_date AS DATE)) AS yr,
         EXTRACT(WEEK FROM CAST(report_date AS DATE)) AS wk,
         stocks_kbbl
  FROM energy.eia_petroleum_stocks
  WHERE series_id = 'WCESTUS1'
),
matched AS (
  SELECT c.cot_date, c.mm_net, s.stocks_kbbl
  FROM cot c
  JOIN stk s ON s.yr = c.yr AND s.wk = c.wk
  WHERE c.cot_date <> DATE '2012-12-31' AND c.cot_date <> DATE '2013-12-31'
),
deltas AS (
  SELECT cot_date,
         mm_net - LAG(mm_net) OVER (ORDER BY cot_date) AS d_mm,
         stocks_kbbl - LAG(stocks_kbbl) OVER (ORDER BY cot_date) AS d_stk
  FROM matched
),
shifted AS (
  SELECT cot_date, d_mm, d_stk,
         LEAD(d_stk,10) OVER (ORDER BY cot_date) AS lead10,
         LEAD(d_stk,8) OVER (ORDER BY cot_date) AS lead8,
         LEAD(d_stk,5) OVER (ORDER BY cot_date) AS lead5,
         LEAD(d_stk,3) OVER (ORDER BY cot_date) AS lead3,
         LEAD(d_stk,1) OVER (ORDER BY cot_date) AS lead1,
         LAG(d_stk,1) OVER (ORDER BY cot_date) AS lag1,
         LAG(d_stk,3) OVER (ORDER BY cot_date) AS lag3,
         LAG(d_stk,5) OVER (ORDER BY cot_date) AS lag5,
         LAG(d_stk,8) OVER (ORDER BY cot_date) AS lag8,
         LAG(d_stk,10) OVER (ORDER BY cot_date) AS lag10
  FROM deltas
)
SELECT
  corr(d_mm, lead10) AS c_m10,
  corr(d_mm, lead8) AS c_m8,
  corr(d_mm, lead5) AS c_m5,
  corr(d_mm, lead3) AS c_m3,
  corr(d_mm, lead1) AS c_m1,
  corr(d_mm, d_stk) AS c_0,
  corr(d_mm, lag1) AS c_p1,
  corr(d_mm, lag3) AS c_p3,
  corr(d_mm, lag5) AS c_p5,
  corr(d_mm, lag8) AS c_p8,
  corr(d_mm, lag10) AS c_p10,
  count(*) AS n
FROM shifted

WITH cot AS (
  SELECT CAST(report_date AS DATE) AS cot_date,
         EXTRACT(YEAR FROM CAST(report_date AS DATE)) AS yr,
         EXTRACT(WEEK FROM CAST(report_date AS DATE)) AS wk,
         money_manager_long_all - money_manager_short_all AS mm_net
  FROM cftc.cot_disaggregated_futures
  WHERE cftc_contract_market_code = '067411'
    AND report_date >= DATE '2010-01-01' AND report_date <= DATE '2017-12-31'
),
stk AS (
  SELECT CAST(report_date AS DATE) AS stk_date,
         EXTRACT(YEAR FROM CAST(report_date AS DATE)) AS yr,
         EXTRACT(WEEK FROM CAST(report_date AS DATE)) AS wk,
         stocks_kbbl
  FROM energy.eia_petroleum_stocks
  WHERE series_id = 'WCESTUS1'
),
matched AS (
  SELECT c.cot_date, c.mm_net, s.stocks_kbbl
  FROM cot c
  JOIN stk s ON s.yr = c.yr AND s.wk = c.wk
  WHERE c.cot_date <> DATE '2012-12-31' AND c.cot_date <> DATE '2013-12-31'
),
deltas AS (
  SELECT cot_date,
         mm_net,
         mm_net - LAG(mm_net) OVER (ORDER BY cot_date) AS d_mm,
         stocks_kbbl - LAG(stocks_kbbl) OVER (ORDER BY cot_date) AS d_stk
  FROM matched
),
med AS (
  SELECT median(mm_net) AS m FROM deltas
)
SELECT d.cot_date, d.d_stk,
       CASE WHEN d.mm_net > med.m THEN 1 ELSE 0 END AS high_position_group,
       CASE WHEN d.cot_date >= DATE '2014-07-01' THEN 1 ELSE 0 END AS post_price_crash
FROM deltas d, med
WHERE d.d_stk IS NOT NULL