I re-ran a 15-year backtest and 85.6% became 53.8%

Some option contracts settle on the morning of expiration rather than at the close. On SPX these are the AM-settled monthlies, and they settle against SET, the exercise-settlement value built from each component's opening print.

You cannot manage a position like that on expiration day. One print on Friday morning ends it. So the only question that matters is how far that print travels from Thursday's close. If I knew that distance, I could judge whether carrying the position overnight was defensible. That was the premise.

So I backtested it.

---

1. The rule I built

Fifteen years · 174 expiration Fridays.

Thursday close → settlement print

  median gap        0.113%   (about 9 points at an index of 7,600)
  inside ±30pt      85.6%
  inside ±50pt      94.8%
  directional bias  none (t=+1.34 · p=0.181)

And I wrote it down as one line.

"The P&L of an AM-settled monthly is effectively fixed at Thursday's close, not on Friday morning."

That is what the numbers said. If the median gap is 9 points and 85.6% of cases land inside ±30 points, then a strike 30 points away from Thursday's close read as safe at least five times out of six. That was the reading.

I put that rule into the market. I opened a call credit spread with the short 28 points above Thursday's close and carried it overnight.

2. The result

The settlement printed 46.77 points above Thursday's close.

2026-08-21 expiration

Thursday close   7,641.16
SET              7,687.93      =  +46.77pt (+0.612%)

Outside the ±30 band. It settled at the maximum loss.

On its own that could simply be the one-in-six day. 85.6% means 14.4% goes the other way, and a single session settles nothing.

So before interpreting the result, I opened the backtest again.

3. 🚨 Much of what I called the «open» was not an open

The backtest used the open column from a free daily index series as a stand-in for the settlement print. I counted that column, row by row.

share of rows where Open == prior Close

  1998 – 2004     96.7%
  2011 – 2015     12.3%

  mean |Open − prior Close| (full sample)     1.05 points

For 1998–2004 there is effectively no open at all. The prior close is sitting in that field.

My sample reaches back into that era, so a large share of it was computed on synthetic opens. And this contamination is not random noise — it pushes gaps toward zero. If the open equals the prior close, the gap is zero by definition.

My backtest concluded "the gaps are small" partly because a chunk of its data had no gap in it.

⚠️ I have not established the cause. Whether whoever assembled the series back-filled the field, or the underlying source genuinely did not publish an index open for that era, I still do not know. What I know is that the column cannot be used as-is.

4. Re-run against the actual settlement prints

There was never a reason to use a proxy here. Cboe publishes SET, and 28 years of it are available. I re-ran everything against that.

	Old backtest (proxy open)	28 years of actual SET
"Expiration-Friday gaps are smaller than those on ordinary Fridays"	significant	t=+0.39 · p=0.70 — does not replicate
Median gap	0.113% (≈9pt)	0.346% (≈26pt)
Inside ±30pt	85.6%	53.8%

53.8% is a coin flip.

The sentence I wrote — "effectively fixed" — turns out to be true a little more than half the time. And the median gap is not 9 points but 26 — nearly three times wider.

I cut the rule's confidence from 0.85 to 0.15.

5. The data retired the rule, not the loss

This is the part I want on the record.

the position settled at max loss   →  one day's result. It proves nothing
much of the data was synthetic     →  the justification itself is gone

Those two collapsed independently. The first did not cause the second.

Had that position won, I would never have opened the data. That is the uncomfortable part. I would have written "85.6% held again," and carried the contaminated justification into the next monthly cycle unchanged.

The loss was the cheaper half. Winning would have cost me the look.

6. How to check this in your own data

All I did was compare two columns.

same_as_prior = df["Open"].eq(df["Close"].shift(1))

# whole sample
same_as_prior.mean()

# split by year — the answer differs by era
same_as_prior.groupby(df.index.year).mean()

⚠️ Compute the shift once, on the full series, before grouping. If you shift inside each year's group, the first row of every year gets a NaN prior close, counts as a mismatch, and every year comes out slightly too low — in a check about data hygiene, that would be its own small version of the same mistake.

Splitting by year is the part that matters. A single average buries it. In my case 2011–2015 came in at 12.3%, which disappears into a full-sample number — but 1998–2004 was 96.7%.

If floating point gets in the way, test (df["Open"] - df["Close"].shift(1)).abs() < 0.01 instead.

What this contamination reaches

Anything judged on the open.

Gap strategies	the gap itself is compressed toward zero
Open-to-close (green/red days)	with Open == prior Close it silently becomes close-to-close
Opening range	the starting point is synthetic, not a real open
AM-settled contracts	my case
Overnight holds	the overnight move disappears

Close-only backtests are generally safe. Nobody has a reason to back-fill a close.

7. A separate lesson — the settlement print is not the open

This one has nothing to do with data quality. I simply had it wrong from the start.

An AM-settled monthly does not settle on the index open. It settles on a separately calculated value built from the first trade in each component on expiration morning. It is not the same number as the index open.

2026-08-21 expiration

  index open        7,665.68
  actual SET        7,687.93
  ────────────────────────────
  difference        22.25 points

Measured against the index open, my spread was profitable. Measured against the actual settlement, it was the maximum loss. Same morning, same position, different number.

I should have known this before the backtest, not after. When I substitute a proxy, the first thing to measure is how far it strays. I skipped that, on the assumption it would be close enough.

📌 What that value actually is, and why it parts company with the open, is written up separately — AM settlement lands on a third number. Checking it against an exchange study of 33 quarterly settlements, 22 points turns out to be an ordinary size — and it can land at a price the index never traded that day.

8. What is still open

Several other analyses were built on the same source. All of them need re-checking.

Fixing only this one leaves the same mistake alive under a different name — and next time I would be less likely to look, because I would consider the method already vetted.

---

Closing

I have written once about a free source differing from the official data by 38%, and once about a 100% win rate that turned out to be a scoring bug. This is the third, and it is a different species from the first two.

Both of those I looked into because the number looked wrong. 38% is conspicuous. So is a 100% win rate.

This time nothing looked wrong at all. A median gap of 0.113% is a plausible number. 174 observations reads like an adequate sample. The significance test passed. If the trade had not forced the question, it would have gone on looking plausible indefinitely.

So the lesson is not "verify your data." It is "the more plausible the result, the more it is worth counting the raw rows once."

Checking a number that looks strange is something everyone does. Checking one that looks good is the hard part.

---

※ This is a record of how I validated a rule of my own. It is not a recommendation of any trading method or security. Any decision you make, and its outcome, is your own.