CalSurprise/README.md

# CalSurprise

An economic calendar release read as a number rather than a category: actual
minus forecast, standardised by each indicator's own dispersion, anchored to
the minute bar the market actually reacted in, and regressed against price to
measure what the event does to a symbol.

Companion code for the MQL5 article: https://www.mql5.com/en/articles/24850

## What it does

A calendar filter usually asks whether news is due. That treats every CPI
print as the same object, when the thing the market trades is the gap between
the number and what was expected of it. A release of 0.4% is enormous where
consensus was 0.1% and unremarkable where consensus was 0.4%, and the two are
indistinguishable to anything keyed on the event id alone.

The gap itself is not comparable across indicators. Nonfarm Payrolls surprises
are measured in thousands of jobs, Core CPI in tenths of a percent, and the
ISM surveys in index points, so `CCalendarSurprise` divides each one by the
dispersion of that indicator's own past surprises. The result is a z, and a z
of 2 means the same thing on every event in the table. Three standardisers are
available; the expanding and robust ones look only at releases before the one
being scored, so a z printed on the day carries no look-ahead.

Two things have to be right before any of that is trustworthy, and both were
found the hard way. The first is that the terminal returns calendar timestamps
in GMT until it connects and in server time afterwards, flipping silently mid
run. On a GMT+3 server that is a 180 minute error with nothing logged, so
`CalendarProbe` gates every reader on the connection flag plus two matching
back-to-back reads. It holds no state between calls, which is what lets an
indicator use it from `OnCalculate` where `Sleep` is refused.

The second is daylight saving. A historical stamp expressed with today's
server offset lands an hour from its own bar whenever the release sits across
a changeover. `CCalendarClock` recovers the broker's schedule from price
instead of assuming it: each release votes for the offset of the largest
absolute minute return near its stamp, but only when that move clears eight
times the median minute of its window, since a quiet print has nothing to say
about where the clock sits. The votes are pooled into one twelve-month table.
Pooling is the part that matters. An earlier version calibrated per event and
was visibly wrong, because one event offers about nine releases a month and
the two ISM events ended up disagreeing about which months shift despite
releasing on the same clock. Alignment also needs no forecast, only an actual,
so the voting pool is larger than the pool usable for standardisation.

Anchoring is worth more than any modelling choice here. Over eleven US events,
four symbols and four horizons, the number of cells clearing |t| >= 1.96 rises
from 16 to 49 of 176 when the offset is applied, and mean |t| rises from 0.91
to 1.41. Nothing else changed between those two runs.

The consensus feed also carries forecasts that cannot be real. Among the
payrolls rows is a consensus of 78,356 thousand jobs, against a median
absolute surprise of 121. A median and MAD scale stops rows like that
inflating sigma, but it does not remove them, and left in they arrive at
several hundred sigma and one of them alone dominates a regression. Both the
robust scale and an explicit impossible-value gate are needed; discovering
that took three rounds of an engine that reported everything as flat.

What the measurement finds is mixed, and the negative half is the more
interesting one. Core CPI m/m is significant on all four symbols, with the
sign correctly inverting on USDJPY where the dollar is the base currency:
-8.50 pips per sigma on EURUSD at t = -4.33, -9.54 on GBPUSD, +12.16 on
USDJPY, -196.6 on XAUUSD. Nonfarm Payrolls, the most watched release on the
calendar, is flat on every symbol and every horizon tested, with a maximum
|t| of 1.46, while still moving EURUSD 21.5 pips on average. It moves price
and the direction is not predictable from the surprise. That result survived
all three data fixes above, so it is reported as a finding rather than
explained away.

There is no Expert Advisor here and no claim about returns. The library
measures a response and says when it cannot find one.

## Layout

```
Include/CalSurprise/CalSurpriseTypes.mqh   shared vocabulary: event, release and response structures, standardiser and rejection enums
Include/CalSurprise/CalSurpriseClock.mqh   CCalendarClock: the broker's daylight saving schedule recovered from price by pooled voting
Include/CalSurprise/CalSurpriseCore.mqh    CCalendarSurprise: settling gate, first-print loading, robust standardisation, response regression
Scripts/CalSurprise/SurpriseLab.mq5        the study: calibrates the clock, then writes the clock table, per-event summary and response table as CSV
Indicators/CalSurprise/SurpriseMeter.mq5   separate-window histogram of the standardised surprise per bar, magnitude and event id exposed for iCustom
Indicators/CalSurprise/SurprisePanel.mq5   chart panel of upcoming releases with their measured response profile, dimming rows whose band spans zero
```

Run `SurpriseLab.mq5` first. It writes `surprise_lab.csv` to the common files
folder and reproduces every number quoted above. Its `InpMode` default must
stay `SURPRISE_STD_ROBUST`: the expanding mean and standard deviation run is
materially weaker on the same data, Core CPI on EURUSD coming out at t = -2.40
rather than -4.33, because the impossible forecasts are still inflating sigma.
Setting `InpAnchor` to false disables the clock and reproduces the raw-stamp
comparison.

Substituting your own events is a single edit. `EventIds[]` in `SurpriseLab`
and `InpEvents` on both indicators are plain lists of calendar event ids, and
nothing downstream knows what an event means. The eleven shipped are the US
releases carrying both an actual and a forecast since May 2017, which is where
the terminal's forecast history begins; rate decisions carry no forecast at
all and cannot be used. Horizons are `InpHorizons`, in minutes.

## Disclaimer

Educational code. This measures whether a release moves a symbol, which is not
the same as a way to trade it: the significant cells describe an average
response over a sample, not an entry, and nothing here accounts for the spread
widening and execution risk around a release. The most famous event on the
calendar comes out unmeasurable. Test on your own data, broker and server
clock before drawing conclusions, since the alignment the library recovers is
a property of the broker you run it against.