Data continuity¶
How completely each signal was recorded: where its gaps are, how long its unbroken stretches run, how it was packaged into messages, and whether a break in the stamps lost data or only moved the clock.
| Techniques | Cumulative duration curves, Remainder test, Junction scores, Null controls |
Studies¶
| Study | Date | Status | Summary |
|---|---|---|---|
| Gaps and unbroken segments in every waveform | 2026-10-19 | draft | Every gap and every unbroken segment in the 6,388 recordings, per waveform. The curves are not smooth: a cliff at about 18 s in every device marks the recorder's pauses, SNUADC segments stop at 5 hours, Primus segments at about 2.0 hours and BIS segments at about 2.3 hours (each device's correction cycle), and BIS's short gaps grow with the time since the last one (clock resynchronisations, not lost data). |
| When each device's clock is corrected, in every recording | 2026-10-19 | draft | From the steps between records in all 6,388 recordings: the BIS is corrected every 8,192 s and the Primus every 7,230 s, on their own timers (the first correction comes at any time after the device starts); the SNUADC is reset when 0.49 s of drift has built up. With the full rerun of the correction fit, it gives each device's drift against the recorder PC in 4,728 BIS and 2,728 SNUADC recordings, and shows that the Primus's 20–23 s AWP pauses belong to its correction cycle. Every rate here is relative to the recorder PC, whose own clock drifts and is corrected. |
| Message sizes and the steps between records | 2026-10-19 | draft | Every record in every waveform: SNUADC writes messages of 100 samples (0.2 s), Primus 30 (0.48 s), BIS 16 (0.125 s), CardioQ no fixed size. Backward steps outnumber gaps by about a hundred to one and come from a few recordings with duplicated or interleaved records. Primus gaps are whole lost messages; most SNUADC and BIS gaps are not. |
| What each break in the stamps is | 2026-10-19 | draft | Every step between waveform records in the release, sorted by mechanism: written twice, a SNUADC clock reset, interleaved streams, a pause shared by every device, whole messages, less than a message, or a fraction of a message beyond whole ones. A continuity test with controls shows which classes lose data: shared pauses and most longer SNUADC gaps do; steps under one message, and SNUADC's lone one-message steps, mostly do not. |
Observations from these studies¶
| Observation | Status | Summary |
|---|---|---|
| Eight recordings store whole messages twice | established | In 8 recordings 15–48% of the waveform records are byte-for-byte copies of the record before them, with the same timestamp: case 15, case 364, case 1946, case 3865, case 4857, case 5414, case 5595 and case 5619. SNUADC, Primus and BIS are affected equally, so the recorder, not a device, wrote them twice. |
| Recording pauses of about 20 s | established | In 29% of files every device jumps forward together, by a median 19.6 s: the recorder paused. Data are missing there, and the stamps show it honestly. The pauses run on a 5-hour timer: consecutive pauses in a recording are 5 h apart (plus the pause) in 86% of cases, and the first comes at any time after the recording starts. |
| The Primus AWP stops for about 21 s every 7,230 s | established | In 5,502 recordings the Primus airway-pressure waveform has gaps of 19–23.5 s (whole messages of 0.48 s, mostly 21.12 s) that recur every 7,230 s, the Primus's correction interval; a clock correction follows about 21.6 s after. The CO₂ waveform carries on through 90% of them. The 21 s of AWP are lost, not merely re-stamped. |
| In some recordings two unrelated streams are interleaved in one track | open | In 7 recordings 64–95% of the records alternate between two streams: each record continues the one two before it, not the one before, and the two streams do not match each other. SNUADC and BIS in case 2219, case 2379, case 2640, case 4836, case 5327 and case 5617; SNUADC in case 1939. In case 94, case 945, case 1132, case 3195 and case 5700 4–7% of SNUADC records interleave. |
The approach¶
The idea. A recorder writes each signal as a run of messages (records), each a fixed number of samples with one time stamp. Between two records the stamps should advance by exactly one message. Any other step is either data lost, data written twice or out of order, or a clock correction that moved the stamps without losing a sample. Each of these leaves its own signature, and the signatures can be told apart from the stamps and the samples alone.
What it gives. For every waveform: its message size, the distribution of its gaps and of its unbroken segments, which gaps are system events shared by every device, which are lost whole messages, which are clock resynchronisations, and which recordings carry duplicated or interleaved records.
How. Distributions are shown as cumulative counts (the number of gaps or segments longer than x) on log–log axes: losses that strike at random give a smooth curve, and a cliff or step marks a mechanism with its own duration. Segments cut short by the end of a recording are censored, not ended by a gap. Gaps are folded modulo the message length; whole lost messages pile up at zero. Gap length against the time since the previous gap separates resynchronisations, which grow with the elapsed time, from losses, which do not.
Pitfalls. Backward steps outnumber forward ones by a hundred to one in some signals and are invisible to a gap rule that only looks forward. One recording with a defect can dominate a count across the release, so shares are reported per recording as well as in total.
Techniques¶
- Cumulative duration curves: Plot the number of gaps or segments longer than x against x on log–log axes, with segments cut by the start or end of a recording treated as censored; cliffs and steps mark mechanisms with a duration of their own.
- Remainder test: Fold each gap modulo the signal's message duration: whole lost messages pile up at zero, while clock corrections and partial losses spread over the whole interval.
- Junction scores: Score the step across a record junction against a linear extrapolation of the samples before it, scaled by the signal's ordinary sample-to-sample steps.
- Null controls: Repeat each test where the effect cannot be: at random times, at the same interval later, sign-flipped, or between two channels of one device. The effect is real only where it beats its control.