Draft
Not yet written up in full.
Message sizes and the steps between records¶
Every record in every waveform: SNUADC writes messages of 100 samples (0.2 s), Primus 30 (0.48 s), BIS 16 (0.125 s), CardioQ no fixed size. Backward steps outnumber gaps by about a hundred to one and come from a few recordings with duplicated or interleaved records. Primus gaps are whole lost messages; most SNUADC and BIS gaps are not.
| Date | 2026-10-19 |
| Status | draft |
| Methods | Data continuity |
| Techniques | Remainder test |
| Clocks | Recorder PC clock, SNUADC sample clock, BIS clock, Primus clock |
| Devices | Vital Recorder, SNUADC, Primus anaesthesia machine, BIS monitor, CardioQ |
| Variables | ECG lead II (SNUADC), Plethysmogram (SNUADC), Arterial pressure (SNUADC), Capnogram (Primus), Airway pressure (Primus), BIS EEG |
Background¶
Each record carries one stamp and a block of samples. The next record should start where the last one ended; the step is the difference: step = start(k+1) − (start(k) + size(k) / rate). Positive steps are gaps, negative steps overlaps. A device that sends fixed-size messages loses data in whole messages, so a gap that is not a whole number of messages means a clock correction, or a loss with one.
Data¶
All 6,388 recordings; every waveform record (about 2.5 billion records). Run-length encoded record sizes and every non-zero step (5.2 million) were cached per recording.
Method¶
- Per track, the distribution of record sizes; the message is the modal size.
- Every step of 1 µs or more, forward and backward, histogrammed on a log scale.
- Remainder test: gaps longer than one message, folded modulo the message duration.
Results¶
- Message sizes. SNUADC: 100 samples at 500 Hz, 0.2 s (99.9% of records; a few 50-sample halves and about 300 rarer sizes). Primus: 30 samples at 62.5 Hz, 0.48 s. BIS: 16 samples at 128 Hz, 0.125 s (all but a handful). CardioQ: no fixed size (about 2,500 distinct sizes, median about 117 samples, 0.65 s).
- Backward steps dominate. SNUADC ECG II: 761,225 backward against 6,205 forward; BIS EEG: 1.05 million against 10,789; Primus AWP 101,805 against 20,397. They form a comb of discrete values from about 10 ms up to one message, and 98% of SNUADC's are in 27 recordings. Two mechanisms explain them: Eight recordings store whole messages twice (a whole message written twice with the same stamp: a step of exactly −1 message) and In some recordings two unrelated streams are interleaved in one track (two streams alternating in one track: steps in pairs that sum to one message).
- Primus gaps are whole messages. Folded modulo 0.48 s, 11,000 of 14,989 AWP gaps sit at zero: lost messages, with the stamps intact. CO₂ has the same peak at zero plus a second near 0.43 of a message.
- SNUADC and BIS gaps mostly are not. Their remainders are spread almost evenly, with small peaks at 0 and 1 (whole messages) and, in BIS, a comb at multiples of 1/8 of a message (2 samples). Most of these gaps are clock corrections or a loss combined with one (Small positive SNUADC gaps are real data loss shows that SNUADC's sub-second gaps do lose data).
- CardioQ forward and backward steps are symmetric around one message: its stamps jitter by about a message.
Discussion¶
A gap rule on the stamps alone sees only the forward steps, a small share of the irregularities. Knowing the message size per device turns the stamps into a test: a whole number of messages is a loss, a remainder is a clock correction. What each break in the stamps is combines this with the samples on either side of each step.
Observations it supports¶
| Observation | Status | Summary |
|---|---|---|
| Eight recordings store whole messages twice | established | In 8 recordings 15–48% of the waveform records are byte-for-byte copies of the record before them, with the same timestamp: case 15, case 364, case 1946, case 3865, case 4857, case 5414, case 5595 and case 5619. SNUADC, Primus and BIS are affected equally, so the recorder, not a device, wrote them twice. |
| In some recordings two unrelated streams are interleaved in one track | open | In 7 recordings 64–95% of the records alternate between two streams: each record continues the one two before it, not the one before, and the two streams do not match each other. SNUADC and BIS in case 2219, case 2379, case 2640, case 4836, case 5327 and case 5617; SNUADC in case 1939. In case 94, case 945, case 1132, case 3195 and case 5700 4–7% of SNUADC records interleave. |
Limitations¶
- Steps under 1 µs are not kept; float rounding of stamps makes some zero steps non-zero.
- The remainder test cannot separate a correction from a loss plus a correction.
Open questions¶
- Lost data or a moved clock? (next step). For SNUADC and BIS gaps that are not whole messages, test whether the waveform continues across the gap (no loss) or needs missing samples to fit (Does the waveform run on?). Partly answered by What each break in the stamps is: steps under one message run on (clock corrections), while about 70% of SNUADC fractional gaps and roughly a third of BIS fractional gaps lose data. What is left is fitting how much.