Data quality

Every network's feed is an undocumented page or file that can change without notice, and the archive cannot be re-recorded later. So the data is checked at three points: before code ships, against the live feeds, and in the archive itself.

One contract, checked three ways

Every network has an adapter that turns its feed into candidate outages. All of them answer to one shared contract: a list of invariants that a normalised outage must satisfy whichever network it came from — a real, unique, stable reference; a valid status; a restoration that is not before its start or in the future; dates in a sensible range; customer counts that are whole and not negative; coordinates inside the network's own state; a location of the kind the network is supposed to publish.

• On every test run, every adapter parses recorded copies of its real feed, with the clock pinned to the moment each was captured, and the output is checked against the contract. Any violation fails the run, and a network without a recorded feed fails it too, so no adapter can ship unchecked.
• Against the live feeds, a probe runs every adapter against its network's real endpoint, using the same client that does the polling, and checks the result against the same contract. It writes nothing to the archive, and a contract error or a failed fetch fails it, so it can stand in front of a deploy.
• In the archive, an audit reads what has actually been stored: how completely each network fills each field, and anything that should not exist — a restoration before its start, invalid geometry, a location of the wrong kind, an outage with no category or state. The completeness figures are what GET/v1/networks publishes as publishes.

What else keeps it honest

• Unrecognised words are reported, not hidden. A status word no adapter recognises is treated as active and logged; a cause the classifier does not recognise is other and listed for review. A network inventing a new phrase during a storm shows up in a report rather than silently changing the map.
• Nothing is invented. No customer count is synthesised and no start time is made up; where a network does not say, the field is null. A restoration is never inferred from a planned window, with one stated exception: Power and Water's page says nothing about when planned work finished, so its planned work is closed at the scheduled end.
• Impossible values are refused rather than published. A few records carry a network's restoration time before its start; their duration_minutes is null, not negative.
• Broken shapes are repaired at the door. A polygon that crosses itself is repaired when it is written, so a location filter never fails on one bad outage.
• A failed poll changes nothing. If a feed is down or returns something that is not a valid feed, the last recorded data stands, feed.status says so, and nothing is closed because a poll failed.

Known limitations

These are true of the data today. Each is stated where it applies elsewhere in these docs too.

• It is not an official source. It is what each network published, recorded as it happened, and a network's feed can be late, wrong or briefly down. Confirm with the distributor before acting, and never rely on it for life support or emergency decisions.
• Nothing exists before each network's start date. Networks publish no history, so the archive begins when recording began and cannot be backfilled. See Networks and coverage.
• Recording has gaps when a feed is down. Changes a network made while its feed was unreachable are seen only once it is back. An outage that opened and closed entirely within a feed outage is not recorded.
• Observed restorations are approximate. Where a network does not publish restoration times, restored_at is when the outage left its feed: accurate to the few minutes between polls, and later than the real restoration when a network is slow to take an outage down.
• Points are approximate. Several networks publish a point rather than an area, and some round it to a kilometre or more. A point says roughly where, never how far.
• Some cancellations look like restorations. Networks that publish no cancelled status simply remove called-off work; the API records it as restored with restored_at_basis: observed.
• Fields networks do not publish are missing for every one of their outages. No start time, no customer counts, no cause — each is null for the networks concerned. The table on Networks and coverage shows which.
• Overrunning planned work in the Northern Territory is recorded as finishing on schedule, because Power and Water's scheduled end is all there is to go on.

If you find something wrong, email support@aussieoutagetracker.com.au with the outage id. We would rather hear about it than not.