Keeping a copy in sync

To keep your own copy of the archive — or of one state, or one network — take a first copy once, then read only what changed. Every outage carries updated_at, the last time its record changed, and two parameters work together on it:

• changed_since — only outages whose record changed after this instant: created, updated or closed. It may reach back at most 31 days.
• sort=updated — oldest change first, so a sync that stops halfway can carry on from where it got to.

The first copy

One query reads at most 31 days of outage start times, up to 500 outages a page (Pagination has the rules). So there are two ways to start:

• A bulk export, for anything longer than a few months. Request it in Settings › API with the states, the dates and what it is for; we quote it and send the file. Nothing is exported automatically.
• A windowed walk, for a shorter stretch: step through it 31 days at a time with from and to, walking each window to its end with cursors, and upsert each outage by id.

Either way, note the time you started, a minute or two early. That is your first mark.

const API = 'https://api.aussieoutagetracker.com.au/v1/outages';
const headers = { Authorization: `Bearer ${process.env.AOT_API_KEY}` };
const WINDOW = 31 * 86_400_000; // max_window_days, in milliseconds
 
export async function firstCopy(fromIso, upsert, filters = {}) {
  const mark = new Date(Date.now() - 2 * 60_000).toISOString(); // a little before now
  for (let start = Date.parse(fromIso); start < Date.now(); start += WINDOW) {
    const end = Math.min(start + WINDOW, Date.now());
    let cursor;
    do {
      const params = new URLSearchParams({
        ...filters,
        from: new Date(start).toISOString(),
        to: new Date(end).toISOString(),
        limit: '500',
      });
      if (cursor) params.set('cursor', cursor);
      const res = await fetch(`${API}?${params}`, { headers });
      if (!res.ok) throw new Error(`${res.status} ${(await res.json()).error.code}`);
      const page = await res.json();
      for (const outage of page.data) await upsert(outage); // keyed on outage.id
      cursor = page.pagination.next_cursor;
    } while (cursor);
  }
  return mark; // store it; it is the first changed_since
}

An outage that starts exactly on a window's edge comes back in both windows. Upserting by id makes that harmless.

Then, every few minutes

const API = 'https://api.aussieoutagetracker.com.au/v1/outages';
const headers = { Authorization: `Bearer ${process.env.AOT_API_KEY}` };
 
export async function sync(mark, upsert, filters = {}) {
  let cursor;
  let highWater = mark;
  do {
    const params = new URLSearchParams({ ...filters, changed_since: mark, sort: 'updated', limit: '500' });
    if (cursor) params.set('cursor', cursor);
    const res = await fetch(`${API}?${params}`, { headers });
    if (!res.ok) throw new Error(`${res.status} ${(await res.json()).error.code}`);
    const page = await res.json();
    for (const outage of page.data) {
      await upsert(outage); // keyed on outage.id
      if (outage.updated_at > highWater) highWater = outage.updated_at;
    }
    cursor = page.pagination.next_cursor;
  } while (cursor);
  return highWater; // store it; pass it in next time
}

Getting it right

• Upsert, never insert. An outage changes many times — its status, its estimate, its customer count — and each change moves its updated_at and brings it back into changed_since. The id is stable for its whole life; key your copy on it.
• Overlap a little. Ask from a minute or two before your mark rather than exactly at it. Upserting is idempotent, so re-reading a few records costs a call and nothing else, and it covers a change that landed in the same instant as the last one you saw.
• Closing is a change. When an outage is restored or cancelled its record changes, so it arrives through changed_since with its final status. You never need to look for outages that disappeared.
• Do not fall more than 31 days behind. A changed_since older than that is refused with 400 invalid_parameter. If a sync has been off that long, catch up with a windowed walk from a little before the mark, or a bulk export, then carry on from a fresh mark.
• Keep the same filters. If you mirror one state, send state on every sync, first copy and incremental.
• Poll at the networks' pace. The feeds update every few minutes; polling faster returns the same data and spends calls. An empty sync is still a call, so every few minutes is plenty — every five minutes is 8,640 calls in a 30-day month.
• Shapes are opt-in. Lists omit geometry unless you ask with include=geometry, because polygons are most of the payload. Fetch them on the sync if you need them, or on demand with GET/v1/outages/{id}.

The history of each outage

changed_since gives you each outage's current state. If you also want every intermediate state — each estimate the network gave, in order — fetch GET/v1/outages/{id}/history for the outages you care about.