Harden traffic history worker lifecycle and query performance

.codex/skills/design-harbor-device-ecosystem/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: design-harbor-device-ecosystem
-description: Design, plan, implement, review, or document Harbor Gateway integrations with current and future companion devices. Use for LAN advertisement and discovery, Connect-Gateway pairing, trusted device identity, connected-client presence, ecosystem membership in the Gateway device list, multi-Gateway selection, capability or protocol negotiation, and related API, persistence, migration, or UI contracts. Do not use for traffic-only LAN inventory or visual polish that does not change ecosystem behavior.
+description: "Use when changing or reviewing Harbor companion-device discovery, pairing, trust, presence, routing handoff, or their API and persistence contracts. Not for traffic-only inventory or visual polish."
 ---
 
 # Design Harbor Device Ecosystem
@@ -18,7 +18,7 @@ Keep discovery, trust, application presence, routing, and LAN observation separa
 
 ## Workflow
 
-1. Read `PRODUCT.md`, `workpack/PRODUCT_PRIORITIES.md`, `workpack/DATA_CONSISTENCY_MODEL.md`, `workpack/DESIGN_OWNER_POLICY.md`, `workpack/STATUS.md`, and the selected workpack task.
+1. Start with the selected workpack task when one exists. Read `PRODUCT.md` and `workpack/PRODUCT_PRIORITIES.md` for product scope, `workpack/DATA_CONSISTENCY_MODEL.md` for persistence or projection changes, and `workpack/STATUS.md` for active roadmap/cutover work. Any visible UI decision requires `workpack/DESIGN_OWNER_POLICY.md`; reuse an exact owner decision already recorded for this scope.
 2. Read [device-ecosystem-contract.md](references/device-ecosystem-contract.md) before changing discovery, pairing, identity, presence, routing handoff, or the Gateway device projection.
 3. Trace the current producer -> transport -> persistence -> canonical snapshot -> UI path. Confirm the owning service and every caller before editing.
 4. Classify each proposed field and state as one of: discovery candidate, persisted trust relationship, authenticated presence lease, route state, network observation, or derived binding. Reject fields that mix categories.
@@ -44,7 +44,7 @@ Keep discovery, trust, application presence, routing, and LAN observation separa
 
 ## Acceptance pass
 
-Before handing off an ecosystem change, verify:
+Verify the affected contracts below. Protocol, authentication, shared projection, or migration changes require coverage across the impacted lifecycle; a documentation-only correction uses consistency/link checks. Do not operate live devices without the exact authorization described above:
 
 - A supported client can discover more than one Gateway and survives address changes without changing Gateway identity.
 - Spoofed discovery cannot create trust, change the active route, or appear as a paired ecosystem member.

.codex/skills/design-vpn-client-ui/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: design-vpn-client-ui
-description: Design, implement, review, or refine the client-facing VPN interfaces in this repository using the established calm monospace visual language and smooth state-driven motion. Use for the current macOS client and future end-user gateway client screens, especially power controls, subscriptions, traffic usage, proxy copy controls, server selection, responsive layout, hover feedback, transitions, and animation polish. Do not use for the administrative gateway UI unless the user explicitly asks to apply the client visual language there.
+description: "Use when changing or reviewing Harbor Connect and Gateway client UI. Apply the established visual and motion language; not for administrative Gateway UI unless explicitly requested."
 ---
 
 # Design VPN Client UI
@@ -9,23 +9,20 @@ Preserve the repo's focused one-screen VPN client language: a centered primary a
 
 ## Workflow
 
-1. Read `PRODUCT.md` and the complete client component and styles before editing.
+1. Inspect the affected client component, state owner, relevant styles, and nearest analogue. Read `PRODUCT.md` when product behavior or mode semantics change; expand context when shared invariants require it.
 2. Inspect supplied evidence and trace the real DOM and state change that causes the visual issue. Follow repository testing policy; do not launch manual or interactive visual testing unless the user explicitly requests it in the current prompt.
 3. Read [visual-language.md](references/visual-language.md) for layout, hierarchy, color, and typography work.
 4. Read [motion-and-interaction.md](references/motion-and-interaction.md) for animation, hover, refresh, input, copy, or state-transition work.
 5. Reuse existing React state, CSS variables, formatters, and API paths. Prefer a narrow CSS/markup change over a new abstraction or dependency.
 6. Keep geometry stable across every state. Reserve space before animating content.
 7. Implement `prefers-reduced-motion` alongside every new animation.
-8. Run `npm test`, `npm run build`, and `git diff --check`. Perform manual visual inspection only when explicitly requested.
+8. Run focused automated checks for affected behavior and `npm run build` for client code changes; use the full `npm test` suite when shared behavior changes or narrower evidence is insufficient. Documentation-only work uses structural/link checks and `git diff --check`. Manual visual inspection requires an explicit request in the current prompt.
 
 ## Communicating a proposed change
 
-For implementation proposals and progress summaries, combine structure instead of maximizing it:
+For a local correction, report the result and focused evidence briefly. For a change spanning
-
+several owners or state lifecycles, use one compact table of affected modules plus the
-- start with one compact table grouped by affected modules such as frontend, UI, backend/API, styles, tests, and versions;
+relevant states, accessibility, and motion decisions. Omit empty sections and unrelated modules.
-- use short lists only for cross-cutting details such as states, accessibility, motion, or changed files;
-- keep unaffected modules visible in the table when that clarifies scope;
-- avoid both an unstructured paragraph and a separate table for every subsection.
 
 ## Non-negotiable decisions
 
@@ -49,7 +46,7 @@ For implementation proposals and progress summaries, combine structure instead o
 
 ## Acceptance pass
 
-Before handing off, verify:
+Check the affected behaviors below using automated evidence where possible. Shared layout or motion changes require broader coverage; a local correction does not require exercising unrelated controls. Manual inspection remains limited to an explicit request in the current prompt:
 
 - Power on/off is unmistakable without reading the label.
 - Switching on/off preserves the exact positions of title, timer, and hint.

src/server/services/trafficHistoryService.ts

@@ -10,74 +10,105 @@ import type { HistoryWorkerRequest } from './trafficHistoryWorker.js';
 
 type Request = HistoryWorkerRequest extends infer R ? R extends { id: number } ? Omit<R, 'id'> : never : never;
 const MAX_QUEUED_CONNECTIONS = 16_384;
+interface Job {
+  id: number;
+  message: Request;
+  resolve: (value: unknown) => void;
+  reject: (error: Error) => void;
+}
 
 export function createTrafficHistoryService({ filePath, source }: {
   filePath: string;
   source: () => LiveTrafficSourceState;
 }) {
   let worker: Worker | null = null;
+  let ready = false;
+  let terminating: Promise<void> | null = null;
+  let active: Job | null = null;
+  const jobs: Job[] = [];
+  let watchdog: NodeJS.Timeout | undefined;
+  let readDeadline: NodeJS.Timeout | undefined;
+  let readers = 0;
   let sequence = 0;
   let pending: NativeTrafficProjectionBatch[] = [];
   let queued = 0;
   let missedSince: number | null = null;
   let storageError = false;
   let stopping = false;
+  let closed = false;
   let flushing: Promise<void> | null = null;
-  const requests = new Map<number, { resolve: (value: unknown) => void; reject: (error: Error) => void; timer: NodeJS.Timeout }>();
 
-  function failed() {
+  function clearDeadlines() {
-    storageError = true;
+    clearTimeout(watchdog);
-    missedSince ??= Date.now();
+    clearTimeout(readDeadline);
-    for (const request of requests.values()) {
+  }
-      clearTimeout(request.timer);
+  function failed(current: Worker) {
-      request.reject(new Error('TRAFFIC_HISTORY_UNAVAILABLE'));
+    if (worker !== current) return;
+    const startupFailed = !ready;
+    worker = null;
+    ready = false;
+    clearDeadlines();
+    active?.reject(new Error('TRAFFIC_HISTORY_UNAVAILABLE'));
+    active = null;
+    if (startupFailed) {
+      for (const job of jobs.splice(0)) job.reject(new Error('TRAFFIC_HISTORY_UNAVAILABLE'));
     }
-    requests.clear();
+    // Native SQLite may still be finishing a call. Never start a second writer until it exits.
+    terminating = current.terminate().then(() => {}, () => {}).then(() => {
+      terminating = null;
+      dispatch();
+    });
   }
   function ensureWorker() {
     if (worker) return worker;
     const current = new Worker(new URL('./trafficHistoryWorker.js', import.meta.url), { workerData: { filePath } });
     worker = current;
-    current.on('message', (message: { id: number; result?: unknown; error?: string }) => {
+    watchdog = setTimeout(() => failed(current), 60_000);
-      const request = requests.get(message.id);
+    current.on('message', (message: { ready?: boolean; id?: number; result?: unknown; error?: string }) => {
-      if (!request) return;
-      requests.delete(message.id);
-      clearTimeout(request.timer);
-      if (message.error) request.reject(new Error('TRAFFIC_HISTORY_UNAVAILABLE'));
-      else request.resolve(message.result);
-    });
-    current.on('error', () => {
       if (worker !== current) return;
-      worker = null;
+      if (message.ready) {
-      failed();
+        clearDeadlines();
-    });
+        ready = true;
-    current.on('exit', () => {
+        dispatch();
-      if (worker !== current) return;
+        return;
-      worker = null;
+      }
-      if (!stopping) failed();
+      if (!active || active.id !== message.id) return;
+      const job = active;
+      active = null;
+      clearDeadlines();
+      if (message.error) job.reject(new Error('TRAFFIC_HISTORY_UNAVAILABLE'));
+      else job.resolve(message.result);
+      dispatch();
     });
+    current.on('error', () => failed(current));
+    current.on('exit', () => failed(current));
     return current;
   }
+  function dispatch() {
+    if (active || terminating || !jobs.length) return;
+    let current: Worker;
+    try { current = ensureWorker(); }
+    catch {
+      for (const job of jobs.splice(0)) job.reject(new Error('TRAFFIC_HISTORY_UNAVAILABLE'));
+      return;
+    }
+    if (!ready) return;
+    const job = jobs.shift()!;
+    active = job;
+    if (job.message.kind === 'query') {
+      // A slow read only expires its caller. Keep the slot until SQL actually finishes.
+      readDeadline = setTimeout(() => job.reject(new Error('TRAFFIC_HISTORY_UNAVAILABLE')), 5_000);
+    }
+    watchdog = setTimeout(() => failed(current), 60_000);
+    try { current.postMessage({ ...job.message, id: job.id }); }
+    catch { failed(current); }
+  }
   function request(message: Request): Promise<unknown> {
-    if (requests.size >= 32) return Promise.reject(new Error('TRAFFIC_HISTORY_BUSY'));
     return new Promise((resolve, reject) => {
-      const id = ++sequence;
+      const job = { id: ++sequence, message, resolve, reject };
-      const timer = setTimeout(() => {
+      if (message.kind === 'ingest') jobs.unshift(job);
-        requests.delete(id);
+      else jobs.push(job);
-        reject(new Error('TRAFFIC_HISTORY_UNAVAILABLE'));
+      dispatch();
-        // Bound worker mailbox growth as well as the main-thread queue after slow SQL.
-        const current = worker;
-        worker = null;
-        void current?.terminate();
-        failed();
-      }, 5_000);
-      requests.set(id, { resolve, reject, timer });
-      try { ensureWorker().postMessage({ ...message, id }); }
-      catch {
-        requests.delete(id);
-        clearTimeout(timer);
-        reject(new Error('TRAFFIC_HISTORY_UNAVAILABLE'));
-      }
     });
   }
   function enqueue(batch: NativeTrafficProjectionBatch) {
@@ -90,6 +121,7 @@ export function createTrafficHistoryService({ filePath, source }: {
     queued += batch.connections.length;
   }
   function flush(): Promise<void> {
+    if (closed) return Promise.resolve();
     if (flushing) return flushing;
     const batches = pending;
     const missed = missedSince;
@@ -115,8 +147,13 @@ export function createTrafficHistoryService({ filePath, source }: {
     enqueue,
     flush,
     async query(query: TrafficHistoryQuery) {
+      let admitted = false;
       try {
+        if (stopping || readers >= 32) throw new Error('TRAFFIC_HISTORY_BUSY');
+        readers++;
+        admitted = true;
         await flush();
+        if (stopping) throw new Error('TRAFFIC_HISTORY_UNAVAILABLE');
         const result = assertTrafficHistorySnapshot(await request({ kind: 'query', query }));
         result.source = source();
         if (missedSince !== null || storageError) result.coverage.partial = true;
@@ -127,20 +164,31 @@ export function createTrafficHistoryService({ filePath, source }: {
         result.storage = { status: 'error', errorCode: 'TRAFFIC_HISTORY_UNAVAILABLE' };
         result.coverage.partial = true;
         return result;
+      } finally {
+        if (admitted) readers--;
       }
     },
     async close() {
       if (stopping) return;
       stopping = true;
       clearInterval(timer);
+      for (let index = jobs.length - 1; index >= 0; index--) {
+        if (jobs[index].message.kind === 'query') {
+          jobs.splice(index, 1)[0].reject(new Error('TRAFFIC_HISTORY_UNAVAILABLE'));
+        }
+      }
       await flushing;
       await flush();
       if (worker) {
         try { await request({ kind: 'close' }); } catch { /* shutdown remains bounded */ }
         const current = worker;
         worker = null;
+        ready = false;
+        clearDeadlines();
         await current?.terminate();
       }
+      await terminating;
+      closed = true;
     },
   };
 }

src/server/services/trafficHistoryStore.ts

@@ -36,6 +36,7 @@ export function openTrafficHistoryStore(filePath: string, now = Date.now) {
         upload INTEGER NOT NULL CHECK(upload >= 0), download INTEGER NOT NULL CHECK(download >= 0),
         PRIMARY KEY(at, resolution, dimension_id)
       ) STRICT, WITHOUT ROWID;
+      CREATE INDEX IF NOT EXISTS buckets_dimension_time ON buckets(dimension_id, at, upload, download);
       CREATE TABLE IF NOT EXISTS checkpoints (
         identity TEXT PRIMARY KEY, upload INTEGER NOT NULL, download INTEGER NOT NULL,
         last_seen INTEGER NOT NULL, closed INTEGER NOT NULL
@@ -46,6 +47,29 @@ export function openTrafficHistoryStore(filePath: string, now = Date.now) {
       PRAGMA user_version = 1;
     `));
   } catch (error) { db.close(); throw error; }
+  db.exec(`CREATE TEMP TABLE period_totals (
+    period TEXT NOT NULL, dimension_id INTEGER NOT NULL, upload INTEGER NOT NULL, download INTEGER NOT NULL,
+    PRIMARY KEY(period, dimension_id)
+  ) STRICT, WITHOUT ROWID`);
+  const periods = new Map<string, { from: number; to: number }>();
+  const dropPeriod = db.prepare('DELETE FROM period_totals WHERE period = ?');
+  function periodTotals(from: number, to: number) {
+    const key = `${from}:${to}`;
+    if (!periods.has(key)) {
+      // Four visible ranges; old frozen pages are recomputed after eviction.
+      if (periods.size === 4) {
+        const oldest = periods.keys().next().value!;
+        dropPeriod.run(oldest);
+        periods.delete(oldest);
+      }
+      db.prepare(`INSERT INTO period_totals
+        SELECT ?, d.id, SUM(b.upload), SUM(b.download)
+        FROM dimensions d CROSS JOIN buckets b INDEXED BY buckets_dimension_time
+        ON b.dimension_id = d.id WHERE b.at >= ? AND b.at < ? GROUP BY d.id`).run(key, from, to);
+      periods.set(key, { from, to });
+    }
+    return key;
+  }
   // SQLite's built-in lower() handles ASCII only; service labels also use Cyrillic.
   db.function('lower_unicode', { deterministic: true }, (value) => String(value).toLowerCase());
   const meta = (key: string) => db.prepare('SELECT value FROM meta WHERE key = ?').get(key)?.value as string | undefined;
@@ -87,12 +111,15 @@ export function openTrafficHistoryStore(filePath: string, now = Date.now) {
       db.prepare('DELETE FROM checkpoints WHERE closed = 1 AND last_seen < ?').run(cutoff);
       db.prepare('DELETE FROM gaps WHERE until_at < ?').run(cutoff);
       db.exec('DELETE FROM dimensions WHERE id NOT IN (SELECT dimension_id FROM buckets)');
+      db.exec('DELETE FROM period_totals');
     });
+    periods.clear();
     lastMaintenance = timestamp;
   }
 
   function ingest(batches: NativeTrafficProjectionBatch[], source: LiveTrafficSourceState, missedSince: number | null = null) {
     const timestamp = now();
+    const changedPeriods = new Set<string>();
     transaction(db, () => {
       const collectionStart = Number(meta('started') || batches[0] && Date.parse(batches[0].observedAt) || timestamp);
       if (!meta('started') && batches.length) setMeta.run('started', String(collectionStart));
@@ -147,7 +174,11 @@ export function openTrafficHistoryStore(filePath: string, now = Date.now) {
           const dimension = saveDimension.get(JSON.stringify(dims), origin, connection.origin.label,
             ...dims.slice(1));
           const resolution = at < Math.floor((timestamp - 7 * DAY) / HOUR) * HOUR ? HOUR : MINUTE;
-          saveBucket.run(Math.floor(at / resolution) * resolution, resolution, dimension!.id, up, down);
+          const bucketAt = Math.floor(at / resolution) * resolution;
+          saveBucket.run(bucketAt, resolution, dimension!.id, up, down);
+          for (const [key, period] of periods) {
+            if (bucketAt >= period.from && bucketAt < period.to) changedPeriods.add(key);
+          }
         }
         if (batch.closedIds.length) {
           const closed = db.prepare(`UPDATE checkpoints SET closed = 1, last_seen = MAX(last_seen, ?)
@@ -161,7 +192,11 @@ export function openTrafficHistoryStore(filePath: string, now = Date.now) {
       if (source === 'degraded' || source === 'stale' || source === 'incompatible'
         || (source === 'connecting' && lastAt)) gap(lastAt || timestamp, timestamp);
       setMeta.run('source', source);
+      for (const key of changedPeriods) dropPeriod.run(key);
     });
+    for (const key of changedPeriods) {
+      periods.delete(key);
+    }
     if (timestamp - lastMaintenance >= HOUR) maintain(timestamp);
   }
 
@@ -187,8 +222,9 @@ export function openTrafficHistoryStore(filePath: string, now = Date.now) {
     result.coverage.gapCount = Number(db.prepare('SELECT COUNT(*) AS count FROM gaps WHERE until_at >= ? AND at < ?').get(effectiveFrom, to)?.count);
     result.coverage.partial = result.coverage.gapCount > 0 || ['stale', 'degraded', 'incompatible'].includes(result.source)
       || (lastAt > 0 && result.source === 'live' && timestamp - lastAt > 5_000);
-    const where = ['b.at >= ?', 'b.at < ?'];
+    const period = periodTotals(effectiveFrom, to);
-    const args: Array<string | number> = [effectiveFrom, to];
+    const where = ['b.period = ?'];
+    const args: Array<string | number> = [period];
     for (const [column, value] of [
       ['origin_id', input.originId], ['service', input.service], ['domain', input.domain], ['hostname', input.hostname],
       ['route', input.route === 'all' ? '' : input.route],
@@ -200,7 +236,7 @@ export function openTrafficHistoryStore(filePath: string, now = Date.now) {
     if (input.search) {
       where.push("instr(lower_unicode(d.hostname || ' ' || d.ip || ' ' || d.service), ?) > 0"); args.push(input.search.toLowerCase());
     }
-    const joined = `FROM buckets b JOIN dimensions d ON d.id = b.dimension_id WHERE ${where.join(' AND ')}`;
+    const joined = `FROM period_totals b JOIN dimensions d ON d.id = b.dimension_id WHERE ${where.join(' AND ')}`;
     const totals = db.prepare(`SELECT COALESCE(SUM(b.upload), 0) AS upload, COALESCE(SUM(b.download), 0) AS download ${joined}`);
     totals.setReadBigInts(true);
     const sum = totals.get(...args)!;
@@ -216,8 +252,8 @@ export function openTrafficHistoryStore(filePath: string, now = Date.now) {
     }));
     result.nextOffset = page.length > 100 ? input.offset + 100 : null;
     const origins = db.prepare(`SELECT d.origin_id AS id, MAX(d.origin_label) AS label FROM dimensions d
-      JOIN buckets b ON b.dimension_id = d.id WHERE b.at >= ? AND b.at < ?
+      JOIN period_totals b ON b.dimension_id = d.id WHERE b.period = ?
-      GROUP BY d.origin_id ORDER BY id LIMIT 257`).all(effectiveFrom, to);
+      GROUP BY d.origin_id ORDER BY id LIMIT 257`).all(period);
     result.origins = origins.slice(0, 256).map((row) => ({ id: String(row.id), label: String(row.label) }));
     result.originsTruncated = origins.length > 256;
     return result;

src/server/services/trafficHistoryWorker.ts

@@ -10,6 +10,7 @@ export type HistoryWorkerRequest =
   | { id: number; kind: 'close' };
 
 const store = openTrafficHistoryStore(workerData.filePath);
+parentPort!.postMessage({ ready: true });
 parentPort!.on('message', (message: HistoryWorkerRequest) => {
   try {
     let result: unknown = null;

src/shared/versions.ts

@@ -1,7 +1,7 @@
 export const HARBOR_VERSIONS = Object.freeze({
-  macClient: '0.37.1',
+  macClient: '0.37.2',
-  gatewayClient: '0.39.1',
+  gatewayClient: '0.39.2',
-  gatewayBackend: '0.39.0',
+  gatewayBackend: '0.39.1',
 });
 
 export interface ParsedVersion {

src/web/App.tsx

@@ -113,9 +113,14 @@ export function App() {
   }
 
   useEffect(() => {
-    loadState();
+    let cancelled = false;
-    const timer = setInterval(loadState, 5000);
+    let timer: ReturnType<typeof setTimeout>;
-    return () => clearInterval(timer);
+    const poll = async () => {
+      await loadState();
+      if (!cancelled) timer = setTimeout(poll, 5000);
+    };
+    void poll();
+    return () => { cancelled = true; clearTimeout(timer); };
   }, []);
 
   useEffect(() => {

src/web/api/harborClient.ts

@@ -50,34 +50,42 @@ export async function request(
   options: RequestOptions = {},
   fetchImpl: FetchImplementation = fetch,
 ): Promise<unknown> {
-  let response: JsonResponse;
+  const deadline = new AbortController();
+  const timer = setTimeout(() => deadline.abort(new HarborApiError({ code: 'CONTROL_UNREACHABLE' })),
+    (options.method || 'GET') === 'GET' ? 15_000 : 60_000);
+  const signal = options.signal ? AbortSignal.any([options.signal, deadline.signal]) : deadline.signal;
   try {
-    response = await fetchImpl(url, {
+    const response = await fetchImpl(url, {
       ...options,
+      signal,
       headers: {
         'content-type': 'application/json',
         ...(options.headers || {}),
       },
     });
-  } catch (error) {
-    if (record(error).name === 'AbortError') throw error;
-    throw new HarborApiError({ code: 'CONTROL_UNREACHABLE' });
-  }
 
-  let data: unknown = {};
+    let data: unknown = {};
-  try {
+    try {
-    data = await response.json();
+      data = await response.json();
-  } catch {
+    } catch {
-    if (response.ok) throw new HarborApiError({ code: 'UNKNOWN' }, response.status);
+      if (signal.aborted) throw signal.reason;
+      if (response.ok) throw new HarborApiError({ code: 'UNKNOWN' }, response.status);
+    }
+    const payload = record(data);
+    if (!response.ok || payload.success === false) {
+      const errorPayload = payload.error && typeof payload.error === 'object'
+        ? payload.error
+        : { code: response.status >= 500 ? 'CONTROL_UNREACHABLE' : 'UNKNOWN' };
+      throw new HarborApiError(errorPayload, response.status);
+    }
+    return data;
+  } catch (error) {
+    if (signal.aborted) throw signal.reason;
+    if (error instanceof HarborApiError || record(error).name === 'AbortError') throw error;
+    throw new HarborApiError({ code: 'CONTROL_UNREACHABLE' });
+  } finally {
+    clearTimeout(timer);
   }
-  const payload = record(data);
-  if (!response.ok || payload.success === false) {
-    const errorPayload = payload.error && typeof payload.error === 'object'
-      ? payload.error
-      : { code: response.status >= 500 ? 'CONTROL_UNREACHABLE' : 'UNKNOWN' };
-    throw new HarborApiError(errorPayload, response.status);
-  }
-  return data;
 }
 
 export const api = {

test/server/traffic-history-load.test.js

@@ -0,0 +1,65 @@
+import assert from 'node:assert/strict';
+import fs from 'node:fs';
+import os from 'node:os';
+import path from 'node:path';
+import { DatabaseSync } from 'node:sqlite';
+import { performance } from 'node:perf_hooks';
+import test from 'node:test';
+import { openTrafficHistoryStore } from '../../dist/server/services/trafficHistoryStore.js';
+import { createTrafficHistoryService } from '../../dist/server/services/trafficHistoryService.js';
+import { parseTrafficHistoryQuery } from '../../dist/shared/trafficHistory.js';
+
+test('large history preserves totals and serves concurrent readers through cleanup and filters', async (t) => {
+  const directory = fs.mkdtempSync(path.join(os.tmpdir(), 'harbor-history-load-'));
+  const file = path.join(directory, 'traffic.sqlite');
+  const at = Math.floor(Date.now() / 60_000) * 60_000;
+  const minutes = process.env.HARBOR_HISTORY_LOAD === '1' ? 3000 : 300;
+  let service;
+  t.after(async () => {
+    await service?.close();
+    fs.rmSync(directory, { recursive: true, force: true });
+  });
+  openTrafficHistoryStore(file).close();
+  const db = new DatabaseSync(file);
+  try {
+    db.exec(`BEGIN;
+      WITH RECURSIVE n(i) AS (VALUES(1) UNION ALL SELECT i+1 FROM n WHERE i<2000)
+      INSERT INTO dimensions SELECT i,'dimension-'||i,'device-'||(i%20),'Device '||(i%20),
+        '192.0.2.1','tproxy-in','service-'||(i%250),'domain-'||(i%250)||'.test',
+        'host-'||i||'.test','203.0.113.'||(i%250),'vpn','vpn-one' FROM n;`);
+    db.prepare(`WITH RECURSIVE minutes(i) AS (VALUES(1) UNION ALL SELECT i+1 FROM minutes WHERE i<?)
+      INSERT INTO buckets SELECT ?-i*60000,60000,d.id,100,200 FROM minutes
+      CROSS JOIN dimensions d WHERE d.id<=1000`).run(minutes, at);
+    db.prepare('INSERT INTO buckets SELECT ?,60000,id,100,200 FROM dimensions WHERE id>1000').run(at - 91 * 86_400_000);
+    db.exec('COMMIT');
+    // Exercise the additive index upgrade on a populated pre-fix database.
+    db.exec('DROP INDEX buckets_dimension_time');
+  } finally { db.close(); }
+
+  service = createTrafficHistoryService({ filePath: file, source: () => 'live' });
+  const query = parseTrafficHistoryQuery(new URLSearchParams(`range=7d&until=${at}`));
+  const start = performance.now();
+  const concurrent = await Promise.all(Array.from({ length: 4 }, () => service.query(query)));
+  const concurrentMs = performance.now() - start;
+  for (const result of concurrent) {
+    assert.equal(result.storage.status, 'ready');
+    assert.equal(result.totals.downloadBytes, String(minutes * 1000 * 200));
+    assert.equal(result.origins.length, 20);
+    assert.equal(result.nextOffset, 100);
+  }
+  const timings = [];
+  for (const [filter, dimensions] of [
+    [{ originId: 'device-1' }, 50],
+    [{ search: 'host-99' }, 11],
+    [{ level: 'ip', service: 'service-1', domain: 'domain-1.test', hostname: 'host-1.test' }, 1],
+    [{ offset: 300 }, 1000],
+  ]) {
+    const begin = performance.now();
+    const result = await service.query({ ...query, ...filter });
+    timings.push({ filter, ms: performance.now() - begin });
+    assert.equal(result.storage.status, 'ready');
+    assert.equal(result.totals.downloadBytes, String(minutes * dimensions * 200));
+    if (filter.offset) assert.equal(result.rows.length, 0);
+  }
+  t.diagnostic(JSON.stringify({ buckets: minutes * 1000, concurrentMs, timings }));
+});

test/server/traffic-history-worker.test.js

@@ -0,0 +1,142 @@
+import assert from 'node:assert/strict';
+import fs from 'node:fs';
+import os from 'node:os';
+import path from 'node:path';
+import { Worker } from 'node:worker_threads';
+import test from 'node:test';
+import { createTrafficHistoryService } from '../../dist/server/services/trafficHistoryService.js';
+import { openTrafficHistoryStore } from '../../dist/server/services/trafficHistoryStore.js';
+import { emptyTrafficHistory, parseTrafficHistoryQuery } from '../../dist/shared/trafficHistory.js';
+
+const query = parseTrafficHistoryQuery(new URLSearchParams());
+async function fixture(t) {
+  const directory = fs.mkdtempSync(path.join(os.tmpdir(), 'harbor-history-queue-'));
+  const service = createTrafficHistoryService({ filePath: path.join(directory, 'traffic.sqlite'), source: () => 'live' });
+  await service.flush();
+  const post = Worker.prototype.postMessage;
+  const sent = [];
+  const waiting = [];
+  const held = [];
+  t.mock.method(Worker.prototype, 'postMessage', function (message) {
+    sent.push(message.kind);
+    if (message.kind !== 'query') return post.call(this, message);
+    const job = { worker: this, message };
+    if (waiting.length) waiting.shift()(job);
+    else held.push(job);
+  });
+  t.mock.timers.enable({ apis: ['setTimeout'] });
+  t.after(async () => {
+    t.mock.restoreAll();
+    t.mock.timers.reset();
+    await service.close();
+    fs.rmSync(directory, { recursive: true, force: true });
+  });
+  return { service, sent, file: path.join(directory, 'traffic.sqlite'),
+    next: () => held.length ? Promise.resolve(held.shift()) : new Promise((resolve) => waiting.push(resolve)),
+    answer: ({ worker, message }) => worker.emit('message', { id: message.id, result: emptyTrafficHistory(message.query, 'live') }),
+  };
+}
+
+test('queued reads get their own execution budget and do not reset the collector', async (t) => {
+  const f = await fixture(t);
+  const results = Array.from({ length: 3 }, () => f.service.query(query));
+  const first = await f.next();
+  const worker = first.worker;
+  assert.equal(f.sent.filter((kind) => kind === 'query').length, 1);
+  t.mock.timers.tick(4_000);
+  f.answer(first);
+  for (let index = 1; index < 3; index++) {
+    const next = await f.next();
+    assert.equal(next.worker, worker);
+    t.mock.timers.tick(4_000);
+    f.answer(next);
+  }
+  assert.deepEqual((await Promise.all(results)).map((result) => result.storage.status), ['ready', 'ready', 'ready']);
+});
+
+test('a slow read expires alone, retains its active slot, and lets pending writes go next', async (t) => {
+  const f = await fixture(t);
+  const firstResult = f.service.query(query);
+  const otherResult = f.service.query(query);
+  const first = await f.next();
+  t.mock.timers.tick(5_000);
+  assert.equal((await firstResult).storage.status, 'error');
+  const write = f.service.flush();
+  assert.equal(f.sent.filter((kind) => kind === 'query').length, 1);
+  const before = f.sent.length;
+  f.answer(first);
+  await write;
+  const second = await f.next();
+  assert.deepEqual(f.sent.slice(before), ['ingest', 'query']);
+  assert.equal(first.worker, second.worker);
+  f.answer(second);
+  const result = await otherResult;
+  assert.equal(result.storage.status, 'ready');
+  assert.equal(result.coverage.partial, false);
+});
+
+test('a full read queue reserves write capacity and shutdown rejects queued and future reads', async (t) => {
+  const f = await fixture(t);
+  const results = Array.from({ length: 32 }, () => f.service.query(query));
+  const first = await f.next();
+  assert.equal((await f.service.query(query)).storage.status, 'error');
+  const write = f.service.flush();
+  const closing = f.service.close();
+  f.answer(first);
+  await write;
+  await closing;
+  const snapshots = await Promise.all(results);
+  assert.equal(snapshots[0].storage.status, 'ready');
+  assert.ok(snapshots.slice(1).every((result) => result.storage.status === 'error'));
+  const sent = f.sent.length;
+  assert.equal((await f.service.query(query)).storage.status, 'error');
+  await f.service.flush();
+  assert.equal(f.sent.length, sent);
+});
+
+test('hard recovery waits for the previous worker to exit before starting a replacement', async (t) => {
+  const f = await fixture(t);
+  const firstResult = f.service.query(query);
+  const otherResult = f.service.query(query);
+  const first = await f.next();
+  const terminate = Worker.prototype.terminate;
+  let release;
+  let stopping = 0;
+  const gate = new Promise((resolve) => { release = resolve; });
+  t.mock.method(Worker.prototype, 'terminate', function () {
+    stopping++;
+    return gate.then(() => terminate.call(this));
+  });
+  t.mock.timers.tick(60_000);
+  assert.equal((await firstResult).storage.status, 'error');
+  assert.equal(stopping, 1);
+  assert.equal(f.sent.filter((kind) => kind === 'query').length, 1);
+  release();
+  const second = await f.next();
+  assert.notEqual(second.worker, first.worker);
+  f.answer(second);
+  assert.equal((await otherResult).storage.status, 'ready');
+});
+
+test('shutdown persists real counters queued behind an occupied read and survives reopen', async (t) => {
+  const f = await fixture(t);
+  const reading = f.service.query(query);
+  const first = await f.next();
+  const at = new Date(Date.now() - 120_000).toISOString();
+  f.service.enqueue({ epoch: 'shutdown', observedAt: at, reset: false, closedIds: [], connections: [{
+    id: 'persist-on-close', startedAt: at, closedAt: at,
+    traffic: { uploadBytes: '17', downloadBytes: '29' },
+    destination: { domain: 'example.org', ip: '203.0.113.1' },
+    source: { ip: '192.0.2.1' }, inbound: { tag: 'tproxy-in' },
+    origin: { kind: 'device', id: 'device-1', label: 'Laptop' },
+    route: { kind: 'vpn', outbound: 'vpn-one' },
+  }] });
+  const closing = f.service.close();
+  f.answer(first);
+  await reading;
+  await closing;
+  const store = openTrafficHistoryStore(f.file);
+  try {
+    assert.deepEqual(store.query(query).totals, { uploadBytes: '17', downloadBytes: '29' });
+  } finally { store.close(); }
+});

test/server/traffic-history.test.js

@@ -297,3 +297,35 @@ test('high-churn sample measures closed-lifecycle storage and epoch reclamation'
   db.close();
   t.diagnostic(JSON.stringify({ closedLifecycles: 100_000, writeMs, queryMs, bytes, reusableBytes }));
 });
+
+test('history searches each destination once, and cached periods stay exact after late data and rollback', (t) => {
+  const f = fixture(t); let clock = base;
+  let searched = 0;
+  const registerFunction = DatabaseSync.prototype.function;
+  t.mock.method(DatabaseSync.prototype, 'function', function (name, options, callback) {
+    return registerFunction.call(this, name, options, name === 'lower_unicode'
+      ? (value) => { searched++; return callback(value); } : callback);
+  });
+  const store = f.register(openTrafficHistoryStore(f.file, () => clock));
+  store.ingest([batch(clock, [], true)], 'live');
+  for (let minute = 0; minute < 60; minute++) {
+    clock = base + minute * 60_000 + 1_000;
+    store.ingest([batch(clock, [connection('a', (minute + 1) * 10, (minute + 1) * 20)])], 'live');
+  }
+  clock = base + 60 * 60_000;
+  const searchedQuery = query({ search: 'яндекс' });
+  assert.deepEqual(store.query(searchedQuery).totals, { uploadBytes: '600', downloadBytes: '1200' });
+  assert.ok(searched <= 2, `one destination must not be searched per time bucket (${searched} calls)`);
+  store.ingest([batch(clock + 1, [connection('a', 601, 1202)])], 'live');
+  assert.equal(store.query(searchedQuery).totals.uploadBytes, '600');
+  store.ingest([batch(base + 30_000, [connection('late', 7, 9)])], 'live');
+  assert.deepEqual(store.query(searchedQuery).totals, { uploadBytes: '607', downloadBytes: '1209' });
+  assert.throws(() => store.ingest([batch(base + 40_000, [
+    connection('rolled-back', 50, 80), connection('invalid', 'invalid', 1),
+  ])], 'live'));
+  assert.equal(store.query(searchedQuery).totals.uploadBytes, '607');
+  clock += 60_000;
+  assert.deepEqual(store.query(searchedQuery).totals, { uploadBytes: '608', downloadBytes: '1211' });
+  store.maintain();
+  assert.equal(store.query(searchedQuery).totals.uploadBytes, '608');
+});

test/web/api-errors.test.js

@@ -143,8 +143,12 @@ test('typed endpoint facade preserves exact request contracts and raw payload id
       assert.equal(await invoke(), payload);
       const [actualUrl, actualOptions] = calls.at(-1);
       assert.equal(actualUrl, url);
-      assert.deepEqual(actualOptions, {
+      assert.ok(actualOptions.signal instanceof AbortSignal);
-        ...options,
+      assert.equal(actualOptions.signal.aborted, false);
+      const { signal: requestedSignal, ...requestOptions } = options;
+      const { signal: actualSignal, ...fetchOptions } = actualOptions;
+      assert.deepEqual(fetchOptions, {
+        ...requestOptions,
         headers: { 'content-type': 'application/json' },
       });
     }
@@ -162,7 +166,9 @@ test('request preserves caller headers, AbortError identity and JSON fallbacks',
     received = [url, options];
     return { ok: true, status: 200, json: async () => value };
   }), value);
-  assert.deepEqual(received, ['/api/test', {
+  assert.ok(received[1].signal instanceof AbortSignal);
+  const { signal, ...options } = received[1];
+  assert.deepEqual([received[0], options], ['/api/test', {
     headers: { 'content-type': 'application/custom', 'x-harbor': 'yes' },
   }]);
 
@@ -188,3 +194,40 @@ test('request preserves caller headers, AbortError identity and JSON fallbacks',
     (error) => error.code === 'CONTROL_UNREACHABLE' && error.status === 503,
   );
 });
+
+test('API deadlines cover headers and response bodies, while caller cancellation keeps its reason', async (t) => {
+  t.mock.timers.enable({ apis: ['setTimeout'] });
+  const stalled = (signal) => new Promise((resolve, reject) => {
+    signal.addEventListener('abort', () => reject(signal.reason), { once: true });
+  });
+  for (const phase of ['headers', 'body']) {
+    let signal;
+    const pending = request('/api/state', {}, async (url, options) => {
+      signal = options.signal;
+      return phase === 'headers' ? stalled(signal)
+        : { ok: true, status: 200, json: () => stalled(signal) };
+    });
+    const checked = assert.rejects(pending, (error) => error.code === 'CONTROL_UNREACHABLE' && error.retryable);
+    await Promise.resolve();
+    t.mock.timers.tick(14_999);
+    assert.equal(signal.aborted, false);
+    t.mock.timers.tick(1);
+    await checked;
+  }
+  const caller = new AbortController();
+  const reason = new DOMException('left the page', 'AbortError');
+  const pending = request('/api/test', { signal: caller.signal }, (url, options) => stalled(options.signal));
+  const checked = assert.rejects(pending, (error) => error === reason);
+  caller.abort(reason);
+  await checked;
+  let commandSignal;
+  const command = request('/api/diagnostics/dns', { method: 'POST' }, (url, options) => {
+    commandSignal = options.signal;
+    return stalled(commandSignal);
+  });
+  const commandChecked = assert.rejects(command, (error) => error.code === 'CONTROL_UNREACHABLE');
+  t.mock.timers.tick(59_999);
+  assert.equal(commandSignal.aborted, false);
+  t.mock.timers.tick(1);
+  await commandChecked;
+});

test/web/app-shell-contract.test.js

@@ -1,6 +1,7 @@
 import assert from 'node:assert/strict';
 import { readFileSync } from 'node:fs';
 import test from 'node:test';
+import { parse } from '@babel/parser';
 
 const index = readFileSync(new URL('../../index.html', import.meta.url), 'utf8');
 const main = readFileSync(new URL('../../src/web/main.tsx', import.meta.url), 'utf8');
@@ -23,3 +24,44 @@ test('App remains the exported composition component without bootstrap side effe
   assert.match(app, /<ClientOverviewPage/);
   assert.match(app, /<StaleBanner/);
 });
+
+test('state polling waits for completion and stops after cleanup', async (t) => {
+  const compiled = readFileSync(new URL('../../.test-dist/src/web/App.js', import.meta.url), 'utf8');
+  const component = parse(compiled, { sourceType: 'module' }).program.body
+    .find((node) => node.declaration?.id?.name === 'App').declaration;
+  const effect = component.body.body.map((node) => node.expression)
+    .filter((node) => node?.callee?.name === 'useEffect')
+    .map((node) => node.arguments[0])
+    .find((node) => compiled.slice(node.start, node.end).includes('loadState('));
+  // Execute the shipped effect with a deferred request; no DOM or copied polling loop.
+  const setup = new Function('loadState', `return (${compiled.slice(effect.start, effect.end)})`);
+  t.mock.timers.enable({ apis: ['setTimeout', 'setInterval'] });
+  let calls = 0;
+  let finish;
+  const start = setup(() => {
+    calls++;
+    return new Promise((resolve) => { finish = resolve; });
+  });
+  const cleanup = start();
+  assert.equal(calls, 1);
+  t.mock.timers.tick(40_000);
+  assert.equal(calls, 1);
+  finish();
+  await Promise.resolve();
+  t.mock.timers.tick(4_999);
+  assert.equal(calls, 1);
+  t.mock.timers.tick(1);
+  assert.equal(calls, 2);
+  cleanup();
+  finish();
+  await Promise.resolve();
+  t.mock.timers.tick(40_000);
+  assert.equal(calls, 2);
+
+  const cleanupScheduled = start();
+  finish();
+  await Promise.resolve();
+  cleanupScheduled();
+  t.mock.timers.tick(40_000);
+  assert.equal(calls, 3);
+});

tsconfig.test.json

@@ -11,6 +11,7 @@
   },
   "include": [
     "src/shared/**/*",
+    "src/web/App.tsx",
     "src/web/api/**/*",
     "src/web/state/harborReducer.ts",
     "src/web/state/operations.ts",