Migrate Harbor state and traffic history to SQLite

docs/product/application-state.md

@@ -1,6 +1,6 @@
 # Harbor application state v1
 
-`GET /api/state` is the canonical Harbor domain snapshot. Successful mutations return the same snapshot as `state`. The persisted owner is `state.json` schema v8; React keeps only drafts, disclosure, focus, animation and transport freshness.
+`GET /api/state` is the canonical Harbor domain snapshot. Successful mutations return the same snapshot as `state`. The persisted owner is the `state` JSON document in `harbor.sqlite` (currently document schema v10). React keeps only drafts, disclosure, focus, animation and transport freshness.
 
 An abbreviated snapshot:
 
@@ -107,7 +107,7 @@ Enabling failover while VPN is stopped validates a temporary dual-channel candid
 
 The dual config keeps one stable inbound and a sing-box selector with `interrupt_exist_connections: false`. A switch changes the outbound for new connections only. Before an automatic switch, the existing `/connections` observer measures VPN byte deltas over a bounded 10-second window. Active or unknown traffic blocks the switch; the public snapshot contains only aggregate speed, connection count and at most three safe device/service labels.
 
-Failover mutations use `PUT /api/failover`, `POST /api/failover/pause` and `POST /api/failover/switch`. Important user events are stored separately in `activity-journal.json` and read through `GET /api/activity-journal`. The journal is not a second state owner, contains no provider URLs or raw diagnostics, uses stable ID cursors and prunes entries after 30 days.
+Failover mutations use `PUT /api/failover`, `POST /api/failover/pause` and `POST /api/failover/switch`. Important user events are separate rows in the `harbor.sqlite` journal table and read through `GET /api/activity-journal`. The journal is not a second state owner, contains no provider URLs or raw diagnostics, uses stable ID cursors and retains at most 30 days and 10,000 entries.
 
 ## Compatibility and migration
 
@@ -115,6 +115,16 @@ Schema v5 migrates the legacy singleton and `subscription-cache.json` into one p
 
 Schema v6 adds the routing-rule outbound. Rules read from schemas v0-v5 migrate to `outbound: "direct"` in their existing order and both desired/applied arrays are normalized together. A schema-v6 rule without a valid outbound is rejected rather than silently rewritten. Schema v7 adds canonical connectivity-diagnostics settings. Schema v8 adds a disabled failover policy, empty runtime history and no applied dual config, so upgrading does not start monitoring or change traffic.
 
-Migration atomically backs up the previous `state.json`. After the embedded profile is committed, Harbor also backs up and removes the legacy subscription cache so there is one persisted owner. Invalid legacy cache/config returns to a truthful stopped first-run state instead of starting stale generated config.
+The initial SQLite migration imports settings, devices and journal in one transaction, including a legacy subscription cache when needed. Original JSON files remain unchanged as transition-time backups, with no parallel writes. Invalid input or a conflicting cache owner aborts migration without replacing state or starting stale configuration. Subsequent starts use only SQLite. Current document normalizers also preserve traffic-display settings and the device inventory's tag catalogue.
 
-The old HTTP projection remains bounded for one release. Schema v8 persistence is not downgrade-compatible: stop Harbor and restore the `state.json.backup-v<fromVersion>-*` matching the rollback binary instead of deploying old code over v8 data. Rolling back before profiles still also requires the matching legacy subscription-cache backup.
+The existing HTTP compatibility projection is unchanged. A pre-SQLite binary cannot read current persistence: restore a complete compatible backup or explicitly export current data before downgrading. Old JSON files do not contain post-migration changes. See [state recovery](../recovery/state-recovery.md).
+
+## Local traffic history
+
+`GET /api/traffic/history` reads only the local collector's `traffic.sqlite`, through a worker and, on a split Gateway, the existing control/dataplane socket. It accepts `range=24h|7d|30d|90d`, `level=service|domain|hostname|ip`, parent filters `service/domain/hostname`, `originId`, `route=all|vpn|direct|other`, `search`, `offset` and an optional `until` timestamp in milliseconds. Pages contain at most 100 groups. Bytes are decimal strings, preserving integers above JavaScript's safe-number range.
+
+The response reports the requested/effective period, first available observation, minute/hour boundary, current collector state, gap count and partial coverage. `query.until` is the effective end of a complete bucket; use it for matching drilldown and pagination. Current history can lag by one minute. Data older than 7 days is hourly; retention and rollup can change available granularity between requests.
+
+History starts with the new collector, not with previously accumulated device counters. Full observed hostnames and IPs remain distinct; service grouping is a local presentation classification, not proof of ownership of an IP. Unknown domains remain unknown. A history storage error reports unavailable/partial data without stopping VPN or exported metrics.
+
+Prometheus is optional, independent and contains only exported metrics—not a copy of this database. There is no synchronization, automatic UI fallback, scrape backfill or restoration from Prometheus. Local cleanup does not delete external history or change its retention.