Update Harbor client and gateway integration workflows
Build and Deploy Gateway / build-and-push (push) Failing after 14s
Build and Deploy Gateway / deploy (push) Has been skipped

This commit is contained in:
2026-08-19 18:16:10 +03:00
parent 416b2b294a
commit daec12e013
63 changed files with 5016 additions and 108 deletions
+8
View File
@@ -75,6 +75,14 @@ http://АДРЕС-GATEWAY:3456
Приватные и локальные адреса не отправляются в VPN, поэтому устройства сохраняют доступ к домашней сети. Общий прокси по умолчанию принимает подключения только из приватных сетей. Приватные и локальные адреса не отправляются в VPN, поэтому устройства сохраняют доступ к домашней сети. Общий прокси по умолчанию принимает подключения только из приватных сетей.
### Резервный канал Gateway
После добавления подписок откройте «Резерв» — вторую кнопку в правой панели. Выберите основной и резервный серверы (они могут быть из одной или разных подписок), сервисы для проверки и отдельный таймаут каждого сервиса. Там же настраиваются длительность сбоя и восстановления, порог активного трафика, период тишины и защита от частых переключений.
При включении Harbor заранее проверяет dual-конфигурацию. Если VPN остановлен, она начнёт работать только после следующего обычного нажатия питания; сохранение само VPN не включает. Переключение меняет маршрут только для новых соединений — уже открытые соединения не закрываются. Если через VPN идёт активный трафик или его активность нельзя надёжно определить, Harbor ждёт и показывает скорость, число передающих соединений и безопасные подписи основных блокирующих потоков.
Выключенный резерв полностью пассивен: Harbor не запускает проверки, таймер выбора и отдельный подсчёт активности. Если dual-конфигурация уже загружена, отключение не перезапускает VPN и не меняет текущий маршрут; обычный stop и следующий запуск вернут single-channel config. Последние важные события — включение VPN, обновления подписок, переключения и ошибки — доступны в последней кнопке «Журнал» и хранятся 30 дней без ссылок подписок и сырых диагностических ответов.
### Устройства Gateway ### Устройства Gateway
Откройте «Устройства» в правой панели Gateway — подписка для просмотра списка не требуется. Harbor раз в 15 секунд читает локальную таблицу соседей и показывает каждое устройство одной компактной строкой: заданное название, hostname или IP, последний контакт, выбранный график трафика и иконку применённого маршрута. По умолчанию график показывает приблизительный выход `VPN`/`Direct`; переключатель `Вход` возвращает накопленную разбивку `Gateway`/`Прокси`. Наведите курсор на имя или переведите на него фокус, чтобы открыть IP, MAC и доступный hostname; нажатие на значение копирует его. Hostname определяется через локальное обратное разрешение имён и может отсутствовать, если сеть его не публикует. Технические interface и manufacturer продолжают храниться для идентификации, но не занимают место в строке. Список разделён на «Закреплённые», «Остальные» и «Фоновые»: последняя группа сохраняется между перезапусками, показывает только identity/presence и кнопку возврата без графика, traffic и route controls. Название, закрепление, фоновое положение и накопленные totals сохраняются в volume Gateway, пока устройство остаётся в inventory. Откройте «Устройства» в правой панели Gateway — подписка для просмотра списка не требуется. Harbor раз в 15 секунд читает локальную таблицу соседей и показывает каждое устройство одной компактной строкой: заданное название, hostname или IP, последний контакт, выбранный график трафика и иконку применённого маршрута. По умолчанию график показывает приблизительный выход `VPN`/`Direct`; переключатель `Вход` возвращает накопленную разбивку `Gateway`/`Прокси`. Наведите курсор на имя или переведите на него фокус, чтобы открыть IP, MAC и доступный hostname; нажатие на значение копирует его. Hostname определяется через локальное обратное разрешение имён и может отсутствовать, если сеть его не публикует. Технические interface и manufacturer продолжают храниться для идентификации, но не занимают место в строке. Список разделён на «Закреплённые», «Остальные» и «Фоновые»: последняя группа сохраняется между перезапусками, показывает только identity/presence и кнопку возврата без графика, traffic и route controls. Название, закрепление, фоновое положение и накопленные totals сохраняются в volume Gateway, пока устройство остаётся в inventory.
+15 -3
View File
@@ -1,6 +1,6 @@
# Harbor application state v1 # Harbor application state v1
`GET /api/state` is the canonical Harbor domain snapshot. Successful mutations return the same snapshot as `state`. The persisted owner is `state.json` schema v6; React keeps only drafts, disclosure, focus, animation and transport freshness. `GET /api/state` is the canonical Harbor domain snapshot. Successful mutations return the same snapshot as `state`. The persisted owner is `state.json` schema v8; React keeps only drafts, disclosure, focus, animation and transport freshness.
An abbreviated snapshot: An abbreviated snapshot:
@@ -67,6 +67,8 @@ The frontend accepts only newer snapshots. Equal revisions preserve object ident
Browser boot/offline/stale state remains a transport envelope beside the domain snapshot. A transport failure retains the last accepted domain state. Browser boot/offline/stale state remains a transport envelope beside the domain snapshot. A transport failure retains the last accepted domain state.
Failover health and traffic observations are transient: they do not write `state.json` or increase the domain `revision` every few seconds. Each control-process lifetime publishes a new `observationEpoch` and increasing `observationSequence`. At the same domain revision the browser accepts only a newer sequence from the active epoch; after accepting a new epoch it retires the old one so a late response cannot restore stale health.
## Desired and applied identity ## Desired and applied identity
`desiredProfileId` and each profile's `desiredServerId` record the next local choice. `appliedProfileId`, `appliedServerId` and `appliedServerSnapshot` describe the runtime that actually owns traffic. There is no third `activeProfileId`. `desiredProfileId` and each profile's `desiredServerId` record the next local choice. `appliedProfileId`, `appliedServerId` and `appliedServerSnapshot` describe the runtime that actually owns traffic. There is no third `activeProfileId`.
@@ -97,12 +99,22 @@ The route-rules mutation uses the whole-array `PUT /api/route-rules/v2` with `ru
In Connect `gateway-direct`, local user rules are intentionally omitted and the snapshot reports no active or pending local rules. Gateway device policy `Напрямую` bypasses sing-box before these rules; policy `VPN` and an ordinary local/Gateway VPN pipeline evaluate them. In Connect `gateway-direct`, local user rules are intentionally omitted and the snapshot reports no active or pending local rules. Gateway device policy `Напрямую` bypasses sing-box before these rules; policy `VPN` and an ordinary local/Gateway VPN pipeline evaluate them.
## Gateway failover and activity journal
`failoverPolicy` is the desired Gateway-only policy: master enable, primary/reserve profile and server, service checks with individual timeouts, health windows, active-traffic guard and flap protection. `failoverRuntimeState` stores switch history, hold and quarantine deadlines separately, so a runtime decision is not mistaken for a desired configuration change. `appliedFailoverPolicy` stores only the two loaded targets and safe configuration fingerprints.
Enabling failover while VPN is stopped validates a temporary dual-channel candidate but does not start VPN. The dual config is loaded only by the next explicit power-on. Enabling it over a running single-channel config remains pending until a later stop and power-on. Disabling automation stops its timer, probes and activity collector immediately, but does not restart sing-box or change the selected route; the already loaded dual config is reported as `passive-loaded` until the ordinary stop lifecycle clears it.
The dual config keeps one stable inbound and a sing-box selector with `interrupt_exist_connections: false`. A switch changes the outbound for new connections only. Before an automatic switch, the existing `/connections` observer measures VPN byte deltas over a bounded 10-second window. Active or unknown traffic blocks the switch; the public snapshot contains only aggregate speed, connection count and at most three safe device/service labels.
Failover mutations use `PUT /api/failover`, `POST /api/failover/pause` and `POST /api/failover/switch`. Important user events are stored separately in `activity-journal.json` and read through `GET /api/activity-journal`. The journal is not a second state owner, contains no provider URLs or raw diagnostics, uses stable ID cursors and prunes entries after 30 days.
## Compatibility and migration ## Compatibility and migration
Schema v5 migrates the legacy singleton and `subscription-cache.json` into one profile named `Основной`. Stable endpoint identity preserves unambiguous desired/applied selection, including transport variants whose normalized IDs differ from old labels. An explicitly stopped legacy state does not resurrect an old applied target. Schema v5 migrates the legacy singleton and `subscription-cache.json` into one profile named `Основной`. Stable endpoint identity preserves unambiguous desired/applied selection, including transport variants whose normalized IDs differ from old labels. An explicitly stopped legacy state does not resurrect an old applied target.
Schema v6 adds the routing-rule outbound. Rules read from schemas v0-v5 migrate to `outbound: "direct"` in their existing order and both desired/applied arrays are normalized together. A schema-v6 rule without a valid outbound is rejected rather than silently rewritten. Schema v6 adds the routing-rule outbound. Rules read from schemas v0-v5 migrate to `outbound: "direct"` in their existing order and both desired/applied arrays are normalized together. A schema-v6 rule without a valid outbound is rejected rather than silently rewritten. Schema v7 adds canonical connectivity-diagnostics settings. Schema v8 adds a disabled failover policy, empty runtime history and no applied dual config, so upgrading does not start monitoring or change traffic.
Migration atomically backs up the previous `state.json`. After the embedded profile is committed, Harbor also backs up and removes the legacy subscription cache so there is one persisted owner. Invalid legacy cache/config returns to a truthful stopped first-run state instead of starting stale generated config. Migration atomically backs up the previous `state.json`. After the embedded profile is committed, Harbor also backs up and removes the legacy subscription cache so there is one persisted owner. Invalid legacy cache/config returns to a truthful stopped first-run state instead of starting stale generated config.
The old HTTP projection remains bounded for one release. Schema v6 persistence is not downgrade-compatible: stop Harbor and restore the `state.json.backup-v<fromVersion>-*` matching the rollback binary instead of deploying old code over v6 data. Rolling back before profiles still also requires the matching legacy subscription-cache backup. The old HTTP projection remains bounded for one release. Schema v8 persistence is not downgrade-compatible: stop Harbor and restore the `state.json.backup-v<fromVersion>-*` matching the rollback binary instead of deploying old code over v8 data. Rolling back before profiles still also requires the matching legacy subscription-cache backup.
+8 -2
View File
@@ -1,6 +1,6 @@
# Harbor state recovery # Harbor state recovery
Harbor keeps the existing data directory and `state.json` path. The current persisted format is `schemaVersion: 6`: schema v2 introduced local route rules, v3 added rule enabled state, v4 added stable server IDs, v5 embeds the canonical `profiles[]` collection with desired/applied profile identity, and v6 adds an explicit `vpn` or `direct` outbound to every route rule. Harbor keeps the existing data directory and `state.json` path. The current persisted format is `schemaVersion: 8`: schema v2 introduced local route rules, v3 added rule enabled state, v4 added stable server IDs, v5 embeds the canonical `profiles[]` collection with desired/applied profile identity, v6 adds an explicit `vpn` or `direct` outbound to every route rule, v7 stores connectivity-diagnostics settings, and v8 adds Gateway failover state.
## Atomic writes ## Atomic writes
@@ -36,6 +36,12 @@ state.json.backup-v5-2026-08-17T12-00-00-000Z
After migration, malformed schema-v6 rules are rejected; Harbor does not reinterpret a missing or unknown outbound as direct. After migration, malformed schema-v6 rules are rejected; Harbor does not reinterpret a missing or unknown outbound as direct.
## Migration to failover
Schemas v0-v7 migrate to v8 with failover disabled, empty switch history and no applied dual config. Migration does not start probes, enable traffic accounting or change the single-channel runtime. The original state is preserved as `state.json.backup-v<fromVersion>-*` before the atomic replacement.
The separate `activity-journal.json` is created on the first important event. It uses the same atomic write and corrupt-file isolation mechanism as state, retains at most 30 days, and can be removed while Harbor is stopped without affecting subscriptions, routing or VPN startup.
## Corrupt JSON ## Corrupt JSON
If `state.json` cannot be parsed, Harbor renames the exact damaged bytes to: If `state.json` cannot be parsed, Harbor renames the exact damaged bytes to:
@@ -55,4 +61,4 @@ Perform recovery while Harbor is stopped:
3. Restore only matching state/cache backups to their original filenames. 3. Restore only matching state/cache backups to their original filenames.
4. Start Harbor and verify `GET /api/state` before applying a profile. 4. Start Harbor and verify `GET /api/state` before applying a profile.
A pre-v6 binary cannot interpret the explicit ordered VPN/Direct rule contract. Restore `state.json.backup-v<fromVersion>-*` matching the rollback binary; deploying old code over schema v6 is not safe. A rollback to pre-v5 additionally requires the matching state and subscription-cache backups because that binary cannot interpret canonical profiles. A pre-v8 binary cannot interpret failover state. Restore `state.json.backup-v<fromVersion>-*` matching the rollback binary; deploying old code over schema v8 is not safe. A rollback to pre-v5 additionally requires the matching state and subscription-cache backups because that binary cannot interpret canonical profiles.
+3
View File
@@ -15,6 +15,8 @@ export const settings = {
port: parsePort(process.env.PORT, 3456), port: parsePort(process.env.PORT, 3456),
proxyPort, proxyPort,
diagnosticsProxyPort: parsePort(process.env.DIAGNOSTICS_PROXY_PORT, 18080), diagnosticsProxyPort: parsePort(process.env.DIAGNOSTICS_PROXY_PORT, 18080),
failoverPrimaryProxyPort: parsePort(process.env.FAILOVER_PRIMARY_PROXY_PORT, 18081),
failoverReserveProxyPort: parsePort(process.env.FAILOVER_RESERVE_PROXY_PORT, 18082),
singboxApiPort: parsePort(process.env.SING_BOX_API_PORT, 19090), singboxApiPort: parsePort(process.env.SING_BOX_API_PORT, 19090),
tproxyPort: parsePort(process.env.TPROXY_PORT, 7895), tproxyPort: parsePort(process.env.TPROXY_PORT, 7895),
tproxyMark: process.env.TPROXY_MARK || "1", tproxyMark: process.env.TPROXY_MARK || "1",
@@ -40,6 +42,7 @@ export const settings = {
cachePath: process.env.SING_BOX_CACHE || "/var/lib/sing-box/cache.db", cachePath: process.env.SING_BOX_CACHE || "/var/lib/sing-box/cache.db",
statePath: path.join(dataDir, "state.json"), statePath: path.join(dataDir, "state.json"),
deviceStatePath: path.join(dataDir, "devices.json"), deviceStatePath: path.join(dataDir, "devices.json"),
activityJournalPath: path.join(dataDir, "activity-journal.json"),
subscriptionCachePath: path.join(dataDir, "subscription-cache.json"), subscriptionCachePath: path.join(dataDir, "subscription-cache.json"),
sharedProxyHost: process.env.SHARED_PROXY_HOST || "", sharedProxyHost: process.env.SHARED_PROXY_HOST || "",
hostNetworkStatePath: hostNetworkStatePath:
+33
View File
@@ -13,6 +13,7 @@ import {
createDomainTrafficService, createDomainTrafficService,
readSingboxConnections, readSingboxConnections,
} from './services/domainTrafficService.js'; } from './services/domainTrafficService.js';
import { createSingboxSelectorService } from './services/singboxSelectorService.js';
const socketPath = settings.dataplaneSocket; const socketPath = settings.dataplaneSocket;
const runtime = createSingboxRuntime({ const runtime = createSingboxRuntime({
@@ -40,6 +41,11 @@ const devicePolicy = createDevicePolicyService({
const connectivityDiagnostics = createConnectivityDiagnosticsService({ const connectivityDiagnostics = createConnectivityDiagnosticsService({
proxyPort: settings.diagnosticsProxyPort, proxyPort: settings.diagnosticsProxyPort,
}); });
const failoverDiagnostics = {
primary: createConnectivityDiagnosticsService({ proxyPort: settings.failoverPrimaryProxyPort }),
reserve: createConnectivityDiagnosticsService({ proxyPort: settings.failoverReserveProxyPort }),
};
const selector = createSingboxSelectorService({ port: settings.singboxApiPort });
const domainTraffic = createDomainTrafficService({ const domainTraffic = createDomainTrafficService({
observe: () => readSingboxConnections(settings.singboxApiPort), observe: () => readSingboxConnections(settings.singboxApiPort),
devices: () => traffic.snapshot().devices, devices: () => traffic.snapshot().devices,
@@ -126,6 +132,33 @@ const server = http.createServer(async (req: IncomingMessage, res: ServerRespons
target, target,
})); }));
} }
if (req.method === 'POST' && req.url === '/failover/probe') {
const { role, services = [], target = null, timeoutMs = 6_000 } = record(await readJson(req));
if (role !== 'primary' && role !== 'reserve') throw new Error('Неизвестная failover role');
return sendJson(res, 200, await failoverDiagnostics[role].runVpn({ services, target, timeoutMs: Number(timeoutMs) }));
}
if (req.method === 'GET' && req.url === '/failover/selector') {
return sendJson(res, 200, await selector.read());
}
if (req.method === 'PUT' && req.url === '/failover/selector') {
const { role } = record(await readJson(req));
if (role !== 'primary' && role !== 'reserve') throw new Error('Неизвестная failover role');
return sendJson(res, 200, await selector.select(role));
}
if (req.method === 'PUT' && req.url === '/failover/activity') {
const { enabled } = record(await readJson(req));
if (enabled === true) domainTraffic.enableActivity();
else domainTraffic.disableActivity();
return sendJson(res, 200, { enabled: enabled === true });
}
if (req.method === 'POST' && req.url === '/failover/activity/read') {
const { thresholdBytesPerSecond = 0 } = record(await readJson(req));
return sendJson(res, 200, { activity: domainTraffic.activitySnapshot(thresholdBytesPerSecond) });
}
if (req.method === 'POST' && req.url === '/config/check') {
const { config } = record(await readJson(req));
return sendJson(res, 200, runtime.checkConfig(config));
}
if (req.method === 'POST' && req.url === '/apply') { if (req.method === 'POST' && req.url === '/apply') {
return sendJson(res, 200, await runtime.apply()); return sendJson(res, 200, await runtime.apply());
} }
+10
View File
@@ -82,6 +82,16 @@ export function createDataplaneClient(socketPath: string, send: SendDataplaneReq
throw new HarborError('DIAGNOSTICS_FAILED', { cause }); throw new HarborError('DIAGNOSTICS_FAILED', { cause });
} }
}, },
checkConfig: (config: unknown) => send(socketPath, '/config/check', 'POST', { config }, 15_000),
runFailoverProbe: (role: 'primary' | 'reserve', services: unknown, target: unknown, timeoutMs: number) => (
send(socketPath, '/failover/probe', 'POST', { role, services, target, timeoutMs }, timeoutMs + 10_000)
),
readFailoverSelector: () => send(socketPath, '/failover/selector', 'GET'),
selectFailoverRole: (role: 'primary' | 'reserve') => send(socketPath, '/failover/selector', 'PUT', { role }),
setFailoverActivityEnabled: (enabled: boolean) => send(socketPath, '/failover/activity', 'PUT', { enabled }),
readFailoverActivity: (thresholdBytesPerSecond: number) => (
send(socketPath, '/failover/activity/read', 'POST', { thresholdBytesPerSecond })
),
apply: () => update('/apply', 'POST'), apply: () => update('/apply', 'POST'),
restart: () => update('/restart', 'POST'), restart: () => update('/restart', 'POST'),
stop: () => update('/stop', 'POST'), stop: () => update('/stop', 'POST'),
@@ -4,7 +4,9 @@ import {
type StoredState, type StoredState,
} from '../../../shared/contracts/state.js'; } from '../../../shared/contracts/state.js';
import { HarborError } from '../../../shared/errors.js'; import { HarborError } from '../../../shared/errors.js';
import { finishRollback } from '../../services/rollback.js'; import { finishRollback, type RollbackStep } from '../../services/rollback.js';
import type { AppliedFailoverPolicy } from '../../../shared/failover.js';
import type { ActivityJournalEventInput } from '../../../shared/activityJournal.js';
interface ConnectionServiceDependencies { interface ConnectionServiceDependencies {
state: { state: {
@@ -26,6 +28,18 @@ interface ConnectionServiceDependencies {
restartCommand(): Promise<RuntimeCommandResult>; restartCommand(): Promise<RuntimeCommandResult>;
}; };
route?: { isGatewayDirect(): boolean }; route?: { isGatewayDirect(): boolean };
failover?: {
build(state: StoredState, source?: 'desired' | 'applied'): {
config: unknown;
applied: AppliedFailoverPolicy;
primaryProfile: StoredProfile;
primaryServer: StoredProfile['servers'][number];
};
prepareActivation(role: 'primary' | 'reserve'): Promise<unknown>;
restoreAppliedActivation(state: StoredState): Promise<unknown>;
reconcile(): Promise<unknown>;
};
onEvent?: (event: ActivityJournalEventInput) => void;
serialize<T>(operation: () => Promise<T>): Promise<T>; serialize<T>(operation: () => Promise<T>): Promise<T>;
now(): Date; now(): Date;
} }
@@ -72,6 +86,33 @@ function withDesiredServer(state: StoredState, profile: StoredProfile, serverId:
} }
export function createConnectionService(dependencies: ConnectionServiceDependencies) { export function createConnectionService(dependencies: ConnectionServiceDependencies) {
const activationTarget = (
state: StoredState,
applied: AppliedFailoverPolicy,
role: 'primary' | 'reserve',
) => {
const target = applied[role];
const profile = profileById(state, target.profileId);
const server = profile?.servers.find(({ id }) => id === target.serverId);
if (!profile || !server) throw new HarborError('SERVER_NOT_FOUND');
return { profile, server };
};
const finishConnectionRollback = async (error: unknown, steps: RollbackStep[], message: string) => {
try {
await finishRollback(error, steps, message);
} catch (cause) {
const code = cause && typeof cause === 'object' && 'code' in cause
&& /^[A-Z0-9_]{1,50}$/.test(String(cause.code)) ? String(cause.code) : 'UNKNOWN';
dependencies.onEvent?.({
type: 'connection.failed',
severity: 'error',
source: 'connection',
dedupeKey: `connection.failed:${dependencies.state.read().revision}:${code}`,
data: { errorCode: code },
});
throw cause;
}
};
const applyWithinQueue = async ( const applyWithinQueue = async (
previousState: StoredState, previousState: StoredState,
profile: StoredProfile, profile: StoredProfile,
@@ -88,18 +129,26 @@ export function createConnectionService(dependencies: ConnectionServiceDependenc
if (!selectedServer) throw new HarborError('SERVER_NOT_FOUND'); if (!selectedServer) throw new HarborError('SERVER_NOT_FOUND');
if (!profile.subscriptionConfig) throw new HarborError('CONFIG_INVALID'); if (!profile.subscriptionConfig) throw new HarborError('CONFIG_INVALID');
const wasRunning = await dependencies.runtime.isRunning();
if (wasRunning && previousState.failoverPolicy?.enabled) {
dependencies.state.update((state) => withDesiredServer(state, profile, selectedServer.id));
return { profileId: profile.id, serverId: selectedServer.id, selectedTag: selectedServer.label };
}
if (dependencies.route?.isGatewayDirect()) { if (dependencies.route?.isGatewayDirect()) {
dependencies.state.update((state) => withDesiredServer(state, profile, selectedServer.id)); dependencies.state.update((state) => withDesiredServer(state, profile, selectedServer.id));
return { profileId: profile.id, serverId: selectedServer.id, selectedTag: selectedServer.label }; return { profileId: profile.id, serverId: selectedServer.id, selectedTag: selectedServer.label };
} }
const nextConfig = dependencies.config.build( const failoverCandidate = previousState.failoverPolicy?.enabled
? dependencies.failover?.build(previousState, wasRunning ? 'applied' : 'desired')
: null;
const nextConfig = failoverCandidate?.config || dependencies.config.build(
profile.subscriptionConfig, profile.subscriptionConfig,
selectedServer.id, selectedServer.id,
previousState.routeRules, previousState.routeRules,
); );
const previousConfig = dependencies.config.read(); const previousConfig = dependencies.config.read();
const wasRunning = await dependencies.runtime.isRunning();
let configMutationStarted = false; let configMutationStarted = false;
let runtimeMutationStarted = false; let runtimeMutationStarted = false;
let stateCommitStarted = false; let stateCommitStarted = false;
@@ -109,18 +158,20 @@ export function createConnectionService(dependencies: ConnectionServiceDependenc
dependencies.config.write(nextConfig); dependencies.config.write(nextConfig);
runtimeMutationStarted = true; runtimeMutationStarted = true;
await dependencies.runtime.start(); await dependencies.runtime.start();
if (failoverCandidate) await dependencies.failover!.prepareActivation('primary');
stateCommitStarted = true; stateCommitStarted = true;
dependencies.state.update((state) => ({ dependencies.state.update((state) => ({
...withDesiredServer(state, profile, selectedServer.id), ...withDesiredServer(state, profile, selectedServer.id),
connectionDesired: 'running', connectionDesired: 'running',
appliedProfileId: profile.id, appliedProfileId: failoverCandidate?.primaryProfile.id || profile.id,
appliedServerId: selectedServer.id, appliedServerId: failoverCandidate?.primaryServer.id || selectedServer.id,
appliedServerSnapshot: selectedServer, appliedServerSnapshot: failoverCandidate?.primaryServer || selectedServer,
appliedFailoverPolicy: failoverCandidate?.applied || null,
appliedAt: dependencies.now().toISOString(), appliedAt: dependencies.now().toISOString(),
appliedRouteRules: state.routeRules, appliedRouteRules: state.routeRules,
})); }));
} catch (error) { } catch (error) {
await finishRollback(error, [ await finishConnectionRollback(error, [
...(stateCommitStarted ? [{ run: () => dependencies.state.update(() => previousState) }] : []), ...(stateCommitStarted ? [{ run: () => dependencies.state.update(() => previousState) }] : []),
...(configMutationStarted ? [{ ...(configMutationStarted ? [{
run: () => previousConfig === null run: () => previousConfig === null
@@ -128,12 +179,28 @@ export function createConnectionService(dependencies: ConnectionServiceDependenc
: dependencies.config.restore(previousConfig), : dependencies.config.restore(previousConfig),
}] : []), }] : []),
...(runtimeMutationStarted ? [{ ...(runtimeMutationStarted ? [{
run: () => wasRunning ? dependencies.runtime.start() : dependencies.runtime.stop(), run: async () => {
if (!wasRunning) return dependencies.runtime.stop();
await dependencies.runtime.start();
await dependencies.failover?.restoreAppliedActivation(previousState);
},
runtime: true, runtime: true,
}] : []), }] : []),
], 'Connection rollback failed'); ], 'Connection rollback failed');
} }
await dependencies.failover?.reconcile();
dependencies.onEvent?.({
type: 'connection.started',
severity: 'info',
source: 'connection',
dedupeKey: `connection.started:${dependencies.state.read().revision}`,
data: {
profileLabel: failoverCandidate?.primaryProfile.label || profile.label,
serverLabel: failoverCandidate?.primaryServer.label || selectedServer.label,
},
});
return { profileId: profile.id, serverId: selectedServer.id, selectedTag: selectedServer.label }; return { profileId: profile.id, serverId: selectedServer.id, selectedTag: selectedServer.label };
}; };
@@ -186,16 +253,29 @@ export function createConnectionService(dependencies: ConnectionServiceDependenc
appliedProfileId: '', appliedProfileId: '',
appliedServerId: '', appliedServerId: '',
appliedServerSnapshot: null, appliedServerSnapshot: null,
appliedFailoverPolicy: null,
})); }));
} catch (error) { } catch (error) {
await finishRollback(error, [ await finishConnectionRollback(error, [
...(runtimeMutationStarted && wasRunning !== null ? [{ ...(runtimeMutationStarted && wasRunning !== null ? [{
run: () => wasRunning ? dependencies.runtime.start() : dependencies.runtime.stop(), run: async () => {
if (!wasRunning) return dependencies.runtime.stop();
await dependencies.runtime.start();
await dependencies.failover?.restoreAppliedActivation(previousState);
},
runtime: true, runtime: true,
}] : []), }] : []),
...(stateCommitStarted ? [{ run: () => dependencies.state.update(() => previousState) }] : []), ...(stateCommitStarted ? [{ run: () => dependencies.state.update(() => previousState) }] : []),
], 'Connection rollback failed'); ], 'Connection rollback failed');
} }
await dependencies.failover?.reconcile();
dependencies.onEvent?.({
type: 'connection.stopped',
severity: 'info',
source: 'connection',
dedupeKey: `connection.stopped:${dependencies.state.read().revision}`,
data: {},
});
}); });
const restart = () => dependencies.serialize(async () => { const restart = () => dependencies.serialize(async () => {
@@ -213,7 +293,18 @@ export function createConnectionService(dependencies: ConnectionServiceDependenc
? previousState.appliedServerSnapshot ? previousState.appliedServerSnapshot
: null); : null);
if (!server || !profile.subscriptionConfig) throw new HarborError('CONFIG_INVALID'); if (!server || !profile.subscriptionConfig) throw new HarborError('CONFIG_INVALID');
const candidateConfig = dependencies.config.build( const failoverCandidate = previousState.failoverPolicy?.enabled
? dependencies.failover?.build(previousState, wasRunning ? 'applied' : 'desired')
: null;
const activationRole = failoverCandidate && wasRunning
&& previousState.appliedProfileId === failoverCandidate.applied.reserve.profileId
&& previousState.appliedServerId === failoverCandidate.applied.reserve.serverId
? 'reserve' as const
: 'primary' as const;
const failoverTarget = failoverCandidate
? activationTarget(previousState, failoverCandidate.applied, activationRole)
: null;
const candidateConfig = failoverCandidate?.config || dependencies.config.build(
profile.subscriptionConfig, profile.subscriptionConfig,
server.id, server.id,
previousState.routeRules, previousState.routeRules,
@@ -229,30 +320,47 @@ export function createConnectionService(dependencies: ConnectionServiceDependenc
const command = await dependencies.runtime.restartCommand(); const command = await dependencies.runtime.restartCommand();
runtimeMutationStarted = command.mutationStarted; runtimeMutationStarted = command.mutationStarted;
if (!command.ok) throw command.error; if (!command.ok) throw command.error;
if (failoverCandidate) await dependencies.failover!.prepareActivation(activationRole);
stateCommitStarted = true; stateCommitStarted = true;
dependencies.state.update((state) => ({ dependencies.state.update((state) => ({
...state, ...state,
desiredProfileId: wasRunning ? state.desiredProfileId : profile.id, desiredProfileId: wasRunning ? state.desiredProfileId : profile.id,
appliedProfileId: profile.id, appliedProfileId: failoverTarget?.profile.id || profile.id,
appliedServerId: server.id, appliedServerId: failoverTarget?.server.id || server.id,
appliedServerSnapshot: server, appliedServerSnapshot: failoverTarget?.server || server,
appliedFailoverPolicy: failoverCandidate?.applied || null,
connectionDesired: 'running', connectionDesired: 'running',
appliedRouteRules: dependencies.route?.isGatewayDirect() ? [] : state.routeRules, appliedRouteRules: dependencies.route?.isGatewayDirect() ? [] : state.routeRules,
})); }));
} catch (error) { } catch (error) {
await finishRollback(error, [ await finishConnectionRollback(error, [
...(configMutationStarted ? [{ ...(configMutationStarted ? [{
run: () => previousConfig === null run: () => previousConfig === null
? dependencies.config.remove() ? dependencies.config.remove()
: dependencies.config.restore(previousConfig), : dependencies.config.restore(previousConfig),
}] : []), }] : []),
...(runtimeMutationStarted ? [{ ...(runtimeMutationStarted ? [{
run: () => wasRunning ? dependencies.runtime.start() : dependencies.runtime.stop(), run: async () => {
if (!wasRunning) return dependencies.runtime.stop();
await dependencies.runtime.start();
await dependencies.failover?.restoreAppliedActivation(previousState);
},
runtime: true, runtime: true,
}] : []), }] : []),
...(stateCommitStarted ? [{ run: () => dependencies.state.update(() => previousState) }] : []), ...(stateCommitStarted ? [{ run: () => dependencies.state.update(() => previousState) }] : []),
], 'Connection rollback failed'); ], 'Connection rollback failed');
} }
await dependencies.failover?.reconcile();
dependencies.onEvent?.({
type: 'connection.started',
severity: 'info',
source: 'connection',
dedupeKey: `connection.started:${dependencies.state.read().revision}`,
data: {
profileLabel: failoverTarget?.profile.label || profile.label,
serverLabel: failoverTarget?.server.label || server.label,
},
});
}); });
return { apply, activate, stop, restart }; return { apply, activate, stop, restart };
@@ -0,0 +1,671 @@
import crypto from 'node:crypto';
import {
createIdleFailoverSnapshot,
isFailoverConfigured,
nextFailoverDecision,
normalizeFailoverPolicy,
type AppliedFailoverPolicy,
type FailoverDecisionMemory,
type FailoverHealth,
type FailoverPolicy,
type FailoverRole,
type FailoverSnapshot,
} from '../../../shared/failover.js';
import type { HarborServer, StoredState } from '../../../shared/contracts/state.js';
import type { ActivityJournalEventInput } from '../../../shared/activityJournal.js';
import { HarborError } from '../../../shared/errors.js';
interface Candidate {
config: unknown;
applied: AppliedFailoverPolicy;
}
interface FailoverServiceDependencies {
state: {
read(): StoredState;
update(mutator: (state: StoredState) => Record<string, unknown>): StoredState;
};
runtime: { isRunning(): Promise<boolean> };
dataplane: {
checkConfig(config: unknown): Promise<unknown>;
runFailoverProbe(role: FailoverRole, services: unknown, target: string, timeoutMs: number): Promise<unknown>;
readFailoverSelector(): Promise<unknown>;
selectFailoverRole(role: FailoverRole): Promise<unknown>;
setFailoverActivityEnabled(enabled: boolean): Promise<unknown>;
readFailoverActivity(thresholdBytesPerSecond: number): Promise<unknown>;
};
buildCandidate(state: StoredState): Candidate;
serialize<T>(operation: () => Promise<T>): Promise<T>;
scheduler?: {
setTimeout(callback: () => void, intervalMs: number): NodeJS.Timeout;
clearTimeout(timer: NodeJS.Timeout): void;
};
now?: () => Date;
onWarning?: (error: unknown) => void;
onSwitch?: (from: FailoverRole, to: FailoverRole, reason: string) => void;
onEvent?: (event: ActivityJournalEventInput) => void;
}
const record = (value: unknown): Record<string, unknown> => (
value && typeof value === 'object' && !Array.isArray(value) ? value as Record<string, unknown> : {}
);
function targetServer(state: StoredState, role: FailoverRole): HarborServer | null {
const target = state.appliedFailoverPolicy?.[role] || state.failoverPolicy[role];
return state.profiles.find(({ id }) => id === target.profileId)
?.servers.find(({ id }) => id === target.serverId) || null;
}
function currentRole(state: StoredState): FailoverRole | null {
const applied = state.appliedFailoverPolicy;
if (!applied) return null;
for (const role of ['primary', 'reserve'] as const) {
if (
state.appliedProfileId === applied[role].profileId
&& state.appliedServerId === applied[role].serverId
) return role;
}
return null;
}
function probeHealth(value: unknown): boolean {
const vpn = record(record(value).vpn);
const sites = Array.isArray(vpn.sites) ? vpn.sites.map(record) : [];
return sites.length === 1 && sites[0].status === 'available';
}
function safeErrorCode(error: unknown) {
const code = error && typeof error === 'object' && 'code' in error ? String(error.code) : '';
return /^[A-Z0-9_]{1,50}$/.test(code) ? code : 'UNKNOWN';
}
export function createFailoverService(dependencies: FailoverServiceDependencies) {
const scheduler = dependencies.scheduler || {
setTimeout: (callback: () => void, intervalMs: number) => setTimeout(callback, intervalMs),
clearTimeout: (timer: NodeJS.Timeout) => clearTimeout(timer),
};
const now = dependencies.now || (() => new Date());
const epoch = crypto.randomUUID();
let sequence = 0;
let generation = 0;
let timer: NodeJS.Timeout | null = null;
let collectorEnabled: boolean | null = null;
let roundPromise: Promise<void> | null = null;
let decisionMemory: FailoverDecisionMemory | undefined;
let snapshot = createIdleFailoverSnapshot(dependencies.state.read().failoverPolicy, epoch, sequence);
function appliedMatchesDesired(state: StoredState) {
if (!state.appliedFailoverPolicy) return false;
try {
return JSON.stringify(dependencies.buildCandidate(state).applied)
=== JSON.stringify(state.appliedFailoverPolicy);
} catch {
return false;
}
}
function publish(next: FailoverSnapshot) {
sequence += 1;
snapshot = { ...next, observationEpoch: epoch, observationSequence: sequence };
}
function clearTimer() {
if (timer) scheduler.clearTimeout(timer);
timer = null;
}
function schedule(delay: number) {
clearTimer();
timer = scheduler.setTimeout(() => {
timer = null;
void runRound().catch(dependencies.onWarning);
}, delay);
timer.unref?.();
}
async function disableCollector() {
if (collectorEnabled === false) return;
await dependencies.dataplane.setFailoverActivityEnabled(false);
collectorEnabled = false;
}
async function deactivate(policy: FailoverPolicy, passiveRole: FailoverRole | null = null) {
generation += 1;
clearTimer();
decisionMemory = undefined;
await disableCollector();
const idle = createIdleFailoverSnapshot(policy, epoch, sequence);
if (passiveRole) {
idle.activation = 'passive-loaded';
idle.currentRole = passiveRole;
idle.reason = 'disabled';
}
publish(idle);
}
function activeSnapshot(state: StoredState, status: FailoverSnapshot['status'] = 'observing'): FailoverSnapshot {
const role = state.failoverRuntimeState.reasonCode === 'selector-unknown' ? null : currentRole(state);
const channel = (target: typeof state.failoverPolicy.primary) => ({
target,
health: 'unknown' as FailoverHealth,
failingServiceIds: [],
checkedAt: null,
stateSince: null,
});
return {
observationEpoch: epoch,
observationSequence: sequence,
configured: isFailoverConfigured(state.failoverPolicy),
enabled: state.failoverPolicy.enabled,
paused: state.failoverPolicy.paused,
activation: appliedMatchesDesired(state) ? 'active' : 'pending',
currentRole: role || 'other',
status,
primary: channel(state.appliedFailoverPolicy?.primary || state.failoverPolicy.primary),
reserve: channel(state.appliedFailoverPolicy?.reserve || state.failoverPolicy.reserve),
nextDecisionAt: null,
reason: null,
trafficActivity: null,
policy: state.failoverPolicy,
};
}
async function reconcile() {
let state = dependencies.state.read();
const policy = state.failoverPolicy;
if (!policy.enabled) {
const role = currentRole(state);
const passiveRole = role && await dependencies.runtime.isRunning() ? role : null;
return deactivate(policy, passiveRole);
}
const running = await dependencies.runtime.isRunning();
if (!running || !state.appliedFailoverPolicy || !currentRole(state)) {
generation += 1;
clearTimer();
await disableCollector();
const pending = activeSnapshot(state, 'idle');
pending.activation = running ? 'pending' : 'inactive';
pending.reason = running ? 'pending-activation' : 'vpn-stopped';
publish(pending);
return;
}
const role = currentRole(state)!;
const selected = record(await dependencies.dataplane.readFailoverSelector());
if (selected.role !== role) await dependencies.dataplane.selectFailoverRole(role);
if (state.failoverRuntimeState.reasonCode === 'selector-unknown') {
state = dependencies.state.update((current) => ({
...current,
failoverRuntimeState: { ...current.failoverRuntimeState, reasonCode: null },
}));
}
if (collectorEnabled !== true) {
await dependencies.dataplane.setFailoverActivityEnabled(true);
collectorEnabled = true;
}
generation += 1;
const active = activeSnapshot(state);
if (active.activation === 'pending') active.reason = 'pending-activation';
publish(active);
schedule(0);
}
async function reconcileAfterCommit() {
try {
await reconcile();
} catch (error) {
dependencies.onWarning?.(error);
const failed = activeSnapshot(dependencies.state.read(), 'error');
failed.reason = 'reconcile-failed';
publish(failed);
}
}
async function assessRole(role: FailoverRole, policy: FailoverPolicy) {
const custom = dependencies.state.read().diagnostics.customServices;
const results = await Promise.all(policy.checks.map(async (check) => {
try {
return {
id: check.serviceId,
ok: probeHealth(await dependencies.dataplane.runFailoverProbe(
role,
custom,
`site:${check.serviceId}`,
check.timeoutMs,
)),
};
} catch {
return { id: check.serviceId, ok: null };
}
}));
return {
health: results.some(({ ok }) => ok === null)
? 'unknown' as const
: results.every(({ ok }) => ok) ? 'healthy' as const : 'unhealthy' as const,
failingServiceIds: results.filter(({ ok }) => ok === false).map(({ id }) => id),
};
}
async function switchWithinQueue(role: FailoverRole, reason: string) {
const before = dependencies.state.read();
const from = currentRole(before);
if (!from || from === role) return;
try {
await dependencies.dataplane.selectFailoverRole(role);
const server = targetServer(before, role);
if (!server) throw new HarborError('SERVER_NOT_FOUND');
const target = before.appliedFailoverPolicy![role];
const switchedAt = now().toISOString();
const cutoff = now().getTime() - before.failoverPolicy.flapProtection.windowMs;
const history = role === 'reserve'
? [...before.failoverRuntimeState.failoverHistory.filter((value) => Date.parse(value) >= cutoff), switchedAt]
: before.failoverRuntimeState.failoverHistory.filter((value) => Date.parse(value) >= cutoff);
const quarantine = history.length >= before.failoverPolicy.flapProtection.count
? new Date(now().getTime() + before.failoverPolicy.flapProtection.quarantineMs).toISOString()
: before.failoverRuntimeState.primaryQuarantineUntil;
dependencies.state.update((state) => ({
...state,
appliedProfileId: target.profileId,
appliedServerId: target.serverId,
appliedServerSnapshot: server,
failoverPolicy: reason === 'manual'
? { ...state.failoverPolicy, paused: true }
: role === 'primary' ? { ...state.failoverPolicy, paused: false } : state.failoverPolicy,
failoverRuntimeState: {
...state.failoverRuntimeState,
lastSwitchAt: switchedAt,
holdUntil: role === 'reserve'
? new Date(now().getTime() + state.failoverPolicy.minimumReserveMs).toISOString()
: null,
primaryQuarantineUntil: quarantine,
failoverHistory: history,
reasonCode: reason,
},
}));
} catch (error) {
try {
await dependencies.dataplane.selectFailoverRole(from);
} catch (rollback) {
generation += 1;
clearTimer();
decisionMemory = undefined;
dependencies.state.update((state) => ({
...state,
failoverPolicy: { ...state.failoverPolicy, paused: true },
failoverRuntimeState: { ...state.failoverRuntimeState, reasonCode: 'selector-unknown' },
}));
const failed = activeSnapshot(dependencies.state.read(), 'error');
failed.reason = 'selector-unknown';
publish(failed);
throw new AggregateError([error, rollback], 'Failover selector rollback failed');
}
throw error;
}
dependencies.onSwitch?.(from, role, reason);
dependencies.onEvent?.({
type: 'failover.switched',
severity: 'info',
source: 'failover',
dedupeKey: `failover.switched:${dependencies.state.read().revision}`,
data: {
fromRole: from,
toRole: role,
primaryLabel: targetServer(dependencies.state.read(), 'primary')?.label || 'Primary',
reserveLabel: targetServer(dependencies.state.read(), 'reserve')?.label || 'Reserve',
reason,
manual: reason === 'manual',
},
});
}
async function performRound(capturedGeneration: number) {
const prepared = await dependencies.serialize(async () => {
const state = dependencies.state.read();
const role = currentRole(state);
if (
capturedGeneration !== generation
|| !state.failoverPolicy.enabled
|| !state.appliedFailoverPolicy
|| !role
) return null;
const selected = record(await dependencies.dataplane.readFailoverSelector());
if (selected.role !== role) await dependencies.dataplane.selectFailoverRole(role);
await dependencies.dataplane.setFailoverActivityEnabled(true);
collectorEnabled = true;
const latest = dependencies.state.read();
return capturedGeneration === generation
&& latest.failoverPolicy.enabled
&& currentRole(latest) === role
? { state: latest, policy: latest.failoverPolicy, role }
: null;
});
if (!prepared) return;
const { state, policy, role } = prepared;
const checkedAt = now().toISOString();
let primary;
let reserve;
try {
[primary, reserve] = await Promise.all([
assessRole('primary', policy),
assessRole('reserve', policy),
]);
} catch {
primary = { health: 'unknown' as const, failingServiceIds: [] };
reserve = { health: 'unknown' as const, failingServiceIds: [] };
}
if (capturedGeneration !== generation || !dependencies.state.read().failoverPolicy.enabled) return;
const activityResponse = record(await dependencies.dataplane.readFailoverActivity(
policy.trafficGuard.thresholdBytesPerSecond,
));
if (capturedGeneration !== generation || !dependencies.state.read().failoverPolicy.enabled) return;
const activity = record(activityResponse.activity);
const observedAt = typeof activity.observedAt === 'string' ? Date.parse(activity.observedAt) : NaN;
const activityState = Number.isFinite(observedAt) && now().getTime() - observedAt <= 4_000
&& (activity.state === 'active' || activity.state === 'quiet')
? activity.state
: 'unknown';
const decision = nextFailoverDecision({
now: now().getTime(),
policy,
currentRole: role,
primaryHealth: primary.health,
reserveHealth: reserve.health,
activity: activityState,
holdUntil: Date.parse(state.failoverRuntimeState.holdUntil || '') || null,
primaryQuarantineUntil: Date.parse(state.failoverRuntimeState.primaryQuarantineUntil || '') || null,
memory: decisionMemory,
});
decisionMemory = decision.memory;
const previousSnapshot = snapshot;
const next = activeSnapshot(state, decision.status);
next.currentRole = role;
next.primary = {
...next.primary,
...primary,
checkedAt,
stateSince: previousSnapshot.primary.health === primary.health
? previousSnapshot.primary.stateSince || checkedAt
: checkedAt,
};
next.reserve = {
...next.reserve,
...reserve,
checkedAt,
stateSince: previousSnapshot.reserve.health === reserve.health
? previousSnapshot.reserve.stateSince || checkedAt
: checkedAt,
};
next.reason = decision.reason;
next.nextDecisionAt = decision.nextDecisionAt ? new Date(decision.nextDecisionAt).toISOString() : null;
next.trafficActivity = activityState === 'unknown' ? {
state: 'unknown',
observedAt: Number.isFinite(observedAt) ? new Date(observedAt).toISOString() : checkedAt,
windowMs: 10_000,
thresholdBytesPerSecond: policy.trafficGuard.thresholdBytesPerSecond,
totalBytesPerSecond: 0,
transmittingConnections: 0,
quietSince: null,
switchTarget: decision.switchTo,
blockers: [],
} : {
state: activityState,
observedAt: String(activity.observedAt),
windowMs: Number(activity.windowMs) || 10_000,
thresholdBytesPerSecond: policy.trafficGuard.thresholdBytesPerSecond,
totalBytesPerSecond: Number(activity.totalBytesPerSecond) || 0,
transmittingConnections: Number(activity.transmittingConnections) || 0,
quietSince: typeof activity.quietSince === 'string' ? activity.quietSince : null,
switchTarget: decision.switchTo,
blockers: (Array.isArray(activity.blockers) ? activity.blockers : []).slice(0, 3) as FailoverSnapshot['trafficActivity'] extends infer T ? T extends { blockers: infer B } ? B : never : never,
};
publish(next);
if (decision.status === 'waiting-for-idle' && previousSnapshot.status !== 'waiting-for-idle') {
dependencies.onEvent?.({
type: 'failover.waiting_for_idle',
severity: 'info',
source: 'failover',
dedupeKey: `failover.waiting_for_idle:${state.revision}:${role}:${decision.reason}`,
data: { fromRole: role, toRole: decision.switchTo || (role === 'primary' ? 'reserve' : 'primary'), reason: decision.reason },
});
}
if (decision.reason === 'both-unhealthy' && previousSnapshot.reason !== 'both-unhealthy') {
dependencies.onEvent?.({
type: 'failover.both_unhealthy',
severity: 'warning',
source: 'failover',
dedupeKey: `failover.both_unhealthy:${state.revision}`,
data: { reason: decision.reason },
});
}
if (primary.health === 'healthy' && previousSnapshot.primary.health === 'unhealthy') {
dependencies.onEvent?.({
type: 'failover.recovered',
severity: 'info',
source: 'failover',
dedupeKey: `failover.recovered:${state.revision}:primary`,
data: { role: 'primary', reason: decision.reason },
});
}
if (decision.switchTo) {
try {
const switched = await dependencies.serialize(async () => {
const current = dependencies.state.read();
if (
capturedGeneration !== generation
|| !current.failoverPolicy.enabled
|| current.failoverPolicy.paused
|| currentRole(current) !== role
|| !current.appliedFailoverPolicy
) return false;
let freshPrimary;
let freshReserve;
try {
[freshPrimary, freshReserve] = await Promise.all([
assessRole('primary', current.failoverPolicy),
assessRole('reserve', current.failoverPolicy),
]);
} catch {
return false;
}
const healthStillAllowsSwitch = decision.switchTo === 'reserve'
? freshPrimary.health === 'unhealthy' && freshReserve.health === 'healthy'
: freshPrimary.health === 'healthy';
if (!healthStillAllowsSwitch || capturedGeneration !== generation) return false;
if (current.failoverPolicy.trafficGuard.enabled) {
const freshResponse = record(await dependencies.dataplane.readFailoverActivity(
current.failoverPolicy.trafficGuard.thresholdBytesPerSecond,
));
const freshActivity = record(freshResponse.activity);
const freshObservedAt = typeof freshActivity.observedAt === 'string'
? Date.parse(freshActivity.observedAt)
: NaN;
const freshQuietSince = typeof freshActivity.quietSince === 'string'
? Date.parse(freshActivity.quietSince)
: NaN;
if (
capturedGeneration !== generation
|| freshActivity.state !== 'quiet'
|| !Number.isFinite(freshObservedAt)
|| now().getTime() - freshObservedAt > 4_000
|| !Number.isFinite(freshQuietSince)
|| now().getTime() - freshQuietSince < current.failoverPolicy.trafficGuard.quietWindowMs
) return false;
}
await switchWithinQueue(decision.switchTo!, decision.reason);
return true;
});
if (!switched) {
const cancelled = activeSnapshot(dependencies.state.read(), 'observing');
cancelled.reason = 'revalidation-required';
publish(cancelled);
decisionMemory = undefined;
return;
}
} catch (error) {
const failed = activeSnapshot(dependencies.state.read(), 'error');
failed.reason = dependencies.state.read().failoverRuntimeState.reasonCode === 'selector-unknown'
? 'selector-unknown'
: 'switch-failed';
publish(failed);
dependencies.onEvent?.({
type: 'failover.switch_failed',
severity: 'error',
source: 'failover',
dedupeKey: `failover.switch_failed:${state.revision}:${role}:${decision.switchTo}`,
data: { fromRole: role, toRole: decision.switchTo, reason: decision.reason, errorCode: safeErrorCode(error) },
});
throw error;
}
if (capturedGeneration !== generation) return;
publish(activeSnapshot(dependencies.state.read(), decision.switchTo === 'reserve' ? 'reserve' : 'primary'));
decisionMemory = undefined;
}
}
async function runRound() {
if (roundPromise) return roundPromise;
const capturedGeneration = generation;
roundPromise = performRound(capturedGeneration).finally(() => {
roundPromise = null;
const policy = dependencies.state.read().failoverPolicy;
if (capturedGeneration === generation && policy.enabled && dependencies.state.read().appliedFailoverPolicy) {
const decisionAt = snapshot.nextDecisionAt ? Date.parse(snapshot.nextDecisionAt) : NaN;
const decisionDelay = Number.isFinite(decisionAt)
? Math.max(250, decisionAt - now().getTime())
: policy.intervalMs;
schedule(Math.min(policy.intervalMs, decisionDelay));
}
});
return roundPromise;
}
function save(value: unknown) {
return dependencies.serialize(async () => {
let policy: FailoverPolicy;
try {
policy = normalizeFailoverPolicy(value, { strict: true });
} catch (cause) {
throw new HarborError('REQUEST_INVALID', { cause });
}
if (policy.enabled && !isFailoverConfigured(policy)) throw new HarborError('REQUEST_INVALID');
const before = dependencies.state.read();
const candidateState = { ...before, failoverPolicy: policy };
if (policy.enabled) {
const candidate = dependencies.buildCandidate(candidateState);
await dependencies.dataplane.checkConfig(candidate.config);
}
dependencies.state.update((state) => {
const role = before.failoverPolicy.enabled && !policy.enabled ? currentRole(state) : null;
const target = role ? state.appliedFailoverPolicy?.[role] : null;
return {
...state,
failoverPolicy: policy,
...(target ? {
desiredProfileId: target.profileId,
profiles: state.profiles.map((profile) => profile.id === target.profileId
? { ...profile, desiredServerId: target.serverId }
: profile),
} : {}),
};
});
decisionMemory = undefined;
if (before.failoverPolicy.enabled !== policy.enabled) {
dependencies.onEvent?.({
type: policy.enabled ? 'failover.enabled' : 'failover.disabled',
severity: 'info',
source: 'failover',
dedupeKey: `failover.${policy.enabled ? 'enabled' : 'disabled'}:${dependencies.state.read().revision}`,
data: {},
});
}
await reconcileAfterCommit();
});
}
function pause(paused: boolean) {
return dependencies.serialize(async () => {
const before = dependencies.state.read();
if (!paused && (
!before.appliedFailoverPolicy
|| !targetServer(before, 'primary')
|| !targetServer(before, 'reserve')
)) throw new HarborError('REQUEST_INVALID');
dependencies.state.update((state) => ({
...state,
failoverPolicy: { ...state.failoverPolicy, paused },
}));
decisionMemory = undefined;
dependencies.onEvent?.({
type: paused ? 'failover.paused' : 'failover.resumed',
severity: 'info',
source: 'failover',
dedupeKey: `failover.${paused ? 'paused' : 'resumed'}:${dependencies.state.read().revision}`,
data: {},
});
await reconcileAfterCommit();
});
}
async function manualSwitch(role: FailoverRole) {
await dependencies.serialize(async () => {
const state = dependencies.state.read();
if (!state.failoverPolicy.enabled || !state.appliedFailoverPolicy || !currentRole(state)) {
throw new HarborError('REQUEST_INVALID');
}
await switchWithinQueue(role, 'manual');
});
await reconcileAfterCommit();
}
const prepareActivation = (role: FailoverRole) => dependencies.dataplane.selectFailoverRole(role);
async function restoreAppliedActivation(state: StoredState) {
if (!state.appliedFailoverPolicy) return;
const role = currentRole(state);
if (!role) throw new HarborError('CONFIG_INVALID');
await prepareActivation(role);
}
function checkNow() {
return dependencies.serialize(async () => {
const state = dependencies.state.read();
if (!state.failoverPolicy.enabled || !state.appliedFailoverPolicy || !currentRole(state)) {
throw new HarborError('REQUEST_INVALID');
}
generation += 1;
clearTimer();
const checkedAt = now().toISOString();
let primary;
let reserve;
try {
[primary, reserve] = await Promise.all([
assessRole('primary', state.failoverPolicy),
assessRole('reserve', state.failoverPolicy),
]);
} catch {
primary = { health: 'unknown' as const, failingServiceIds: [] };
reserve = { health: 'unknown' as const, failingServiceIds: [] };
}
const next = activeSnapshot(dependencies.state.read(), 'observing');
next.primary = { ...next.primary, ...primary, checkedAt, stateSince: checkedAt };
next.reserve = { ...next.reserve, ...reserve, checkedAt, stateSince: checkedAt };
next.reason = 'manual-check';
publish(next);
schedule(state.failoverPolicy.intervalMs);
});
}
return {
snapshot: () => snapshot,
save,
pause,
manualSwitch,
checkNow,
prepareActivation,
restoreAppliedActivation,
reconcile,
runRound,
shutdown: () => deactivate(dependencies.state.read().failoverPolicy),
};
}
export type FailoverService = ReturnType<typeof createFailoverService>;
@@ -35,6 +35,8 @@ interface RouteRulesDependencies {
route?: { isGatewayDirect(): boolean }; route?: { isGatewayDirect(): boolean };
serialize<T>(operation: () => Promise<T>): Promise<T>; serialize<T>(operation: () => Promise<T>): Promise<T>;
runOperation<T>(operation: () => Promise<T>): Promise<T>; runOperation<T>(operation: () => Promise<T>): Promise<T>;
afterApply?: () => Promise<unknown>;
restoreAppliedActivation?: (state: StoredState) => Promise<unknown>;
} }
export function createRouteRulesService(dependencies: RouteRulesDependencies) { export function createRouteRulesService(dependencies: RouteRulesDependencies) {
@@ -96,6 +98,7 @@ export function createRouteRulesService(dependencies: RouteRulesDependencies) {
...(wasRunning ? { appliedRouteRules: routeRules } : {}), ...(wasRunning ? { appliedRouteRules: routeRules } : {}),
routeRulesRevision: state.routeRulesRevision + 1, routeRulesRevision: state.routeRulesRevision + 1,
})); }));
await dependencies.afterApply?.();
} catch (error) { } catch (error) {
await finishRollback(error, [ await finishRollback(error, [
...(configMutationStarted ? [{ ...(configMutationStarted ? [{
@@ -104,7 +107,10 @@ export function createRouteRulesService(dependencies: RouteRulesDependencies) {
: dependencies.config.restore(previousConfig), : dependencies.config.restore(previousConfig),
}] : []), }] : []),
...(wasRunning && runtimeMutationStarted ? [{ ...(wasRunning && runtimeMutationStarted ? [{
run: () => dependencies.runtime.restoreRunning(), run: async () => {
await dependencies.runtime.restoreRunning();
await dependencies.restoreAppliedActivation?.(previousState);
},
runtime: true, runtime: true,
}] : []), }] : []),
...(stateCommitStarted ? [{ run: () => dependencies.state.update(() => previousState) }] : []), ...(stateCommitStarted ? [{ run: () => dependencies.state.update(() => previousState) }] : []),
@@ -6,6 +6,7 @@ import {
type StateSnapshot, type StateSnapshot,
type StoredState, type StoredState,
} from '../../../shared/contracts/state.js'; } from '../../../shared/contracts/state.js';
import type { FailoverSnapshot } from '../../../shared/failover.js';
interface RuntimeState { interface RuntimeState {
running?: boolean; running?: boolean;
@@ -26,6 +27,7 @@ interface StateServiceDependencies {
getGatewayAutoState: () => GatewayAutoState; getGatewayAutoState: () => GatewayAutoState;
getOperationState: () => OperationState; getOperationState: () => OperationState;
configExists: () => boolean; configExists: () => boolean;
getFailoverSnapshot?: () => FailoverSnapshot;
} }
function subscriptionHost(value: unknown) { function subscriptionHost(value: unknown) {
@@ -51,6 +53,7 @@ export function createStateService(dependencies: StateServiceDependencies) {
configExists, configExists,
subscriptionHost: subscriptionHost(storedState.subscriptionUrl), subscriptionHost: subscriptionHost(storedState.subscriptionUrl),
operation: dependencies.getOperationState(), operation: dependencies.getOperationState(),
failoverSnapshot: dependencies.getFailoverSnapshot?.(),
}); });
return { snapshot, storedState, gatewayAuto, configExists }; return { snapshot, storedState, gatewayAuto, configExists };
}, },
@@ -8,6 +8,7 @@ import {
type StoredState, type StoredState,
} from '../../../shared/contracts/state.js'; } from '../../../shared/contracts/state.js';
import { HarborError } from '../../../shared/errors.js'; import { HarborError } from '../../../shared/errors.js';
import type { ActivityJournalEventInput } from '../../../shared/activityJournal.js';
import { finishRollback } from '../../services/rollback.js'; import { finishRollback } from '../../services/rollback.js';
interface ParsedSubscription { interface ParsedSubscription {
@@ -55,6 +56,11 @@ interface SubscriptionServiceDependencies {
clearInterval(handle: TimerHandle): void; clearInterval(handle: TimerHandle): void;
}; };
onRefreshError(error: unknown): void; onRefreshError(error: unknown): void;
onEvent?: (event: ActivityJournalEventInput) => void;
failover?: {
reconcile(): Promise<unknown>;
restoreAppliedActivation(state: StoredState): Promise<unknown>;
};
now?: () => Date; now?: () => Date;
} }
@@ -106,6 +112,10 @@ function mutationResult(profile: Pick<StoredProfile, 'id' | 'label'>): ProfileMu
return { success: true, profileId: profile.id, label: profile.label }; return { success: true, profileId: profile.id, label: profile.label };
} }
function publicHost(url: string) {
try { return new URL(url).hostname; } catch { return ''; }
}
export function createSubscriptionService(dependencies: SubscriptionServiceDependencies) { export function createSubscriptionService(dependencies: SubscriptionServiceDependencies) {
const refreshPromises = new Map<string, Promise<ProfileMutationResult>>(); const refreshPromises = new Map<string, Promise<ProfileMutationResult>>();
let refreshTimer: TimerHandle | null = null; let refreshTimer: TimerHandle | null = null;
@@ -166,6 +176,13 @@ export function createSubscriptionService(dependencies: SubscriptionServiceDepen
profiles: [...current.profiles, profile], profiles: [...current.profiles, profile],
desiredProfileId: current.profiles.length ? current.desiredProfileId : profile.id, desiredProfileId: current.profiles.length ? current.desiredProfileId : profile.id,
})); }));
dependencies.onEvent?.({
type: 'subscription.added',
severity: 'info',
source: 'subscription',
dedupeKey: `subscription.added:${dependencies.state.read().revision}`,
data: { profileId: profile.id, profileLabel: profile.label, host: publicHost(profile.subscriptionUrl), serverCount: profile.servers.length },
});
return mutationResult(profile); return mutationResult(profile);
}); });
}; };
@@ -217,6 +234,7 @@ export function createSubscriptionService(dependencies: SubscriptionServiceDepen
profileId: string, profileId: string,
subscriptionUrl: string, subscriptionUrl: string,
error: unknown, error: unknown,
origin: 'manual' | 'scheduled',
) => dependencies.serialize(async () => { ) => dependencies.serialize(async () => {
const state = dependencies.state.read(); const state = dependencies.state.read();
const profile = requireProfile(state, profileId); const profile = requireProfile(state, profileId);
@@ -230,12 +248,27 @@ export function createSubscriptionService(dependencies: SubscriptionServiceDepen
...current, ...current,
profiles: replaceProfile(current, failed), profiles: replaceProfile(current, failed),
})); }));
dependencies.onEvent?.({
type: 'subscription.refresh_failed',
severity: 'warning',
source: 'subscription',
dedupeKey: origin === 'scheduled'
? `subscription.refresh_failed:${failed.id}:${failed.fetchedAt || 'never'}:${failed.lastRefreshErrorCode}`
: `subscription.refresh_failed:${dependencies.state.read().revision}`,
data: {
profileId: failed.id,
profileLabel: failed.label,
host: publicHost(failed.subscriptionUrl),
errorCode: failed.lastRefreshErrorCode || 'UNKNOWN',
},
});
}); });
const commitRefresh = ( const commitRefresh = (
profileId: string, profileId: string,
subscriptionUrl: string, subscriptionUrl: string,
parsed: ParsedSubscription, parsed: ParsedSubscription,
origin: 'manual' | 'scheduled',
) => dependencies.serialize(async () => { ) => dependencies.serialize(async () => {
// Re-read the profile after provider I/O and guard its owner instead of rejecting background-only revisions. // Re-read the profile after provider I/O and guard its owner instead of rejecting background-only revisions.
const previousState = dependencies.state.read(); const previousState = dependencies.state.read();
@@ -257,8 +290,46 @@ export function createSubscriptionService(dependencies: SubscriptionServiceDepen
lastRefreshAttemptAt: parsed.fetchedAt, lastRefreshAttemptAt: parsed.fetchedAt,
lastRefreshErrorCode: null, lastRefreshErrorCode: null,
}; };
const contentChanged = JSON.stringify({
subscriptionConfig: previousProfile.subscriptionConfig,
servers: previousProfile.servers,
userInfo: previousProfile.userInfo,
}) !== JSON.stringify({
subscriptionConfig: refreshedProfile.subscriptionConfig,
servers: refreshedProfile.servers,
userInfo: refreshedProfile.userInfo,
});
const appendRefreshEvent = () => {
if (origin === 'scheduled' && !contentChanged && !previousProfile.lastRefreshErrorCode) return;
dependencies.onEvent?.({
type: 'subscription.refreshed',
severity: 'info',
source: 'subscription',
dedupeKey: `subscription.refreshed:${dependencies.state.read().revision}`,
data: {
profileId: refreshedProfile.id,
profileLabel: refreshedProfile.label,
host: publicHost(refreshedProfile.subscriptionUrl),
serverCount: refreshedProfile.servers.length,
added: refreshedProfile.servers.filter(({ id }) => !previousProfile.servers.some((server) => server.id === id)).length,
removed: previousProfile.servers.filter(({ id }) => !refreshedProfile.servers.some((server) => server.id === id)).length,
},
});
};
const loadedTargets = previousState.appliedFailoverPolicy
? [previousState.appliedFailoverPolicy.primary, previousState.appliedFailoverPolicy.reserve]
: [];
const missingLoadedTarget = loadedTargets.some((target) => (
target.profileId === profileId
&& !refreshedProfile.servers.some(({ id }) => id === target.serverId)
));
const pauseFailover = previousState.failoverPolicy?.enabled
&& !previousState.failoverPolicy.paused
&& missingLoadedTarget;
const running = await dependencies.runtime.isRunning(); const running = await dependencies.runtime.isRunning();
const refreshesApplied = running && previousState.appliedProfileId === profileId; const refreshesApplied = running
&& previousState.appliedProfileId === profileId
&& !previousState.appliedFailoverPolicy;
const nextAppliedServerId = refreshesApplied const nextAppliedServerId = refreshesApplied
? dependencies.provider.selectRefreshedServer( ? dependencies.provider.selectRefreshedServer(
previousState.appliedServerId, previousState.appliedServerId,
@@ -271,7 +342,17 @@ export function createSubscriptionService(dependencies: SubscriptionServiceDepen
dependencies.state.update((current) => ({ dependencies.state.update((current) => ({
...current, ...current,
profiles: replaceProfile(current, refreshedProfile), profiles: replaceProfile(current, refreshedProfile),
...(pauseFailover ? { failoverPolicy: { ...current.failoverPolicy, paused: true } } : {}),
})); }));
if (pauseFailover) dependencies.onEvent?.({
type: 'failover.paused',
severity: 'warning',
source: 'failover',
dedupeKey: `failover.paused:missing-target:${profileId}:${dependencies.state.read().revision}`,
data: {},
});
await dependencies.failover?.reconcile();
appendRefreshEvent();
return mutationResult(refreshedProfile); return mutationResult(refreshedProfile);
} }
@@ -305,13 +386,25 @@ export function createSubscriptionService(dependencies: SubscriptionServiceDepen
await finishRollback(error, [ await finishRollback(error, [
...(stateCommitStarted ? [{ run: () => dependencies.state.update(() => previousState) }] : []), ...(stateCommitStarted ? [{ run: () => dependencies.state.update(() => previousState) }] : []),
...(configMutationStarted ? [{ run: () => restoreConfig(previousConfig) }] : []), ...(configMutationStarted ? [{ run: () => restoreConfig(previousConfig) }] : []),
...(runtimeMutationStarted ? [{ run: () => dependencies.runtime.start(), runtime: true }] : []), ...(runtimeMutationStarted ? [{
run: async () => {
await dependencies.runtime.start();
await dependencies.failover?.restoreAppliedActivation(previousState);
},
runtime: true,
}] : []),
], 'Subscription refresh rollback failed'); ], 'Subscription refresh rollback failed');
} }
await dependencies.failover?.reconcile();
appendRefreshEvent();
return mutationResult(refreshedProfile); return mutationResult(refreshedProfile);
}); });
const refreshProfile = (profileIdValue: unknown, expectedRevision?: unknown) => { const refreshProfile = (
profileIdValue: unknown,
expectedRevision?: unknown,
origin: 'manual' | 'scheduled' = 'manual',
) => {
const profileId = String(profileIdValue || '').trim(); const profileId = String(profileIdValue || '').trim();
const existing = refreshPromises.get(profileId); const existing = refreshPromises.get(profileId);
if (existing) return existing; if (existing) return existing;
@@ -328,6 +421,7 @@ export function createSubscriptionService(dependencies: SubscriptionServiceDepen
profileId, profileId,
initialProfile.subscriptionUrl, initialProfile.subscriptionUrl,
error, error,
origin,
); );
} }
throw error; throw error;
@@ -336,6 +430,7 @@ export function createSubscriptionService(dependencies: SubscriptionServiceDepen
profileId, profileId,
initialProfile.subscriptionUrl, initialProfile.subscriptionUrl,
parsed, parsed,
origin,
); );
})().finally(() => refreshPromises.delete(profileId)); })().finally(() => refreshPromises.delete(profileId));
refreshPromises.set(profileId, operation); refreshPromises.set(profileId, operation);
@@ -353,12 +448,18 @@ export function createSubscriptionService(dependencies: SubscriptionServiceDepen
const mode = String(modeValue || 'delete'); const mode = String(modeValue || 'delete');
if (!['delete', 'stop-and-delete'].includes(mode)) throw new HarborError('REQUEST_INVALID'); if (!['delete', 'stop-and-delete'].includes(mode)) throw new HarborError('REQUEST_INVALID');
const running = await dependencies.runtime.isRunning(); const running = await dependencies.runtime.isRunning();
const applied = running && previousState.appliedProfileId === profile.id; const failoverReferencesProfile = Boolean(previousState.appliedFailoverPolicy && (
previousState.appliedFailoverPolicy.primary.profileId === profile.id
|| previousState.appliedFailoverPolicy.reserve.profileId === profile.id
));
const desiredFailoverReferencesProfile = previousState.failoverPolicy?.primary.profileId === profile.id
|| previousState.failoverPolicy?.reserve.profileId === profile.id;
const applied = running && (previousState.appliedProfileId === profile.id || failoverReferencesProfile);
if (applied && mode !== 'stop-and-delete') throw new HarborError('PROFILE_IN_USE'); if (applied && mode !== 'stop-and-delete') throw new HarborError('PROFILE_IN_USE');
const previousConfig = dependencies.config.read(); const previousConfig = dependencies.config.read();
const previousGatewayAuto = dependencies.gatewayAuto.read(); const previousGatewayAuto = dependencies.gatewayAuto.read();
const removesAppliedTarget = previousState.appliedProfileId === profile.id; const removesAppliedTarget = previousState.appliedProfileId === profile.id || failoverReferencesProfile;
let runtimeMutationStarted = false; let runtimeMutationStarted = false;
let configMutationStarted = false; let configMutationStarted = false;
let gatewayMutationStarted = false; let gatewayMutationStarted = false;
@@ -390,16 +491,49 @@ export function createSubscriptionService(dependencies: SubscriptionServiceDepen
appliedServerSnapshot: current.appliedProfileId === profile.id appliedServerSnapshot: current.appliedProfileId === profile.id
? null ? null
: current.appliedServerSnapshot, : current.appliedServerSnapshot,
...(current.appliedProfileId === profile.id ? { connectionDesired: 'stopped' } : {}), ...(removesAppliedTarget ? {
connectionDesired: 'stopped',
appliedProfileId: '',
appliedServerId: '',
appliedServerSnapshot: null,
appliedFailoverPolicy: null,
} : {}),
...(failoverReferencesProfile || desiredFailoverReferencesProfile ? {
failoverPolicy: {
...current.failoverPolicy,
enabled: false,
paused: false,
primary: current.failoverPolicy.primary.profileId === profile.id
? { profileId: '', serverId: '' }
: current.failoverPolicy.primary,
reserve: current.failoverPolicy.reserve.profileId === profile.id
? { profileId: '', serverId: '' }
: current.failoverPolicy.reserve,
},
} : {}),
})); }));
} catch (error) { } catch (error) {
await finishRollback(error, [ await finishRollback(error, [
...(stateCommitStarted ? [{ run: () => dependencies.state.update(() => previousState) }] : []), ...(stateCommitStarted ? [{ run: () => dependencies.state.update(() => previousState) }] : []),
...(gatewayMutationStarted ? [{ run: () => dependencies.gatewayAuto.set(previousGatewayAuto) }] : []), ...(gatewayMutationStarted ? [{ run: () => dependencies.gatewayAuto.set(previousGatewayAuto) }] : []),
...(configMutationStarted ? [{ run: () => restoreConfig(previousConfig) }] : []), ...(configMutationStarted ? [{ run: () => restoreConfig(previousConfig) }] : []),
...(runtimeMutationStarted ? [{ run: () => dependencies.runtime.start(), runtime: true }] : []), ...(runtimeMutationStarted ? [{
run: async () => {
await dependencies.runtime.start();
await dependencies.failover?.restoreAppliedActivation(previousState);
},
runtime: true,
}] : []),
], 'Subscription delete rollback failed'); ], 'Subscription delete rollback failed');
} }
await dependencies.failover?.reconcile();
dependencies.onEvent?.({
type: 'subscription.deleted',
severity: 'info',
source: 'subscription',
dedupeKey: `subscription.deleted:${dependencies.state.read().revision}`,
data: { profileId: profile.id, profileLabel: profile.label },
});
return mutationResult(profile); return mutationResult(profile);
}); });
@@ -436,7 +570,7 @@ export function createSubscriptionService(dependencies: SubscriptionServiceDepen
void (async () => { void (async () => {
for (const { id } of dependencies.state.read().profiles) { for (const { id } of dependencies.state.read().profiles) {
try { try {
await refreshProfile(id); await refreshProfile(id, undefined, 'scheduled');
} catch (error) { } catch (error) {
dependencies.onRefreshError(error); dependencies.onRefreshError(error);
} }
@@ -0,0 +1,17 @@
import type { IncomingMessage, ServerResponse } from 'node:http';
import type { ActivityJournalService } from '../../services/activityJournalService.js';
import { sendJson } from '../response.js';
export function createActivityJournalRoute({ journal }: { journal: ActivityJournalService }) {
return {
async handle(req: IncomingMessage, res: ServerResponse) {
const url = new URL(req.url || '/', 'http://localhost');
if (url.pathname !== '/api/activity-journal' || req.method !== 'GET') return false;
sendJson(res, 200, journal.page(
Number(url.searchParams.get('limit')) || 50,
url.searchParams.get('cursor'),
));
return true;
},
};
}
+49
View File
@@ -0,0 +1,49 @@
import type { IncomingMessage, ServerResponse } from 'node:http';
import type { FailoverService } from '../../features/failover/failoverService.js';
import { HarborError } from '../../../shared/errors.js';
interface FailoverRouteDependencies {
appMode: string;
failover: Pick<FailoverService, 'save' | 'pause' | 'manualSwitch' | 'checkNow'>;
readBody(req: IncomingMessage): Promise<Record<string, unknown>>;
withOperation<T>(kind: string, operation: () => Promise<T>, options?: { expectedRevision?: unknown }): Promise<T>;
sendState(res: ServerResponse): Promise<void>;
}
export function createFailoverRoute(dependencies: FailoverRouteDependencies) {
return {
async handle(req: IncomingMessage, res: ServerResponse) {
const pathname = new URL(req.url || '/', 'http://localhost').pathname;
if (!pathname.startsWith('/api/failover')) return false;
if (dependencies.appMode !== 'gateway') throw new HarborError('ENDPOINT_NOT_FOUND');
const body = await dependencies.readBody(req);
if (pathname === '/api/failover' && req.method === 'PUT') {
await dependencies.withOperation(
'failover-save',
() => dependencies.failover.save(body.policy),
{ expectedRevision: body.expectedRevision },
);
} else if (pathname === '/api/failover/pause' && req.method === 'POST') {
if (typeof body.paused !== 'boolean') throw new HarborError('REQUEST_INVALID');
await dependencies.withOperation(
body.paused ? 'failover-pause' : 'failover-resume',
() => dependencies.failover.pause(body.paused as boolean),
{ expectedRevision: body.expectedRevision },
);
} else if (pathname === '/api/failover/switch' && req.method === 'POST') {
if (body.role !== 'primary' && body.role !== 'reserve') throw new HarborError('REQUEST_INVALID');
await dependencies.withOperation(
'failover-switch',
() => dependencies.failover.manualSwitch(body.role as 'primary' | 'reserve'),
{ expectedRevision: body.expectedRevision },
);
} else if (pathname === '/api/failover/check' && req.method === 'POST') {
await dependencies.failover.checkNow();
} else {
throw new HarborError('ENDPOINT_NOT_FOUND');
}
await dependencies.sendState(res);
return true;
},
};
}
+204 -10
View File
@@ -16,7 +16,10 @@ import {
import { createSingboxRuntime } from './singboxRuntime.js'; import { createSingboxRuntime } from './singboxRuntime.js';
import { tcpPing } from './ping.js'; import { tcpPing } from './ping.js';
import { import {
buildDualChannelGatewayConfig,
buildGatewayConfig, buildGatewayConfig,
dualChannelConfigMatchesApplied,
fingerprintSelectedOutbound,
removeSingboxConfig, removeSingboxConfig,
restoreSingboxConfig, restoreSingboxConfig,
writeSingboxConfig, writeSingboxConfig,
@@ -81,6 +84,13 @@ import { createConnectivityDiagnosticsRoute } from './http/routes/connectivityDi
import { createGatewayPresenceRoute } from './http/routes/gatewayPresenceRoute.js'; import { createGatewayPresenceRoute } from './http/routes/gatewayPresenceRoute.js';
import { createSharedProxyRoute } from './http/routes/sharedProxyRoute.js'; import { createSharedProxyRoute } from './http/routes/sharedProxyRoute.js';
import { createVersionRoute } from './http/routes/versionRoute.js'; import { createVersionRoute } from './http/routes/versionRoute.js';
import { createSingboxSelectorService } from './services/singboxSelectorService.js';
import { createFailoverService } from './features/failover/failoverService.js';
import { createFailoverRoute } from './http/routes/failoverRoute.js';
import { createActivityJournalService } from './services/activityJournalService.js';
import { createActivityJournalRoute } from './http/routes/activityJournalRoute.js';
import { createDomainTrafficService, readSingboxConnections } from './services/domainTrafficService.js';
import type { ActivityJournalEventInput } from '../shared/activityJournal.js';
const MAX_BODY_BYTES = 1_000_000; const MAX_BODY_BYTES = 1_000_000;
const SUBSCRIPTION_REFRESH_INTERVAL_MS = 15 * 60 * 1000; const SUBSCRIPTION_REFRESH_INTERVAL_MS = 15 * 60 * 1000;
@@ -142,6 +152,14 @@ if (legacyCacheOwnerMismatch) {
} }
} }
const stateStore = createStateStore(settings.statePath, { legacySubscriptionCache }); const stateStore = createStateStore(settings.statePath, { legacySubscriptionCache });
const activityJournal = createActivityJournalService({ filePath: settings.activityJournalPath });
const appendJournal = (event: ActivityJournalEventInput) => {
try {
activityJournal.append(event);
} catch (error) {
console.warn(`[journal] событие не сохранено: ${errorMessage(error)}`);
}
};
const deviceStore = createJsonStore<InventoryState>({ const deviceStore = createJsonStore<InventoryState>({
filePath: settings.deviceStatePath, filePath: settings.deviceStatePath,
defaultValue: migrateDeviceInventoryState({}), defaultValue: migrateDeviceInventoryState({}),
@@ -283,6 +301,37 @@ const deviceInventory = settings.appMode === 'gateway'
const localConnectivityDiagnostics = !remoteDataplane const localConnectivityDiagnostics = !remoteDataplane
? createConnectivityDiagnosticsService({ proxyPort: settings.diagnosticsProxyPort }) ? createConnectivityDiagnosticsService({ proxyPort: settings.diagnosticsProxyPort })
: null; : null;
const localFailoverDiagnostics = !remoteDataplane ? {
primary: createConnectivityDiagnosticsService({ proxyPort: settings.failoverPrimaryProxyPort }),
reserve: createConnectivityDiagnosticsService({ proxyPort: settings.failoverReserveProxyPort }),
} : null;
const localSelector = !remoteDataplane && settings.appMode === 'gateway'
? createSingboxSelectorService({ port: settings.singboxApiPort })
: null;
const localFailoverTraffic = !remoteDataplane && settings.appMode === 'gateway'
? createDomainTrafficService({
observe: () => readSingboxConnections(settings.singboxApiPort),
devices: () => record(deviceInventory?.snapshot()).devices,
})
: null;
let localFailoverTrafficTimer: NodeJS.Timeout | null = null;
function setLocalFailoverActivityEnabled(enabled: boolean) {
if (!localFailoverTraffic) throw new Error('Failover activity недоступна');
if (!enabled) {
if (localFailoverTrafficTimer) clearInterval(localFailoverTrafficTimer);
localFailoverTrafficTimer = null;
localFailoverTraffic.disableActivity();
return;
}
localFailoverTraffic.enableActivity();
if (localFailoverTrafficTimer) return;
const refresh = () => localFailoverTraffic.refresh()
.catch((error: unknown) => console.warn(`[control] failover activity: ${errorMessage(error)}`));
void refresh();
localFailoverTrafficTimer = setInterval(refresh, 2_000);
localFailoverTrafficTimer.unref();
}
function requireLocalConnectivityDiagnostics() { function requireLocalConnectivityDiagnostics() {
if (!localConnectivityDiagnostics) throw new Error('Harbor local diagnostics are not configured'); if (!localConnectivityDiagnostics) throw new Error('Harbor local diagnostics are not configured');
@@ -355,6 +404,48 @@ const gatewayAutoService = createGatewayAutoService({
onDiscoveryWarning: (reason) => console.warn(`[control] Gateway не используется: ${reason}`), onDiscoveryWarning: (reason) => console.warn(`[control] Gateway не используется: ${reason}`),
onTimerError: (error) => console.warn(`[control] Gateway detection failed: ${errorMessage(error)}`), onTimerError: (error) => console.warn(`[control] Gateway detection failed: ${errorMessage(error)}`),
}); });
const failoverDataplane = remoteDataplane ? {
checkConfig: (config: unknown) => requireRemoteRuntime().checkConfig(config),
runFailoverProbe: (role: 'primary' | 'reserve', services: unknown, target: unknown, timeoutMs: number) => (
requireRemoteRuntime().runFailoverProbe(role, services, target, timeoutMs)
),
readFailoverSelector: () => requireRemoteRuntime().readFailoverSelector(),
selectFailoverRole: (role: 'primary' | 'reserve') => requireRemoteRuntime().selectFailoverRole(role),
setFailoverActivityEnabled: (enabled: boolean) => requireRemoteRuntime().setFailoverActivityEnabled(enabled),
readFailoverActivity: (threshold: number) => requireRemoteRuntime().readFailoverActivity(threshold),
} : {
checkConfig: async (config: unknown) => singboxRuntime.checkConfig(config),
runFailoverProbe: async (role: 'primary' | 'reserve', services: unknown, target: unknown, timeoutMs: number) => {
if (!localFailoverDiagnostics) throw new Error('Failover diagnostics недоступна');
return localFailoverDiagnostics[role].runVpn({ services, target, timeoutMs });
},
readFailoverSelector: async () => {
if (!localSelector) throw new Error('Failover selector недоступен');
return localSelector.read();
},
selectFailoverRole: async (role: 'primary' | 'reserve') => {
if (!localSelector) throw new Error('Failover selector недоступен');
return localSelector.select(role);
},
setFailoverActivityEnabled: async (enabled: boolean) => setLocalFailoverActivityEnabled(enabled),
readFailoverActivity: async (threshold: number) => ({
activity: localFailoverTraffic?.activitySnapshot(threshold) || null,
}),
};
const failoverService = createFailoverService({
state: {
read: () => normalizeStoredState(stateStore.read()),
update: updateStoredState,
},
runtime: { isRunning: async () => Boolean((await singboxRuntime.refresh()).running) },
dataplane: failoverDataplane,
buildCandidate: buildFailoverCandidate,
serialize: serializeControl,
onWarning: (error) => console.warn(`[control] failover: ${errorMessage(error)}`),
onSwitch: (from, to, reason) => console.log(`[control] failover ${from} -> ${to}: ${reason}`),
onEvent: appendJournal,
});
const gatewayFailover = settings.appMode === 'gateway' ? failoverService : null;
const stateService = createStateService({ const stateService = createStateService({
appMode: settings.appMode, appMode: settings.appMode,
readStoredState: () => stateStore.read(), readStoredState: () => stateStore.read(),
@@ -362,6 +453,7 @@ const stateService = createStateService({
getGatewayAutoState: gatewayAutoService.read, getGatewayAutoState: gatewayAutoService.read,
getOperationState: () => operationState, getOperationState: () => operationState,
configExists: () => fs.existsSync(settings.configPath), configExists: () => fs.existsSync(settings.configPath),
getFailoverSnapshot: failoverService.snapshot,
}); });
const stateRoute = createStateRoute({ const stateRoute = createStateRoute({
stateService, stateService,
@@ -375,6 +467,14 @@ const gatewayAutoRoute = createGatewayAutoRoute({
withOperation, withOperation,
readStatePayload: stateRoute.readPayload, readStatePayload: stateRoute.readPayload,
}); });
const failoverRoute = createFailoverRoute({
appMode: settings.appMode,
failover: failoverService,
readBody,
withOperation,
sendState: (res) => stateRoute.send(res),
});
const activityJournalRoute = createActivityJournalRoute({ journal: activityJournal });
const deviceInventoryRoute = createDeviceInventoryRoute({ const deviceInventoryRoute = createDeviceInventoryRoute({
deviceInventory, deviceInventory,
readBody, readBody,
@@ -427,9 +527,12 @@ const subscriptionService = createSubscriptionService({
update: updateStoredState, update: updateStoredState,
}, },
config: { config: {
build: (subscriptionConfig, selectedServerId, routeRules) => ( build: (subscriptionConfig, selectedServerId, routeRules) => {
buildActiveConfig(subscriptionConfig, selectedServerId, routeRules) const state = normalizeStoredState(stateStore.read());
), return state.appliedFailoverPolicy
? buildFailoverCandidate({ ...state, routeRules }, 'applied').config
: buildActiveConfig(subscriptionConfig, selectedServerId, routeRules);
},
read: () => fs.existsSync(settings.configPath) read: () => fs.existsSync(settings.configPath)
? fs.readFileSync(settings.configPath, 'utf8') ? fs.readFileSync(settings.configPath, 'utf8')
: null, : null,
@@ -453,6 +556,12 @@ const subscriptionService = createSubscriptionService({
clearInterval: (timer) => clearInterval(timer), clearInterval: (timer) => clearInterval(timer),
}, },
onRefreshError: (error) => console.warn(`[control] подписка не обновлена: ${errorMessage(error)}`), onRefreshError: (error) => console.warn(`[control] подписка не обновлена: ${errorMessage(error)}`),
onEvent: appendJournal,
failover: gatewayFailover ? {
reconcile: () => gatewayFailover.reconcile()
.catch((error) => console.warn(`[control] failover reconcile: ${errorMessage(error)}`)),
restoreAppliedActivation: gatewayFailover.restoreAppliedActivation,
} : undefined,
now: () => new Date(), now: () => new Date(),
}); });
const serverHealthRoute = createServerHealthRoute({ const serverHealthRoute = createServerHealthRoute({
@@ -484,6 +593,14 @@ const connectionService = createConnectionService({
isGatewayDirect: () => settings.appMode === 'client' isGatewayDirect: () => settings.appMode === 'client'
&& gatewayAutoService.read().mode === 'gateway-direct', && gatewayAutoService.read().mode === 'gateway-direct',
}, },
failover: gatewayFailover ? {
build: buildFailoverCandidate,
prepareActivation: gatewayFailover.prepareActivation,
restoreAppliedActivation: gatewayFailover.restoreAppliedActivation,
reconcile: () => gatewayFailover.reconcile()
.catch((error) => console.warn(`[control] failover reconcile: ${errorMessage(error)}`)),
} : undefined,
onEvent: appendJournal,
runtime: { runtime: {
isRunning: async () => Boolean((await singboxRuntime.refresh()).running), isRunning: async () => Boolean((await singboxRuntime.refresh()).running),
start: () => startSingbox(), start: () => startSingbox(),
@@ -524,9 +641,12 @@ const routeRulesService = createRouteRulesService({
readConfig: (profileId) => readProfileConfig(profileId), readConfig: (profileId) => readProfileConfig(profileId),
}, },
config: { config: {
build: (subscriptionConfig, selectedServerId, routeRules) => ( build: (subscriptionConfig, selectedServerId, routeRules) => {
buildActiveConfig(subscriptionConfig, selectedServerId, routeRules) const state = normalizeStoredState(stateStore.read());
), return state.appliedFailoverPolicy
? buildFailoverCandidate({ ...state, routeRules }, 'applied').config
: buildActiveConfig(subscriptionConfig, selectedServerId, routeRules);
},
read: () => fs.existsSync(settings.configPath) read: () => fs.existsSync(settings.configPath)
? fs.readFileSync(settings.configPath, 'utf8') ? fs.readFileSync(settings.configPath, 'utf8')
: null, : null,
@@ -548,6 +668,9 @@ const routeRulesService = createRouteRulesService({
}, },
serialize: serializeControl, serialize: serializeControl,
runOperation: (operation) => withOperation('route-rules', operation), runOperation: (operation) => withOperation('route-rules', operation),
afterApply: gatewayFailover ? () => gatewayFailover.reconcile()
.catch((error) => console.warn(`[control] failover reconcile: ${errorMessage(error)}`)) : undefined,
restoreAppliedActivation: gatewayFailover?.restoreAppliedActivation,
}); });
const routeRulesRoute = createRouteRulesRoute({ const routeRulesRoute = createRouteRulesRoute({
routeRules: routeRulesService, routeRules: routeRulesService,
@@ -676,6 +799,47 @@ function buildActiveConfig(
}); });
} }
function buildFailoverCandidate(state: StoredState, source: 'desired' | 'applied' = 'desired') {
const policy = source === 'applied' ? state.appliedFailoverPolicy : state.failoverPolicy;
if (!policy) throw new HarborError('CONFIG_INVALID');
if (
policy.primary.profileId === policy.reserve.profileId
&& policy.primary.serverId === policy.reserve.serverId
) throw new HarborError('REQUEST_INVALID');
const channel = (role: 'primary' | 'reserve') => {
const target = policy[role];
const profile = state.profiles.find(({ id }) => id === target.profileId);
const server = profile?.servers.find(({ id }) => id === target.serverId);
if (!profile || !server || !profile.subscriptionConfig) throw new HarborError('SERVER_NOT_FOUND');
return { profile, server };
};
const primary = channel('primary');
const reserve = channel('reserve');
const applied = {
primary: policy.primary,
reserve: policy.reserve,
primaryConfigFingerprint: fingerprintSelectedOutbound(primary.profile.subscriptionConfig, primary.server.id),
reserveConfigFingerprint: fingerprintSelectedOutbound(reserve.profile.subscriptionConfig, reserve.server.id),
};
if (source === 'applied' && JSON.stringify(applied) !== JSON.stringify(state.appliedFailoverPolicy)) {
throw new HarborError('CONFIG_INVALID');
}
const defaultRole = source === 'applied'
&& state.appliedProfileId === policy.reserve.profileId
&& state.appliedServerId === policy.reserve.serverId
? 'reserve'
: 'primary';
return {
config: buildDualChannelGatewayConfig({
primary: { subscriptionConfig: primary.profile.subscriptionConfig, selectedServerId: primary.server.id },
reserve: { subscriptionConfig: reserve.profile.subscriptionConfig, selectedServerId: reserve.server.id },
}, { routeRules: state.routeRules, defaultRole }),
applied,
primaryProfile: primary.profile,
primaryServer: primary.server,
};
}
const stopSingbox = () => singboxRuntime.stop(); const stopSingbox = () => singboxRuntime.stop();
const startSingbox = () => singboxRuntime.apply(); const startSingbox = () => singboxRuntime.apply();
@@ -689,7 +853,13 @@ function writeCurrentConfig() {
const server = profile?.servers.find((candidate) => candidate.id === serverId); const server = profile?.servers.find((candidate) => candidate.id === serverId);
const subscriptionConfig = profile ? readProfileConfig(profile.id) : null; const subscriptionConfig = profile ? readProfileConfig(profile.id) : null;
if (!profile || !server || !subscriptionConfig) return null; if (!profile || !server || !subscriptionConfig) return null;
const activeConfig = buildActiveConfig(subscriptionConfig, server.id); let activeConfig: unknown;
if (state.appliedFailoverPolicy) {
const candidate = buildFailoverCandidate(state, 'applied');
activeConfig = candidate.config;
} else {
activeConfig = buildActiveConfig(subscriptionConfig, server.id);
}
const previousConfig = fs.existsSync(settings.configPath) const previousConfig = fs.existsSync(settings.configPath)
? fs.readFileSync(settings.configPath, 'utf8') ? fs.readFileSync(settings.configPath, 'utf8')
: null; : null;
@@ -704,7 +874,7 @@ function writeCurrentConfig() {
} }
throw error; throw error;
} }
return { profile, server }; return { profile, server, failoverApplied: state.appliedFailoverPolicy };
} }
const CONFIG_PROXY_TYPES = new Set(['vless', 'vmess', 'trojan', 'shadowsocks', 'hysteria2']); const CONFIG_PROXY_TYPES = new Set(['vless', 'vmess', 'trojan', 'shadowsocks', 'hysteria2']);
@@ -717,6 +887,13 @@ function currentConfigMatchesAppliedTarget(state: StoredState) {
} catch { } catch {
return false; return false;
} }
if (state.appliedFailoverPolicy) {
const expectedRole = state.appliedProfileId === state.appliedFailoverPolicy.reserve.profileId
&& state.appliedServerId === state.appliedFailoverPolicy.reserve.serverId
? 'reserve'
: 'primary';
return dualChannelConfigMatchesApplied(config, state.appliedFailoverPolicy, expectedRole);
}
const proxyOutbounds = (Array.isArray(config.outbounds) ? config.outbounds : []) const proxyOutbounds = (Array.isArray(config.outbounds) ? config.outbounds : [])
.map(record) .map(record)
.filter((outbound) => CONFIG_PROXY_TYPES.has(String(outbound.type || ''))); .filter((outbound) => CONFIG_PROXY_TYPES.has(String(outbound.type || '')));
@@ -750,6 +927,7 @@ async function reconcileStoppedBoot({ removeConfig = false } = {}) {
|| state.appliedProfileId || state.appliedProfileId
|| state.appliedServerId || state.appliedServerId
|| state.appliedServerSnapshot || state.appliedServerSnapshot
|| state.appliedFailoverPolicy
) { ) {
updateStoredState((current) => ({ updateStoredState((current) => ({
...current, ...current,
@@ -757,6 +935,7 @@ async function reconcileStoppedBoot({ removeConfig = false } = {}) {
appliedProfileId: '', appliedProfileId: '',
appliedServerId: '', appliedServerId: '',
appliedServerSnapshot: null, appliedServerSnapshot: null,
appliedFailoverPolicy: null,
})); }));
} }
} }
@@ -770,6 +949,8 @@ async function handleApi(req: IncomingMessage, res: ServerResponse) {
if (await connectionRuntimeRoute.handle(req, res)) return; if (await connectionRuntimeRoute.handle(req, res)) return;
if (await routeRulesRoute.handle(req, res)) return; if (await routeRulesRoute.handle(req, res)) return;
if (await gatewayAutoRoute.handle(req, res)) return; if (await gatewayAutoRoute.handle(req, res)) return;
if (await failoverRoute.handle(req, res)) return;
if (await activityJournalRoute.handle(req, res)) return;
if (await connectivityDiagnosticsRoute.handle(req, res)) return; if (await connectivityDiagnosticsRoute.handle(req, res)) return;
if (await versionRoute.handle(req, res)) return; if (await versionRoute.handle(req, res)) return;
@@ -822,6 +1003,7 @@ async function shutdown() {
subscriptionService.stopAutoRefresh(); subscriptionService.stopAutoRefresh();
gatewayAutoService.stopDiscovery(); gatewayAutoService.stopDiscovery();
if (deviceDiscoveryTimer) clearInterval(deviceDiscoveryTimer); if (deviceDiscoveryTimer) clearInterval(deviceDiscoveryTimer);
await gatewayFailover?.shutdown().catch((error) => console.warn(`[control] failover shutdown: ${errorMessage(error)}`));
await serializeControl(() => singboxRuntime.shutdown()); await serializeControl(() => singboxRuntime.shutdown());
process.exit(0); process.exit(0);
} }
@@ -846,8 +1028,14 @@ if (bootWantsRunning) {
&& currentConfigMatchesAppliedTarget(normalizeStoredState(stateStore.read())); && currentConfigMatchesAppliedTarget(normalizeStoredState(stateStore.read()));
if (target || canReuseCurrentConfig) { if (target || canReuseCurrentConfig) {
await startSingbox() await startSingbox()
.then(() => { .then(async () => {
const current = normalizeStoredState(stateStore.read()); const current = normalizeStoredState(stateStore.read());
const bootRole = current.appliedFailoverPolicy
&& current.appliedProfileId === current.appliedFailoverPolicy.reserve.profileId
&& current.appliedServerId === current.appliedFailoverPolicy.reserve.serverId
? 'reserve'
: current.appliedFailoverPolicy ? 'primary' : null;
if (bootRole) await failoverDataplane.selectFailoverRole(bootRole);
const appliedProfile = target?.profile const appliedProfile = target?.profile
|| current.profiles.find((profile) => profile.id === current.appliedProfileId); || current.profiles.find((profile) => profile.id === current.appliedProfileId);
const appliedServer = target?.server const appliedServer = target?.server
@@ -866,7 +1054,10 @@ if (bootWantsRunning) {
})); }));
} }
}) })
.catch((error: unknown) => console.warn(`[control] sing-box не запущен: ${errorMessage(error)}`)); .catch(async (error: unknown) => {
console.warn(`[control] sing-box не запущен: ${errorMessage(error)}`);
await reconcileStoppedBoot();
});
} else { } else {
await reconcileStoppedBoot({ removeConfig: true }); await reconcileStoppedBoot({ removeConfig: true });
} }
@@ -874,6 +1065,9 @@ if (bootWantsRunning) {
await reconcileStoppedBoot(); await reconcileStoppedBoot();
} }
await gatewayFailover?.reconcile()
.catch((error) => console.warn(`[control] failover reconcile: ${errorMessage(error)}`));
if (deviceInventory) { if (deviceInventory) {
await deviceInventory.reconcilePolicies() await deviceInventory.reconcilePolicies()
.catch((error: unknown) => console.warn(`[control] device policy не применена: ${errorMessage(error)}`)); .catch((error: unknown) => console.warn(`[control] device policy не применена: ${errorMessage(error)}`));
@@ -0,0 +1,152 @@
import crypto from 'node:crypto';
import fs from 'node:fs';
import {
ACTIVITY_JOURNAL_MAX_EVENTS,
ACTIVITY_JOURNAL_RETENTION_DAYS,
normalizeActivityEventInput,
normalizeStoredActivityEvent,
type ActivityJournalEvent,
type ActivityJournalEventInput,
type ActivityJournalPage,
} from '../../shared/activityJournal.js';
import { createJsonStore } from './stateStore.js';
interface JournalState {
schemaVersion: 1;
events: ActivityJournalEvent[];
}
const migrateJournal = (value: unknown): JournalState => {
const candidate = value && typeof value === 'object' && !Array.isArray(value)
? value as Record<string, unknown>
: {};
return {
schemaVersion: 1,
events: (Array.isArray(candidate.events) ? candidate.events : [])
.map(normalizeStoredActivityEvent)
.filter((event): event is ActivityJournalEvent => Boolean(event)),
};
};
export function createActivityJournalService({
filePath,
now = () => new Date(),
}: {
filePath: string;
now?: () => Date;
}) {
const store = createJsonStore<JournalState>({
filePath,
defaultValue: { schemaVersion: 1, events: [] },
migrate: migrateJournal,
});
let recoveryRecorded = false;
let writeFailed = false;
function retained(events: ActivityJournalEvent[]) {
const cutoff = now().getTime() - ACTIVITY_JOURNAL_RETENTION_DAYS * 86_400_000;
return events
.filter(({ occurredAt }) => Date.parse(occurredAt) >= cutoff)
.slice(-ACTIVITY_JOURNAL_MAX_EVENTS);
}
function append(value: ActivityJournalEventInput) {
const input = normalizeActivityEventInput(value);
const storedInput = input.dedupeKey ? {
...input,
dedupeKey: `${input.type}:sha256:${crypto.createHash('sha256').update(input.dedupeKey).digest('hex')}`,
} : input;
let appended: ActivityJournalEvent | null = null;
try {
store.update((state) => {
const events = retained(state.events);
if (storedInput.dedupeKey && events.some(({ dedupeKey }) => dedupeKey === storedInput.dedupeKey)) {
return { schemaVersion: 1, events };
}
appended = {
id: crypto.randomUUID(),
occurredAt: now().toISOString(),
...storedInput,
};
return { schemaVersion: 1, events: retained([...events, appended]) };
});
writeFailed = false;
} catch (error) {
writeFailed = true;
throw error;
}
return appended;
}
function ensureRecoveryEvent() {
if (!store.recovery || recoveryRecorded) return;
append({
type: 'journal.recovered',
severity: 'warning',
source: 'storage',
dedupeKey: `journal.recovered:${store.recovery.recoveredAt}`,
data: {},
});
recoveryRecorded = true;
}
function page(limitValue: unknown = 50, cursorValue: unknown = null): ActivityJournalPage {
try {
let state = store.read();
ensureRecoveryEvent();
if (store.recovery) state = store.read();
const retainedEvents = retained(state.events);
if (retainedEvents.length !== state.events.length) {
state = store.update(() => ({ schemaVersion: 1, events: retainedEvents }));
}
const events = [...state.events].reverse();
const limit = Math.min(100, Math.max(1, Number.isSafeInteger(limitValue) ? Number(limitValue) : 50));
const cursor = typeof cursorValue === 'string' ? cursorValue : '';
const cursorIndex = cursor ? events.findIndex(({ id }) => id === cursor) : -1;
if (cursor && cursorIndex < 0) return {
events: [],
nextCursor: null,
retentionDays: 30,
generatedAt: now().toISOString(),
storage: writeFailed
? { status: 'error', errorCode: 'JOURNAL_UNAVAILABLE' }
: { status: 'ready', errorCode: null },
};
const safeStart = cursorIndex + 1;
const selected = events.slice(safeStart, safeStart + limit);
return {
events: selected.map((event) => ({ ...event, dedupeKey: null })),
nextCursor: safeStart + selected.length < events.length ? selected.at(-1)?.id || null : null,
retentionDays: 30,
generatedAt: now().toISOString(),
storage: writeFailed
? { status: 'error', errorCode: 'JOURNAL_UNAVAILABLE' }
: { status: 'ready', errorCode: null },
};
} catch {
return {
events: [],
nextCursor: null,
retentionDays: 30,
generatedAt: now().toISOString(),
storage: { status: 'error', errorCode: 'JOURNAL_UNAVAILABLE' },
};
}
}
if (fs.existsSync(filePath)) {
try {
const state = store.read();
const retainedEvents = retained(state.events);
if (retainedEvents.length !== state.events.length) {
store.update(() => ({ schemaVersion: 1, events: retainedEvents }));
}
} catch {
writeFailed = true;
}
}
return { append, page };
}
export type ActivityJournalService = ReturnType<typeof createActivityJournalService>;
@@ -44,6 +44,7 @@ interface RequestOptions {
ipv4?: boolean; ipv4?: boolean;
follow?: boolean; follow?: boolean;
resolve?: string | null; resolve?: string | null;
timeoutMs?: number;
} }
interface RequestResult { interface RequestResult {
@@ -179,7 +180,9 @@ async function request(probe: BaseProbe, path: PathKind, proxyPort: number, exec
ipv4 = false, ipv4 = false,
follow = true, follow = true,
resolve = null, resolve = null,
timeoutMs = 6_000,
}: RequestOptions = {}): Promise<RequestResult> { }: RequestOptions = {}): Promise<RequestResult> {
const boundedTimeoutMs = Math.min(30_000, Math.max(1_000, Math.round(timeoutMs)));
const args = [ const args = [
'--silent', '--silent',
'--show-error', '--show-error',
@@ -189,9 +192,9 @@ async function request(probe: BaseProbe, path: PathKind, proxyPort: number, exec
'--proto-redir', '--proto-redir',
'=https', '=https',
'--connect-timeout', '--connect-timeout',
'3', String(Math.min(3_000, boundedTimeoutMs) / 1_000),
'--max-time', '--max-time',
'6', String(boundedTimeoutMs / 1_000),
'--user-agent', '--user-agent',
'Harbor-Diagnostics/1', 'Harbor-Diagnostics/1',
'--output', '--output',
@@ -379,6 +382,8 @@ async function siteProbe(
proxyPort: number, proxyPort: number,
execute: CurlExecutor, execute: CurlExecutor,
sampleCount = 1, sampleCount = 1,
timeoutMs = 6_000,
retryFailure = true,
): Promise<SiteProbeResult> { ): Promise<SiteProbeResult> {
if (probe.validationError) return { if (probe.validationError) return {
id: probe.id, id: probe.id,
@@ -391,12 +396,12 @@ async function siteProbe(
stage: 'validation', stage: 'validation',
error: probe.validationError, error: probe.validationError,
}; };
const options = { follow: probe.follow !== false, resolve: probe.resolve }; const options = { follow: probe.follow !== false, resolve: probe.resolve, timeoutMs };
const samples = []; const samples = [];
for (let attempt = 0; attempt < sampleCount; attempt += 1) { for (let attempt = 0; attempt < sampleCount; attempt += 1) {
samples.push(await request(probe, path, proxyPort, execute, options)); samples.push(await request(probe, path, proxyPort, execute, options));
} }
if (sampleCount === 1 && samples[0] && !samples[0].ok) { if (retryFailure && sampleCount === 1 && samples[0] && !samples[0].ok) {
samples.push(await request(probe, path, proxyPort, execute, options)); samples.push(await request(probe, path, proxyPort, execute, options));
} }
const status = mostCommon(samples.map(siteStatus)) || 'unavailable'; const status = mostCommon(samples.map(siteStatus)) || 'unavailable';
@@ -469,15 +474,18 @@ async function probeTarget(
path: PathKind, path: PathKind,
proxyPort: number, proxyPort: number,
execute: CurlExecutor, execute: CurlExecutor,
timeoutMs = 6_000,
sampleCount = TARGET_SAMPLE_COUNT,
retryFailure = true,
): Promise<ConnectivityPathResult> { ): Promise<ConnectivityPathResult> {
const network = target.kind === 'network' const network = target.kind === 'network'
? await networkProbe(path, proxyPort, execute, TARGET_SAMPLE_COUNT) ? await networkProbe(path, proxyPort, execute, sampleCount)
: null; : null;
const ip = target.kind === 'ip' const ip = target.kind === 'ip'
? await ipProbe(target.probe, path, proxyPort, execute, TARGET_SAMPLE_COUNT) ? await ipProbe(target.probe, path, proxyPort, execute, sampleCount)
: null; : null;
const site = target.kind === 'site' const site = target.kind === 'site'
? await siteProbe(target.probe, path, proxyPort, execute, TARGET_SAMPLE_COUNT) ? await siteProbe(target.probe, path, proxyPort, execute, sampleCount, timeoutMs, retryFailure)
: null; : null;
const ipv4Sources = ip?.family === 4 ? [ip] : []; const ipv4Sources = ip?.family === 4 ? [ip] : [];
const ipv6Source = ip?.family === 6 ? ip : null; const ipv6Source = ip?.family === 6 ? ip : null;
@@ -537,7 +545,25 @@ export function createConnectivityDiagnosticsService({
assessment: assessConnectivity(direct, vpn), assessment: assessConnectivity(direct, vpn),
}; };
} }
async function runVpn({ services = [], target: targetId = null, timeoutMs = 6_000 }: {
services?: unknown;
target?: unknown;
timeoutMs?: number;
}) {
const requestedServices = typeof targetId === 'string' && targetId.startsWith('site:custom-')
? (Array.isArray(services) ? services : []).filter((service) => `site:${String(record(service).id || '')}` === targetId)
: [];
const customProbes = await prepareCustomProbes(requestedServices, lookup);
const siteProbes = [...SITE_PROBES, ...customProbes];
const target = resolveTarget(targetId, siteProbes);
if (!target || target.kind !== 'site') throw new Error('Failover check ожидает site target');
return { return {
checkedAt: now(),
vpn: await probeTarget(target, 'vpn', proxyPort, execute, timeoutMs, TARGET_SAMPLE_COUNT, false),
};
}
return {
run: runOnce, run: runOnce,
runVpn,
}; };
} }
+104 -2
View File
@@ -84,12 +84,33 @@ interface DomainTrafficSnapshot {
}>; }>;
} }
interface ActivityEntry {
device: string;
service: string;
upload: bigint;
download: bigint;
}
interface ActivitySample {
at: number;
entries: ActivityEntry[];
}
function record(value: unknown): Record<string, unknown> { function record(value: unknown): Record<string, unknown> {
return value && typeof value === 'object' && !Array.isArray(value) return value && typeof value === 'object' && !Array.isArray(value)
? value as Record<string, unknown> ? value as Record<string, unknown>
: {}; : {};
} }
function publicDeviceLabel(value: unknown) {
const device = record(value);
for (const candidate of [device.alias, device.hostname]) {
const label = typeof candidate === 'string' ? candidate.trim() : '';
if (label && label.length <= 64 && !/[\/?#@\\]/.test(label) && !net.isIP(label)) return label;
}
return 'Устройство';
}
const matchesDomain = (domain: string, suffix: string) => domain === suffix || domain.endsWith(`.${suffix}`); const matchesDomain = (domain: string, suffix: string) => domain === suffix || domain.endsWith(`.${suffix}`);
export function classifyDomain(value: unknown): { domain: string; service: string } | null { export function classifyDomain(value: unknown): { domain: string; service: string } | null {
@@ -209,6 +230,10 @@ export function createDomainTrafficService({
unsupported_source: 0n, unsupported_source: 0n,
}; };
let refreshPromise: Promise<DomainTrafficSnapshot> | null = null; let refreshPromise: Promise<DomainTrafficSnapshot> | null = null;
let activityEnabled = false;
let activityStartedAt = 0;
let activitySamples: ActivitySample[] = [];
let quietSince: string | null = null;
let current: DomainTrafficSnapshot = { let current: DomainTrafficSnapshot = {
epoch, epoch,
observedAt: null, observedAt: null,
@@ -270,6 +295,7 @@ export function createDomainTrafficService({
const response = record(await observe()); const response = record(await observe());
if (!Array.isArray(response.connections)) throw new Error('Sing-box не вернул connections array'); if (!Array.isArray(response.connections)) throw new Error('Sing-box не вернул connections array');
const devicesByIp = new Map<string, string | null>(); const devicesByIp = new Map<string, string | null>();
const deviceLabels = new Map<string, string>();
const observedDevices = devices(); const observedDevices = devices();
for (const value of Array.isArray(observedDevices) ? observedDevices : []) { for (const value of Array.isArray(observedDevices) ? observedDevices : []) {
const device = record(value); const device = record(value);
@@ -277,9 +303,11 @@ export function createDomainTrafficService({
const id = typeof device.mac === 'string' ? deviceId(device.mac.toLowerCase()) : null; const id = typeof device.mac === 'string' ? deviceId(device.mac.toLowerCase()) : null;
if (!net.isIPv4(ip) || !id) continue; if (!net.isIPv4(ip) || !id) continue;
devicesByIp.set(ip, devicesByIp.has(ip) ? null : id); devicesByIp.set(ip, devicesByIp.has(ip) ? null : id);
deviceLabels.set(id, publicDeviceLabel(device));
} }
const connections = response.connections.map((connection) => parseConnection(connection, devicesByIp)); const connections = response.connections.map((connection) => parseConnection(connection, devicesByIp));
const activeConnections = new Map<string, PreviousConnection>(); const activeConnections = new Map<string, PreviousConnection>();
const activityEntries: ActivityEntry[] = [];
for (const connection of connections) { for (const connection of connections) {
const previous = previousConnections.get(connection.id); const previous = previousConnections.get(connection.id);
if (connection.outcome !== 'classified' && previous?.outcome !== connection.outcome) { if (connection.outcome !== 'classified' && previous?.outcome !== connection.outcome) {
@@ -302,6 +330,16 @@ export function createDomainTrafficService({
tracked.uploadBytes += uploadDelta; tracked.uploadBytes += uploadDelta;
tracked.downloadBytes += downloadDelta; tracked.downloadBytes += downloadDelta;
trackedTotals.set(trackedKey, tracked); trackedTotals.set(trackedKey, tracked);
if (activityEnabled && connection.outbound === 'vpn' && uploadDelta + downloadDelta > 0n) {
activityEntries.push({
device: 'deviceId' in connection
? deviceLabels.get(connection.deviceId) || 'Устройство'
: 'Неизвестное устройство',
service: 'service' in connection ? connection.service : 'Не распознано',
upload: uploadDelta,
download: downloadDelta,
});
}
} }
if (connection.outcome === 'unknown_device' || connection.outcome === 'unsupported_source') { if (connection.outcome === 'unknown_device' || connection.outcome === 'unsupported_source') {
activeConnections.set(connection.id, { activeConnections.set(connection.id, {
@@ -372,7 +410,12 @@ export function createDomainTrafficService({
}); });
} }
previousConnections = activeConnections; previousConnections = activeConnections;
current = { ...current, observedAt: now().toISOString() }; const observed = now();
if (activityEnabled) {
activitySamples.push({ at: observed.getTime(), entries: activityEntries });
activitySamples = activitySamples.filter(({ at }) => at >= observed.getTime() - 10_000);
}
current = { ...current, observedAt: observed.toISOString() };
current = buildSnapshot(); current = buildSnapshot();
return current; return current;
} catch (error) { } catch (error) {
@@ -390,5 +433,64 @@ export function createDomainTrafficService({
return refreshPromise; return refreshPromise;
} }
return { snapshot: () => current, refresh }; function enableActivity() {
if (activityEnabled) return;
activityEnabled = true;
activityStartedAt = now().getTime();
activitySamples = [];
quietSince = null;
}
function disableActivity() {
activityEnabled = false;
activityStartedAt = 0;
activitySamples = [];
quietSince = null;
}
function activitySnapshot(thresholdBytesPerSecond: unknown = 0) {
if (!activityEnabled || !current.observedAt) return null;
const observedAt = Date.parse(current.observedAt);
const threshold = Math.max(0, Number(thresholdBytesPerSecond) || 0);
const divisorMs = Math.max(1_000, Math.min(10_000, observedAt - activityStartedAt || 1_000));
const totals = new Map<string, ActivityEntry>();
let bytes = 0n;
for (const sample of activitySamples) {
for (const entry of sample.entries) {
bytes += entry.upload + entry.download;
const key = `${entry.device}\0${entry.service}`;
const total = totals.get(key) || { ...entry, upload: 0n, download: 0n };
total.upload += entry.upload;
total.download += entry.download;
totals.set(key, total);
}
}
const totalBytesPerSecond = Number(bytes * 1_000n / BigInt(divisorMs));
const active = totalBytesPerSecond > threshold;
quietSince = active ? null : quietSince || current.observedAt;
const latest = activitySamples.at(-1);
return {
state: active ? 'active' : 'quiet',
observedAt: current.observedAt,
windowMs: 10_000,
thresholdBytesPerSecond: threshold,
totalBytesPerSecond,
transmittingConnections: latest?.entries.length || 0,
quietSince,
blockers: [...totals.values()]
.map((entry) => ({
device: entry.device,
service: entry.service,
uploadBytesPerSecond: Number(entry.upload * 1_000n / BigInt(divisorMs)),
downloadBytesPerSecond: Number(entry.download * 1_000n / BigInt(divisorMs)),
}))
.sort((left, right) => (
right.uploadBytesPerSecond + right.downloadBytesPerSecond
- left.uploadBytesPerSecond - left.downloadBytesPerSecond
))
.slice(0, 3),
};
}
return { snapshot: () => current, refresh, enableActivity, disableActivity, activitySnapshot };
} }
@@ -0,0 +1,66 @@
import http from 'node:http';
import {
FAILOVER_PRIMARY_TAG,
FAILOVER_RESERVE_TAG,
FAILOVER_SELECTOR_TAG,
} from '../singbox.js';
type Role = 'primary' | 'reserve';
function request(port: number, method: string, body?: unknown): Promise<unknown> {
return new Promise((resolve, reject) => {
const encoded = body === undefined ? null : JSON.stringify(body);
const req = http.request({
host: '127.0.0.1',
port,
path: `/proxies/${encodeURIComponent(FAILOVER_SELECTOR_TAG)}`,
method,
headers: encoded ? {
'content-type': 'application/json',
'content-length': Buffer.byteLength(encoded),
} : {},
}, (res) => {
const chunks: Buffer[] = [];
res.on('data', (chunk: Buffer) => chunks.push(chunk));
res.on('end', () => {
if ((res.statusCode || 500) >= 400) return reject(new Error(`Sing-box selector HTTP ${res.statusCode}`));
if (!chunks.length) return resolve({});
try {
resolve(JSON.parse(Buffer.concat(chunks).toString('utf8')));
} catch (cause) {
reject(new Error('Sing-box selector вернул невалидный JSON', { cause }));
}
});
});
req.setTimeout(2_000, () => req.destroy(new Error('Sing-box selector timeout')));
req.on('error', reject);
req.end(encoded);
});
}
const tagFor = (role: Role) => role === 'primary' ? FAILOVER_PRIMARY_TAG : FAILOVER_RESERVE_TAG;
const roleFor = (tag: unknown): Role | null => (
tag === FAILOVER_PRIMARY_TAG ? 'primary' : tag === FAILOVER_RESERVE_TAG ? 'reserve' : null
);
export function createSingboxSelectorService({
port,
send = (method: string, body?: unknown) => request(port, method, body),
}: {
port: number;
send?: (method: string, body?: unknown) => Promise<unknown>;
}) {
async function read() {
const value = await send('GET') as Record<string, unknown>;
const role = roleFor(value.now);
if (!role) throw new Error('Sing-box selector вернул неизвестный outbound');
return { role };
}
async function select(role: Role) {
await send('PUT', { name: tagFor(role) });
const selected = await read();
if (selected.role !== role) throw new Error('Sing-box selector не подтвердил переключение');
return selected;
}
return { read, select };
}
+1 -1
View File
@@ -13,7 +13,7 @@ import {
type NormalizedServer, type NormalizedServer,
} from '../../shared/serverIdentity.js'; } from '../../shared/serverIdentity.js';
export const STATE_SCHEMA_VERSION = 7; export const STATE_SCHEMA_VERSION = 8;
export interface AtomicWriteOptions { export interface AtomicWriteOptions {
beforeRename?: (temporaryPath: string, filePath: string) => void; beforeRename?: (temporaryPath: string, filePath: string) => void;
+131 -6
View File
@@ -1,13 +1,20 @@
import fs from 'node:fs'; import fs from 'node:fs';
import crypto from 'node:crypto';
import { settings } from './config.js'; import { settings } from './config.js';
import { HarborError } from '../shared/errors.js'; import { HarborError } from '../shared/errors.js';
import { normalizeRouteRules } from '../shared/routingRules.js'; import { normalizeRouteRules } from '../shared/routingRules.js';
import type { AppliedFailoverPolicy } from '../shared/failover.js';
import { atomicWriteFile, atomicWriteJson } from './services/stateStore.js'; import { atomicWriteFile, atomicWriteJson } from './services/stateStore.js';
const PROXY_TYPES = new Set(['vless', 'vmess', 'trojan', 'shadowsocks', 'hysteria2']); const PROXY_TYPES = new Set(['vless', 'vmess', 'trojan', 'shadowsocks', 'hysteria2']);
const MIXED_INBOUND = 'mixed-in'; const MIXED_INBOUND = 'mixed-in';
const TPROXY_INBOUND = 'tproxy-in'; const TPROXY_INBOUND = 'tproxy-in';
const DIAGNOSTICS_INBOUND = 'diagnostics-vpn-in'; const DIAGNOSTICS_INBOUND = 'diagnostics-vpn-in';
const DIAGNOSTICS_PRIMARY_INBOUND = 'diagnostics-primary-in';
const DIAGNOSTICS_RESERVE_INBOUND = 'diagnostics-reserve-in';
export const FAILOVER_SELECTOR_TAG = 'channel-selector';
export const FAILOVER_PRIMARY_TAG = 'channel-primary';
export const FAILOVER_RESERVE_TAG = 'channel-reserve';
const SNIFF_TIMEOUT = '1s'; const SNIFF_TIMEOUT = '1s';
const SNIFFERS = ['http', 'tls', 'quic']; const SNIFFERS = ['http', 'tls', 'quic'];
@@ -34,18 +41,66 @@ function findOutbound(subscriptionConfig: unknown, selectedTag: unknown): ProxyO
)); ));
} }
function selectedOutbound(subscriptionConfig: unknown, selectedTag: unknown, tag?: string) {
const outbound = structuredClone(findOutbound(subscriptionConfig, selectedTag));
if (!outbound) throw new HarborError('SERVER_NOT_FOUND');
if (tag) outbound.tag = tag;
else if (!outbound.tag) outbound.tag = 'vpn-out';
if (outbound.type === 'vless' && !outbound.packet_encoding) outbound.packet_encoding = 'xudp';
return outbound;
}
export interface DualChannelConfig {
primary: { subscriptionConfig: unknown; selectedServerId: string };
reserve: { subscriptionConfig: unknown; selectedServerId: string };
}
export function fingerprintConfiguredOutbound(value: unknown, selectedServerId: string) {
const outbound = structuredClone(record(value)) as ProxyOutbound;
if (!PROXY_TYPES.has(String(outbound.type || ''))) throw new HarborError('CONFIG_INVALID');
outbound.tag = selectedServerId;
if (outbound.type === 'vless' && !outbound.packet_encoding) outbound.packet_encoding = 'xudp';
return crypto.createHash('sha256').update(JSON.stringify(outbound)).digest('hex');
}
export function fingerprintSelectedOutbound(subscriptionConfig: unknown, selectedServerId: string) {
const outbound = findOutbound(subscriptionConfig, selectedServerId);
if (!outbound) throw new HarborError('SERVER_NOT_FOUND');
return fingerprintConfiguredOutbound(outbound, selectedServerId);
}
export function dualChannelConfigMatchesApplied(
configValue: unknown,
applied: AppliedFailoverPolicy,
expectedRole: 'primary' | 'reserve',
) {
const config = record(configValue);
const outbounds = (Array.isArray(config.outbounds) ? config.outbounds : []).map(record);
const primary = outbounds.filter(({ tag }) => tag === FAILOVER_PRIMARY_TAG);
const reserve = outbounds.filter(({ tag }) => tag === FAILOVER_RESERVE_TAG);
const selector = outbounds.find(({ tag }) => tag === FAILOVER_SELECTOR_TAG);
try {
return primary.length === 1
&& reserve.length === 1
&& fingerprintConfiguredOutbound(primary[0], applied.primary.serverId) === applied.primaryConfigFingerprint
&& fingerprintConfiguredOutbound(reserve[0], applied.reserve.serverId) === applied.reserveConfigFingerprint
&& selector?.type === 'selector'
&& JSON.stringify(selector.outbounds) === JSON.stringify([FAILOVER_PRIMARY_TAG, FAILOVER_RESERVE_TAG])
&& selector.default === (expectedRole === 'reserve' ? FAILOVER_RESERVE_TAG : FAILOVER_PRIMARY_TAG)
&& selector.interrupt_exist_connections === false
&& record(config.route).final === FAILOVER_SELECTOR_TAG;
} catch {
return false;
}
}
export function buildGatewayConfig(subscriptionConfig: unknown, selectedTag: unknown, { export function buildGatewayConfig(subscriptionConfig: unknown, selectedTag: unknown, {
clientDirect = false, clientDirect = false,
routeRules = [], routeRules = [],
}: { clientDirect?: boolean; routeRules?: unknown } = {}) { }: { clientDirect?: boolean; routeRules?: unknown } = {}) {
const clientMode = settings.appMode === 'client'; const clientMode = settings.appMode === 'client';
const directClient = clientMode && clientDirect; const directClient = clientMode && clientDirect;
const vpnOutbound = structuredClone(findOutbound(subscriptionConfig, selectedTag)); const vpnOutbound = selectedOutbound(subscriptionConfig, selectedTag);
if (!vpnOutbound) throw new HarborError('SERVER_NOT_FOUND');
if (!vpnOutbound.tag) vpnOutbound.tag = 'vpn-out';
if (vpnOutbound.type === 'vless' && !vpnOutbound.packet_encoding) {
vpnOutbound.packet_encoding = 'xudp';
}
const outboundTag = directClient ? 'direct' : vpnOutbound.tag; const outboundTag = directClient ? 'direct' : vpnOutbound.tag;
const inbounds = [ const inbounds = [
@@ -124,6 +179,76 @@ export function buildGatewayConfig(subscriptionConfig: unknown, selectedTag: unk
}; };
} }
export function buildDualChannelGatewayConfig(
channels: DualChannelConfig,
{ routeRules = [], defaultRole = 'primary' }: { routeRules?: unknown; defaultRole?: 'primary' | 'reserve' } = {},
) {
if (settings.appMode === 'client') throw new Error('Dual-channel config доступен только Gateway');
const primary = selectedOutbound(
channels.primary.subscriptionConfig,
channels.primary.selectedServerId,
FAILOVER_PRIMARY_TAG,
);
const reserve = selectedOutbound(
channels.reserve.subscriptionConfig,
channels.reserve.selectedServerId,
FAILOVER_RESERVE_TAG,
);
const userRules = normalizeRouteRules(routeRules)
.filter((rule) => rule.enabled)
.map((rule) => ({
[rule.type]: [rule.value],
outbound: rule.outbound === 'vpn' ? FAILOVER_SELECTOR_TAG : 'direct',
}));
const userInbounds = [TPROXY_INBOUND, MIXED_INBOUND];
return {
log: { level: settings.logLevel, timestamp: true },
experimental: {
cache_file: { enabled: true, path: settings.cachePath },
clash_api: { external_controller: `127.0.0.1:${settings.singboxApiPort}` },
},
dns: { independent_cache: true },
inbounds: [
{ type: 'tproxy', tag: TPROXY_INBOUND, listen: '::', listen_port: settings.tproxyPort },
{ type: 'mixed', tag: MIXED_INBOUND, listen: settings.bindIp, listen_port: settings.proxyPort, set_system_proxy: false },
{ type: 'mixed', tag: DIAGNOSTICS_INBOUND, listen: '127.0.0.1', listen_port: settings.diagnosticsProxyPort, set_system_proxy: false },
{ type: 'mixed', tag: DIAGNOSTICS_PRIMARY_INBOUND, listen: '127.0.0.1', listen_port: settings.failoverPrimaryProxyPort, set_system_proxy: false },
{ type: 'mixed', tag: DIAGNOSTICS_RESERVE_INBOUND, listen: '127.0.0.1', listen_port: settings.failoverReserveProxyPort, set_system_proxy: false },
],
outbounds: [
primary,
reserve,
{
type: 'selector',
tag: FAILOVER_SELECTOR_TAG,
outbounds: [FAILOVER_PRIMARY_TAG, FAILOVER_RESERVE_TAG],
default: defaultRole === 'reserve' ? FAILOVER_RESERVE_TAG : FAILOVER_PRIMARY_TAG,
interrupt_exist_connections: false,
},
{ type: 'direct', tag: 'direct' },
],
route: {
rule_set: [],
rules: [
{
inbound: [TPROXY_INBOUND, MIXED_INBOUND, DIAGNOSTICS_INBOUND, DIAGNOSTICS_PRIMARY_INBOUND, DIAGNOSTICS_RESERVE_INBOUND],
action: 'sniff',
sniffer: SNIFFERS,
timeout: SNIFF_TIMEOUT,
},
{ inbound: [DIAGNOSTICS_PRIMARY_INBOUND], outbound: FAILOVER_PRIMARY_TAG },
{ inbound: [DIAGNOSTICS_RESERVE_INBOUND], outbound: FAILOVER_RESERVE_TAG },
{ inbound: [DIAGNOSTICS_INBOUND], outbound: FAILOVER_SELECTOR_TAG },
...userRules,
{ inbound: userInbounds, outbound: FAILOVER_SELECTOR_TAG },
],
final: FAILOVER_SELECTOR_TAG,
auto_detect_interface: true,
},
};
}
export function writeSingboxConfig(config: unknown) { export function writeSingboxConfig(config: unknown) {
atomicWriteJson(settings.configPath, config); atomicWriteJson(settings.configPath, config);
} }
+18
View File
@@ -19,6 +19,23 @@ export function createSingboxRuntime({
const state = () => ({ running: Boolean(child), startedAt }); const state = () => ({ running: Boolean(child), startedAt });
function checkConfig(config: unknown) {
const directory = fs.mkdtempSync(`${configPath}.check-`);
const candidatePath = `${directory}/config.json`;
try {
fs.writeFileSync(candidatePath, JSON.stringify(config));
const check = spawnSync('sing-box', ['check', '-c', candidatePath], { encoding: 'utf8' });
if (check.status !== 0) {
throw new HarborError('CONFIG_INVALID', {
cause: new Error((check.stderr || check.stdout || check.error?.message || 'sing-box check failed').trim()),
});
}
return { valid: true };
} finally {
fs.rmSync(directory, { recursive: true, force: true });
}
}
async function stop() { async function stop() {
if (gateway) setGatewayInterception(false, tproxyChain); if (gateway) setGatewayInterception(false, tproxyChain);
if (!child) { if (!child) {
@@ -100,6 +117,7 @@ export function createSingboxRuntime({
get running() { return Boolean(child); }, get running() { return Boolean(child); },
get startedAt() { return startedAt; }, get startedAt() { return startedAt; },
refresh: async () => state(), refresh: async () => state(),
checkConfig,
apply, apply,
restart: () => apply({ force: true }), restart: () => apply({ force: true }),
stop, stop,
+156
View File
@@ -0,0 +1,156 @@
export const ACTIVITY_JOURNAL_RETENTION_DAYS = 30;
export const ACTIVITY_JOURNAL_MAX_EVENTS = 10_000;
export const ACTIVITY_EVENT_TYPES = [
'connection.started', 'connection.stopped', 'connection.failed',
'subscription.added', 'subscription.refreshed', 'subscription.refresh_failed', 'subscription.deleted',
'failover.enabled', 'failover.disabled', 'failover.paused', 'failover.resumed',
'failover.waiting_for_idle', 'failover.switched', 'failover.switch_failed',
'failover.both_unhealthy', 'failover.recovered', 'journal.recovered',
] as const;
export type ActivityEventType = typeof ACTIVITY_EVENT_TYPES[number];
export type ActivityEventSeverity = 'info' | 'warning' | 'error';
export type ActivityEventSource = 'connection' | 'subscription' | 'failover' | 'storage';
export interface ActivityJournalEvent {
id: string;
occurredAt: string;
type: ActivityEventType | 'unknown';
severity: ActivityEventSeverity;
source: ActivityEventSource;
dedupeKey: string | null;
data: Record<string, string | number | boolean | null>;
}
export type ActivityJournalEventInput = Omit<ActivityJournalEvent, 'id' | 'occurredAt'>;
export interface ActivityJournalPage {
events: ActivityJournalEvent[];
nextCursor: string | null;
retentionDays: 30;
generatedAt: string;
storage: { status: 'ready' | 'error'; errorCode: string | null };
}
const ALLOWED_DATA_KEYS: Record<ActivityEventType, readonly string[]> = {
'connection.started': ['profileLabel', 'serverLabel'],
'connection.stopped': [],
'connection.failed': ['errorCode'],
'subscription.added': ['profileId', 'profileLabel', 'host', 'serverCount'],
'subscription.refreshed': ['profileId', 'profileLabel', 'host', 'serverCount', 'added', 'removed'],
'subscription.refresh_failed': ['profileId', 'profileLabel', 'host', 'errorCode'],
'subscription.deleted': ['profileId', 'profileLabel'],
'failover.enabled': ['primaryLabel', 'reserveLabel'],
'failover.disabled': [],
'failover.paused': [],
'failover.resumed': [],
'failover.waiting_for_idle': ['fromRole', 'toRole', 'reason'],
'failover.switched': ['fromRole', 'toRole', 'primaryLabel', 'reserveLabel', 'reason', 'manual'],
'failover.switch_failed': ['fromRole', 'toRole', 'reason', 'errorCode'],
'failover.both_unhealthy': ['reason'],
'failover.recovered': ['role', 'reason'],
'journal.recovered': [],
};
const typeSet = new Set<string>(ACTIVITY_EVENT_TYPES);
const severitySet = new Set(['info', 'warning', 'error']);
const sourceSet = new Set(['connection', 'subscription', 'failover', 'storage']);
const record = (value: unknown): Record<string, unknown> => (
value && typeof value === 'object' && !Array.isArray(value) ? value as Record<string, unknown> : {}
);
const LABEL_KEYS = new Set(['profileLabel', 'serverLabel', 'primaryLabel', 'reserveLabel']);
const ROLE_KEYS = new Set(['fromRole', 'toRole', 'role']);
const SAFE_LABEL_FALLBACKS: Record<string, string> = {
profileLabel: 'Подписка',
serverLabel: 'Сервер',
primaryLabel: 'Основной канал',
reserveLabel: 'Резервный канал',
};
function safeScalar(key: string, value: unknown) {
if (value === null || typeof value === 'boolean') return value;
if (typeof value === 'number' && Number.isSafeInteger(value) && value >= 0) return value;
if (typeof value !== 'string' || value.length > 120 || /[\r\n]/.test(value)) {
throw new TypeError('Unsafe journal value');
}
if (LABEL_KEYS.has(key) && (
!value.trim()
|| /(?:[a-z][a-z0-9+.-]*:\/\/)|[\/?#@\\]/i.test(value)
|| /(?:^|\D)(?:\d{1,3}\.){3}\d{1,3}(?:\D|$)/.test(value)
|| /(?:^|[^0-9a-f])(?:[0-9a-f]{0,4}:){2,}[0-9a-f]{0,4}(?:[^0-9a-f]|$)/i.test(value)
)) return SAFE_LABEL_FALLBACKS[key];
if (ROLE_KEYS.has(key) && !['primary', 'reserve'].includes(value)) throw new TypeError('Unsafe journal role');
if (key === 'errorCode' && !/^[A-Z0-9_]{1,50}$/.test(value)) throw new TypeError('Unsafe journal error code');
if (key === 'reason' && !/^[a-z0-9-]{1,80}$/.test(value)) throw new TypeError('Unsafe journal reason');
if (key === 'profileId' && !/^[a-zA-Z0-9_-]{1,80}$/.test(value)) throw new TypeError('Unsafe journal profile id');
if (key === 'host' && (
!/^[a-z0-9.-]{1,120}$/i.test(value)
|| /^(?:\d{1,3}\.){3}\d{1,3}$/.test(value)
|| value.includes('..')
)) return 'Провайдер';
return value;
}
export function normalizeActivityEventInput(value: unknown): ActivityJournalEventInput {
const candidate = record(value);
const type = String(candidate.type || '') as ActivityEventType;
const severity = String(candidate.severity || '') as ActivityEventSeverity;
const source = String(candidate.source || '') as ActivityEventSource;
if (!typeSet.has(type) || !severitySet.has(severity) || !sourceSet.has(source)) {
throw new TypeError('Unknown journal event');
}
const inputData = record(candidate.data);
const allowed = new Set(ALLOWED_DATA_KEYS[type]);
if (Object.keys(inputData).some((key) => !allowed.has(key))) throw new TypeError('Unsafe journal data key');
const data = Object.fromEntries(Object.entries(inputData).map(([key, item]) => [key, safeScalar(key, item)]));
const dedupeKey = candidate.dedupeKey == null ? null : String(candidate.dedupeKey).trim();
if (dedupeKey !== null && (
!dedupeKey.startsWith(`${type}:`)
|| !/^[a-zA-Z0-9_.:-]{1,160}$/.test(dedupeKey)
)) {
throw new TypeError('Invalid journal dedupe key');
}
return { type, severity, source, dedupeKey, data };
}
export function normalizeStoredActivityEvent(value: unknown): ActivityJournalEvent | null {
const candidate = record(value);
const id = typeof candidate.id === 'string' && /^[a-f0-9-]{20,50}$/i.test(candidate.id) ? candidate.id : '';
const occurredAt = typeof candidate.occurredAt === 'string' && Number.isFinite(Date.parse(candidate.occurredAt))
? candidate.occurredAt
: '';
if (!id || !occurredAt) return null;
try {
return { id, occurredAt, ...normalizeActivityEventInput(candidate) };
} catch {
const severity = String(candidate.severity || '') as ActivityEventSeverity;
const source = String(candidate.source || '') as ActivityEventSource;
return severitySet.has(severity) && sourceSet.has(source) && typeof candidate.type === 'string'
&& /^[a-z][a-z0-9_.-]{0,79}$/.test(candidate.type)
? { id, occurredAt, type: 'unknown', severity, source, dedupeKey: null, data: {} }
: null;
}
}
export function assertActivityJournalPage(value: unknown): ActivityJournalPage {
const candidate = record(value);
const events = Array.isArray(candidate.events) ? candidate.events.map(normalizeStoredActivityEvent) : [];
const storage = record(candidate.storage);
if (
!Array.isArray(candidate.events) || events.some((event) => event === null)
|| !(candidate.nextCursor === null || typeof candidate.nextCursor === 'string')
|| candidate.retentionDays !== ACTIVITY_JOURNAL_RETENTION_DAYS
|| typeof candidate.generatedAt !== 'string' || !Number.isFinite(Date.parse(candidate.generatedAt))
|| !['ready', 'error'].includes(String(storage.status || ''))
|| !(storage.errorCode === null || typeof storage.errorCode === 'string')
) throw new TypeError('Invalid activity journal page');
return {
events: events as ActivityJournalEvent[],
nextCursor: candidate.nextCursor as string | null,
retentionDays: 30,
generatedAt: candidate.generatedAt,
storage: { status: storage.status as 'ready' | 'error', errorCode: storage.errorCode as string | null },
};
}
+77 -8
View File
@@ -8,6 +8,16 @@ import {
normalizeDiagnosticSettings, normalizeDiagnosticSettings,
type DiagnosticSettings, type DiagnosticSettings,
} from '../connectivityDiagnostics.js'; } from '../connectivityDiagnostics.js';
import {
createIdleFailoverSnapshot,
normalizeAppliedFailoverPolicy,
normalizeFailoverPolicy,
normalizeFailoverRuntimeState,
type AppliedFailoverPolicy,
type FailoverPolicy,
type FailoverRuntimeState,
type FailoverSnapshot,
} from '../failover.js';
export type HarborMode = 'client' | 'gateway'; export type HarborMode = 'client' | 'gateway';
export type ConnectionState = 'running' | 'stopped'; export type ConnectionState = 'running' | 'stopped';
@@ -83,6 +93,7 @@ export interface StateSnapshot {
lastError: string | null; lastError: string | null;
}; };
diagnostics: DiagnosticSettings; diagnostics: DiagnosticSettings;
failover: FailoverSnapshot;
route: { route: {
rulesContractVersion?: typeof ROUTE_RULES_CONTRACT_VERSION; rulesContractVersion?: typeof ROUTE_RULES_CONTRACT_VERSION;
mode: string; mode: string;
@@ -122,6 +133,9 @@ export interface PersistedState extends Record<string, unknown> {
connectionDesired?: ConnectionState; connectionDesired?: ConnectionState;
gatewayAutoEnabled?: boolean; gatewayAutoEnabled?: boolean;
diagnostics: DiagnosticSettings; diagnostics: DiagnosticSettings;
failoverPolicy: FailoverPolicy;
failoverRuntimeState: FailoverRuntimeState;
appliedFailoverPolicy: AppliedFailoverPolicy | null;
} }
// Legacy fields are derived in memory for bounded callers during the v5 cutover. // Legacy fields are derived in memory for bounded callers during the v5 cutover.
@@ -293,6 +307,9 @@ export function normalizeStoredState(value: unknown): StoredState {
? state.routeRulesRevision ? state.routeRulesRevision
: 0, : 0,
diagnostics: normalizeDiagnosticSettings(state.diagnostics), diagnostics: normalizeDiagnosticSettings(state.diagnostics),
failoverPolicy: normalizeFailoverPolicy(state.failoverPolicy),
failoverRuntimeState: normalizeFailoverRuntimeState(state.failoverRuntimeState),
appliedFailoverPolicy: normalizeAppliedFailoverPolicy(state.appliedFailoverPolicy),
subscriptionUrl: selectedProfile?.subscriptionUrl || '', subscriptionUrl: selectedProfile?.subscriptionUrl || '',
selectedServerId, selectedServerId,
selectedTag: selectedServer?.label || '', selectedTag: selectedServer?.label || '',
@@ -327,6 +344,7 @@ export function createStateSnapshot({
appMode, appMode,
configExists, configExists,
operation = { kind: null, status: 'idle', startedAt: null, error: null }, operation = { kind: null, status: 'idle', startedAt: null, error: null },
failoverSnapshot,
now = new Date(), now = new Date(),
}: { }: {
storedState: unknown; storedState: unknown;
@@ -336,6 +354,7 @@ export function createStateSnapshot({
configExists: boolean; configExists: boolean;
subscriptionHost?: string; subscriptionHost?: string;
operation?: OperationState; operation?: OperationState;
failoverSnapshot?: FailoverSnapshot | null;
now?: Date; now?: Date;
}): StateSnapshot { }): StateSnapshot {
const stored = normalizeStoredState(storedState); const stored = normalizeStoredState(storedState);
@@ -391,6 +410,7 @@ export function createStateSnapshot({
lastError: null, lastError: null,
}, },
diagnostics: stored.diagnostics, diagnostics: stored.diagnostics,
failover: failoverSnapshot || createIdleFailoverSnapshot(stored.failoverPolicy),
route: { route: {
rulesContractVersion: ROUTE_RULES_CONTRACT_VERSION, rulesContractVersion: ROUTE_RULES_CONTRACT_VERSION,
mode: routeMode, mode: routeMode,
@@ -429,18 +449,24 @@ export function assertStateSnapshot(snapshot: unknown): StateSnapshot {
const candidateWithDiagnostics = rawCandidate && rawCandidate.diagnostics === undefined const candidateWithDiagnostics = rawCandidate && rawCandidate.diagnostics === undefined
? { ...rawCandidate, diagnostics: normalizeDiagnosticSettings(null) } ? { ...rawCandidate, diagnostics: normalizeDiagnosticSettings(null) }
: rawCandidate; : rawCandidate;
const candidate = candidateWithDiagnostics?.route?.rulesContractVersion === undefined const candidateWithFailover = candidateWithDiagnostics && candidateWithDiagnostics.failover === undefined
&& Array.isArray(candidateWithDiagnostics?.route?.localRules)
&& Array.isArray(candidateWithDiagnostics?.route?.activeLocalRules)
? { ? {
...candidateWithDiagnostics, ...candidateWithDiagnostics,
route: { failover: createIdleFailoverSnapshot(normalizeFailoverPolicy(null)),
...candidateWithDiagnostics.route,
localRules: candidateWithDiagnostics.route.localRules.map(legacyRule),
activeLocalRules: candidateWithDiagnostics.route.activeLocalRules.map(legacyRule),
},
} }
: candidateWithDiagnostics; : candidateWithDiagnostics;
const candidate = candidateWithFailover?.route?.rulesContractVersion === undefined
&& Array.isArray(candidateWithFailover?.route?.localRules)
&& Array.isArray(candidateWithFailover?.route?.activeLocalRules)
? {
...candidateWithFailover,
route: {
...candidateWithFailover.route,
localRules: candidateWithFailover.route.localRules.map(legacyRule),
activeLocalRules: candidateWithFailover.route.activeLocalRules.map(legacyRule),
},
}
: candidateWithFailover;
const validDate = (value: unknown) => typeof value === 'string' && Number.isFinite(Date.parse(value)); const validDate = (value: unknown) => typeof value === 'string' && Number.isFinite(Date.parse(value));
const nullableDate = (value: unknown) => value === null || validDate(value); const nullableDate = (value: unknown) => value === null || validDate(value);
const nullableString = (value: unknown) => value === null || typeof value === 'string'; const nullableString = (value: unknown) => value === null || typeof value === 'string';
@@ -486,6 +512,37 @@ export function assertStateSnapshot(snapshot: unknown): StateSnapshot {
return false; return false;
} }
}; };
const validFailover = (value: FailoverSnapshot) => {
const channel = (item: FailoverSnapshot['primary']) => (
item && typeof item.target?.profileId === 'string' && typeof item.target?.serverId === 'string'
&& ['healthy', 'unhealthy', 'unknown', 'not-monitoring'].includes(item.health)
&& Array.isArray(item.failingServiceIds) && item.failingServiceIds.every((id) => typeof id === 'string')
&& nullableDate(item.checkedAt) && nullableDate(item.stateSince)
);
const activity = value.trafficActivity;
try {
normalizeFailoverPolicy(value.policy, { strict: true });
} catch {
return false;
}
return channel(value.primary) && channel(value.reserve)
&& nullableDate(value.nextDecisionAt) && nullableString(value.reason)
&& (activity === null || (
['active', 'quiet', 'unknown'].includes(activity.state)
&& validDate(activity.observedAt)
&& [activity.windowMs, activity.thresholdBytesPerSecond, activity.totalBytesPerSecond, activity.transmittingConnections]
.every((number) => Number.isFinite(number) && number >= 0)
&& nullableDate(activity.quietSince)
&& (activity.switchTarget === null || ['primary', 'reserve'].includes(activity.switchTarget))
&& Array.isArray(activity.blockers)
&& activity.blockers.length <= 3
&& activity.blockers.every((blocker) => (
typeof blocker.device === 'string' && typeof blocker.service === 'string'
&& Number.isFinite(blocker.uploadBytesPerSecond) && blocker.uploadBytesPerSecond >= 0
&& Number.isFinite(blocker.downloadBytesPerSecond) && blocker.downloadBytesPerSecond >= 0
))
));
};
if ( if (
!snapshot || !snapshot ||
@@ -516,6 +573,18 @@ export function assertStateSnapshot(snapshot: unknown): StateSnapshot {
!nullableDate(candidate.connection.startedAt) || !nullableDate(candidate.connection.startedAt) ||
!nullableString(candidate.connection.lastError) || !nullableString(candidate.connection.lastError) ||
!validDiagnostics(candidate.diagnostics) || !validDiagnostics(candidate.diagnostics) ||
!candidate.failover ||
typeof candidate.failover.observationEpoch !== 'string' ||
!Number.isSafeInteger(candidate.failover.observationSequence) ||
candidate.failover.observationSequence < 0 ||
typeof candidate.failover.enabled !== 'boolean' ||
typeof candidate.failover.paused !== 'boolean' ||
typeof candidate.failover.configured !== 'boolean' ||
!candidate.failover.policy ||
!['inactive', 'active', 'pending', 'passive-loaded'].includes(candidate.failover.activation) ||
!['primary', 'reserve', 'other', 'none'].includes(candidate.failover.currentRole) ||
!['idle', 'observing', 'primary', 'reserve', 'waiting-for-idle', 'blocked', 'switching', 'error'].includes(candidate.failover.status) ||
!validFailover(candidate.failover) ||
!candidate.route || !candidate.route ||
![undefined, ROUTE_RULES_CONTRACT_VERSION].includes(candidate.route.rulesContractVersion) || ![undefined, ROUTE_RULES_CONTRACT_VERSION].includes(candidate.route.rulesContractVersion) ||
typeof candidate.route.mode !== 'string' || typeof candidate.route.mode !== 'string' ||
+355
View File
@@ -0,0 +1,355 @@
export type FailoverRole = 'primary' | 'reserve';
export type FailoverHealth = 'healthy' | 'unhealthy' | 'unknown' | 'not-monitoring';
export interface FailoverTarget {
profileId: string;
serverId: string;
}
export interface FailoverCheck {
serviceId: string;
timeoutMs: number;
}
export interface FailoverPolicy {
version: 1;
enabled: boolean;
paused: boolean;
primary: FailoverTarget;
reserve: FailoverTarget;
checks: FailoverCheck[];
intervalMs: number;
failureWindowMs: number;
recoveryWindowMs: number;
trafficGuard: {
enabled: boolean;
thresholdBytesPerSecond: number;
quietWindowMs: number;
};
minimumReserveMs: number;
flapProtection: {
count: number;
windowMs: number;
quarantineMs: number;
};
}
export interface FailoverRuntimeState {
lastSwitchAt: string | null;
holdUntil: string | null;
primaryQuarantineUntil: string | null;
failoverHistory: string[];
reasonCode: string | null;
}
export interface AppliedFailoverPolicy {
primary: FailoverTarget;
reserve: FailoverTarget;
primaryConfigFingerprint: string;
reserveConfigFingerprint: string;
}
export interface FailoverActivity {
state: 'active' | 'quiet' | 'unknown';
observedAt: string;
windowMs: number;
thresholdBytesPerSecond: number;
totalBytesPerSecond: number;
transmittingConnections: number;
quietSince: string | null;
switchTarget: FailoverRole | null;
blockers: Array<{
device: string;
service: string;
uploadBytesPerSecond: number;
downloadBytesPerSecond: number;
}>;
}
export interface FailoverSnapshot {
observationEpoch: string;
observationSequence: number;
configured: boolean;
enabled: boolean;
paused: boolean;
activation: 'inactive' | 'active' | 'pending' | 'passive-loaded';
currentRole: FailoverRole | 'other' | 'none';
status: 'idle' | 'observing' | 'primary' | 'reserve' | 'waiting-for-idle' | 'blocked' | 'switching' | 'error';
primary: { target: FailoverTarget; health: FailoverHealth; failingServiceIds: string[]; checkedAt: string | null; stateSince: string | null };
reserve: { target: FailoverTarget; health: FailoverHealth; failingServiceIds: string[]; checkedAt: string | null; stateSince: string | null };
nextDecisionAt: string | null;
reason: string | null;
trafficActivity: FailoverActivity | null;
policy: FailoverPolicy;
}
export interface FailoverDecisionMemory {
primaryFailedSince: number | null;
primaryRecoveredSince: number | null;
quietSince: number | null;
}
export interface FailoverDecisionInput {
now: number;
policy: FailoverPolicy;
currentRole: FailoverRole;
primaryHealth: Exclude<FailoverHealth, 'not-monitoring'>;
reserveHealth: Exclude<FailoverHealth, 'not-monitoring'>;
activity: 'active' | 'quiet' | 'unknown';
holdUntil?: number | null;
primaryQuarantineUntil?: number | null;
memory?: FailoverDecisionMemory;
}
export interface FailoverDecision {
status: FailoverSnapshot['status'];
reason: string;
switchTo: FailoverRole | null;
memory: FailoverDecisionMemory;
nextDecisionAt: number | null;
}
const text = (value: unknown) => typeof value === 'string' ? value.trim() : '';
const record = (value: unknown): Record<string, unknown> => (
value && typeof value === 'object' && !Array.isArray(value) ? value as Record<string, unknown> : {}
);
const bounded = (value: unknown, fallback: number, minimum: number, maximum: number) => {
const candidate = Number(value);
return Number.isFinite(candidate) ? Math.min(maximum, Math.max(minimum, Math.round(candidate))) : fallback;
};
const equivalent = (left: unknown, right: unknown): boolean => {
if (Array.isArray(left) || Array.isArray(right)) {
return Array.isArray(left) && Array.isArray(right)
&& left.length === right.length
&& left.every((value, index) => equivalent(value, right[index]));
}
if (left && right && typeof left === 'object' && typeof right === 'object') {
const leftRecord = left as Record<string, unknown>;
const rightRecord = right as Record<string, unknown>;
const leftKeys = Object.keys(leftRecord).sort();
const rightKeys = Object.keys(rightRecord).sort();
return leftKeys.length === rightKeys.length
&& leftKeys.every((key, index) => key === rightKeys[index] && equivalent(leftRecord[key], rightRecord[key]));
}
return Object.is(left, right);
};
export const DEFAULT_FAILOVER_POLICY: FailoverPolicy = Object.freeze({
version: 1,
enabled: false,
paused: false,
primary: { profileId: '', serverId: '' },
reserve: { profileId: '', serverId: '' },
checks: [{ serviceId: 'youtube', timeoutMs: 6_000 }, { serviceId: 'google', timeoutMs: 6_000 }],
intervalMs: 60_000,
failureWindowMs: 120_000,
recoveryWindowMs: 900_000,
trafficGuard: { enabled: true, thresholdBytesPerSecond: 32 * 1024, quietWindowMs: 30_000 },
minimumReserveMs: 600_000,
flapProtection: { count: 3, windowMs: 86_400_000, quarantineMs: 21_600_000 },
});
export const DEFAULT_FAILOVER_RUNTIME_STATE: FailoverRuntimeState = Object.freeze({
lastSwitchAt: null,
holdUntil: null,
primaryQuarantineUntil: null,
failoverHistory: [],
reasonCode: null,
});
function target(value: unknown): FailoverTarget {
const candidate = record(value);
return { profileId: text(candidate.profileId), serverId: text(candidate.serverId) };
}
export function normalizeFailoverPolicy(value: unknown, { strict = false } = {}): FailoverPolicy {
const candidate = record(value);
const requestedChecks = Array.isArray(candidate.checks) ? candidate.checks : DEFAULT_FAILOVER_POLICY.checks;
const checks = requestedChecks.map((value) => {
const check = record(value);
return {
serviceId: text(check.serviceId).slice(0, 100),
timeoutMs: bounded(check.timeoutMs, 6_000, 2_000, 30_000),
};
}).filter(({ serviceId }, index, all) => serviceId && all.findIndex((item) => item.serviceId === serviceId) === index).slice(0, 10);
const trafficGuard = record(candidate.trafficGuard);
const flapProtection = record(candidate.flapProtection);
const policy: FailoverPolicy = {
version: 1,
enabled: candidate.enabled === true,
paused: candidate.paused === true,
primary: target(candidate.primary),
reserve: target(candidate.reserve),
checks,
intervalMs: bounded(candidate.intervalMs, 60_000, 15_000, 900_000),
failureWindowMs: 0,
recoveryWindowMs: bounded(candidate.recoveryWindowMs, 900_000, 60_000, 86_400_000),
trafficGuard: {
enabled: trafficGuard.enabled !== false,
thresholdBytesPerSecond: bounded(trafficGuard.thresholdBytesPerSecond, 32 * 1024, 1024, 100 * 1024 * 1024),
quietWindowMs: bounded(trafficGuard.quietWindowMs, 30_000, 5_000, 600_000),
},
minimumReserveMs: bounded(candidate.minimumReserveMs, 600_000, 60_000, 86_400_000),
flapProtection: {
count: bounded(flapProtection.count, 3, 2, 10),
windowMs: bounded(flapProtection.windowMs, 86_400_000, 3_600_000, 72 * 3_600_000),
quarantineMs: bounded(flapProtection.quarantineMs, 21_600_000, 600_000, 7 * 86_400_000),
},
};
policy.failureWindowMs = bounded(
candidate.failureWindowMs,
120_000,
policy.intervalMs * 2,
1_800_000,
);
if (strict && !equivalent(policy, value)) {
throw new TypeError('Invalid failover policy');
}
return policy;
}
export function normalizeFailoverRuntimeState(value: unknown): FailoverRuntimeState {
const candidate = record(value);
const date = (value: unknown) => typeof value === 'string' && Number.isFinite(Date.parse(value)) ? value : null;
return {
lastSwitchAt: date(candidate.lastSwitchAt),
holdUntil: date(candidate.holdUntil),
primaryQuarantineUntil: date(candidate.primaryQuarantineUntil),
failoverHistory: (Array.isArray(candidate.failoverHistory) ? candidate.failoverHistory : [])
.map(date).filter((item): item is string => Boolean(item)).slice(-20),
reasonCode: text(candidate.reasonCode) || null,
};
}
export function normalizeAppliedFailoverPolicy(value: unknown): AppliedFailoverPolicy | null {
if (!value) return null;
const candidate = record(value);
const primary = target(candidate.primary);
const reserve = target(candidate.reserve);
const primaryConfigFingerprint = text(candidate.primaryConfigFingerprint);
const reserveConfigFingerprint = text(candidate.reserveConfigFingerprint);
return primary.profileId && primary.serverId && reserve.profileId && reserve.serverId
&& /^[a-f0-9]{64}$/.test(primaryConfigFingerprint)
&& /^[a-f0-9]{64}$/.test(reserveConfigFingerprint)
? { primary, reserve, primaryConfigFingerprint, reserveConfigFingerprint }
: null;
}
export function isFailoverConfigured(policy: FailoverPolicy) {
return Boolean(
policy.primary.profileId && policy.primary.serverId
&& policy.reserve.profileId && policy.reserve.serverId
&& policy.checks.length
&& (policy.primary.profileId !== policy.reserve.profileId
|| policy.primary.serverId !== policy.reserve.serverId)
);
}
export function createIdleFailoverSnapshot(
policy: FailoverPolicy,
observationEpoch = '',
observationSequence = 0,
): FailoverSnapshot {
const channel = (target: FailoverTarget) => ({
target,
health: 'not-monitoring' as const,
failingServiceIds: [],
checkedAt: null,
stateSince: null,
});
return {
observationEpoch,
observationSequence,
configured: isFailoverConfigured(policy),
enabled: policy.enabled,
paused: policy.paused,
activation: 'inactive',
currentRole: 'none',
status: 'idle',
primary: channel(policy.primary),
reserve: channel(policy.reserve),
nextDecisionAt: null,
reason: null,
trafficActivity: null,
policy,
};
}
export function nextFailoverDecision(input: FailoverDecisionInput): FailoverDecision {
const memory = input.memory || {
primaryFailedSince: null,
primaryRecoveredSince: null,
quietSince: null,
};
const next = { ...memory };
if (!input.policy.enabled || input.policy.paused) {
return { status: 'idle', reason: input.policy.paused ? 'paused' : 'disabled', switchTo: null, memory: next, nextDecisionAt: null };
}
if (input.primaryHealth === 'unhealthy' && input.reserveHealth === 'unhealthy') {
next.quietSince = null;
return { status: 'blocked', reason: 'both-unhealthy', switchTo: null, memory: next, nextDecisionAt: null };
}
let target: FailoverRole | null = null;
let readyAt: number | null = null;
if (input.currentRole === 'primary') {
next.primaryRecoveredSince = null;
if (input.primaryHealth !== 'unhealthy') {
next.primaryFailedSince = null;
next.quietSince = null;
return { status: 'primary', reason: input.primaryHealth === 'healthy' ? 'primary-healthy' : 'health-unknown', switchTo: null, memory: next, nextDecisionAt: null };
}
next.primaryFailedSince ??= input.now;
readyAt = next.primaryFailedSince + input.policy.failureWindowMs;
if (input.now < readyAt || input.reserveHealth !== 'healthy') {
next.quietSince = null;
return { status: 'observing', reason: input.reserveHealth === 'healthy' ? 'failure-window' : 'reserve-not-healthy', switchTo: null, memory: next, nextDecisionAt: readyAt };
}
target = 'reserve';
} else {
next.primaryFailedSince = null;
if (input.primaryHealth !== 'healthy') {
next.primaryRecoveredSince = null;
next.quietSince = null;
return { status: 'reserve', reason: 'primary-not-recovered', switchTo: null, memory: next, nextDecisionAt: null };
}
if ((input.primaryQuarantineUntil || 0) > input.now) {
next.primaryRecoveredSince = null;
next.quietSince = null;
return {
status: 'reserve',
reason: 'recovery-hold',
switchTo: null,
memory: next,
nextDecisionAt: input.primaryQuarantineUntil || null,
};
}
next.primaryRecoveredSince ??= input.now;
const recoveredAt = next.primaryRecoveredSince + input.policy.recoveryWindowMs;
readyAt = input.reserveHealth === 'unhealthy'
? recoveredAt
: Math.max(recoveredAt, input.holdUntil || 0, input.primaryQuarantineUntil || 0);
if (input.now < readyAt) {
next.quietSince = null;
return { status: 'reserve', reason: 'recovery-hold', switchTo: null, memory: next, nextDecisionAt: readyAt };
}
target = 'primary';
}
if (input.policy.trafficGuard.enabled) {
if (input.activity === 'unknown') {
next.quietSince = null;
return { status: 'blocked', reason: 'activity-unknown', switchTo: null, memory: next, nextDecisionAt: null };
}
if (input.activity === 'active') {
next.quietSince = null;
return { status: 'waiting-for-idle', reason: 'active-traffic', switchTo: null, memory: next, nextDecisionAt: null };
}
next.quietSince ??= input.now;
readyAt = next.quietSince + input.policy.trafficGuard.quietWindowMs;
if (input.now < readyAt) {
return { status: 'waiting-for-idle', reason: 'quiet-window', switchTo: null, memory: next, nextDecisionAt: readyAt };
}
}
return { status: 'switching', reason: target === 'reserve' ? 'primary-failed' : 'primary-recovered', switchTo: target, memory: next, nextDecisionAt: null };
}
+3 -3
View File
@@ -1,7 +1,7 @@
export const HARBOR_VERSIONS = Object.freeze({ export const HARBOR_VERSIONS = Object.freeze({
macClient: '0.27.0', macClient: '0.28.0',
gatewayClient: '0.28.0', gatewayClient: '0.29.0',
gatewayBackend: '0.28.0', gatewayBackend: '0.29.0',
}); });
export interface ParsedVersion { export interface ParsedVersion {
+25
View File
@@ -25,6 +25,7 @@ const componentActions = {
setDevicePolicy: api.devices.setPolicy, setDevicePolicy: api.devices.setPolicy,
pingServers: api.servers.ping, pingServers: api.servers.ping,
runConnectivityDiagnostics: api.diagnostics.connectivity, runConnectivityDiagnostics: api.diagnostics.connectivity,
loadActivityJournal: api.activityJournal.page,
}; };
interface UiError { interface UiError {
@@ -51,6 +52,10 @@ const operationErrorContext: Record<string, string> = {
'subscription-refresh': 'subscription', 'subscription-refresh': 'subscription',
'subscription-forget': 'subscription', 'subscription-forget': 'subscription',
'route-rules': 'routing', 'route-rules': 'routing',
'failover-save': 'failover',
'failover-pause': 'failover',
'failover-resume': 'failover',
'failover-switch': 'failover',
}; };
function asHarborApiError(error: unknown) { function asHarborApiError(error: unknown) {
@@ -304,6 +309,26 @@ export function App() {
() => api.diagnostics.updateSettings(settings, revisionRef.current), () => api.diagnostics.updateSettings(settings, revisionRef.current),
'diagnostics', 'diagnostics',
)} )}
onSaveFailover={(policy: unknown) => run(
'failover',
() => api.failover.save(policy, revisionRef.current),
'failover',
)}
onPauseFailover={(paused: boolean) => run(
'failover',
() => api.failover.pause(paused, revisionRef.current),
'failover',
)}
onSwitchFailover={(role: 'primary' | 'reserve') => run(
'failover',
() => api.failover.switch(role, revisionRef.current),
'failover',
)}
onCheckFailover={() => run(
'failover',
() => api.failover.check(),
'failover',
)}
onDismissError={() => { onDismissError={() => {
setError(null); setError(null);
setDismissedCanonicalError(canonicalErrorId); setDismissedCanonicalError(canonicalErrorId);
+21
View File
@@ -192,6 +192,26 @@ export const api = {
}, },
), ),
}, },
failover: {
save: (policy: unknown, expectedRevision: number) => request('/api/failover', {
method: 'PUT',
body: JSON.stringify({ policy, expectedRevision }),
}),
pause: (paused: boolean, expectedRevision: number) => request('/api/failover/pause', {
method: 'POST',
body: JSON.stringify({ paused, expectedRevision }),
}),
switch: (role: 'primary' | 'reserve', expectedRevision: number) => request('/api/failover/switch', {
method: 'POST',
body: JSON.stringify({ role, expectedRevision }),
}),
check: () => request('/api/failover/check', {
method: 'POST',
}),
},
activityJournal: {
page: (cursor: string | null = null) => request(`/api/activity-journal?limit=50${cursor ? `&cursor=${encodeURIComponent(cursor)}` : ''}`),
},
singbox: { singbox: {
stop: () => request('/api/singbox/stop', { method: 'POST' }), stop: () => request('/api/singbox/stop', { method: 'POST' }),
restart: () => request('/api/singbox/restart', { method: 'POST' }), restart: () => request('/api/singbox/restart', { method: 'POST' }),
@@ -237,6 +257,7 @@ export function parseHarborState(value: unknown): HarborClientState {
selection: snapshot.selection, selection: snapshot.selection,
connection: snapshot.connection, connection: snapshot.connection,
diagnostics: snapshot.diagnostics, diagnostics: snapshot.diagnostics,
failover: snapshot.failover,
route: snapshot.route, route: snapshot.route,
operation: snapshot.operation, operation: snapshot.operation,
servers: snapshot.servers, servers: snapshot.servers,
+98 -3
View File
@@ -47,6 +47,13 @@ import {
InstructionsToggle, InstructionsToggle,
useInstructionsFeature, useInstructionsFeature,
} from '../features/instructions/index.js'; } from '../features/instructions/index.js';
import { FailoverPanel, FailoverToggle, useFailoverFeature } from '../features/failover/index.js';
import {
ActivityJournalPanel,
ActivityJournalToggle,
useActivityJournalFeature,
} from '../features/activity-journal/index.js';
import type { FailoverPolicy } from '../../shared/failover.js';
import { import {
HARBOR_VERSIONS, HARBOR_VERSIONS,
parseVersion, parseVersion,
@@ -65,9 +72,33 @@ const VERSION_PARTS = [
] as const; ] as const;
const DRAWER_SWITCH_MS = 620; const DRAWER_SWITCH_MS = 620;
const DRAWER_ORDER = ['subscription', 'instructions', 'devices', 'diagnostics', 'routing'] as const; const DRAWER_ORDER = ['subscription', 'failover', 'instructions', 'devices', 'diagnostics', 'routing', 'journal'] as const;
type DrawerKey = typeof DRAWER_ORDER[number]; type DrawerKey = typeof DRAWER_ORDER[number];
const failoverReasonLabel = (reason: string | null) => ({
'primary-healthy': 'основной работает',
'health-unknown': 'ожидаем проверку',
'failure-window': 'подтверждаем сбой',
'reserve-not-healthy': 'резерв не подтверждён',
'both-unhealthy': 'оба канала недоступны',
'primary-not-recovered': 'основной восстанавливается',
'recovery-hold': 'проверяем стабильность',
'activity-unknown': 'активность неизвестна',
'active-traffic': 'ждём завершения работы',
'quiet-window': 'проверяем тишину',
'primary-failed': 'основной недоступен',
'primary-recovered': 'основной восстановился',
'pending-activation': 'изменения ожидают запуска',
'vpn-stopped': 'VPN выключен',
paused: 'автоматика на паузе',
disabled: 'резерв выключен',
'switch-failed': 'не удалось переключить',
'selector-unknown': 'текущий канал неизвестен',
'reconcile-failed': 'мониторинг временно недоступен',
'revalidation-required': 'условия проверяются заново',
'manual-check': 'оба канала проверены',
}[reason || ''] || 'наблюдение');
interface UiError { interface UiError {
context?: string; context?: string;
profileId?: string; profileId?: string;
@@ -93,6 +124,7 @@ interface ComponentActions {
setDevicePolicy: (id: string, mode: 'vpn' | 'direct', expectedRevision: number) => Promise<unknown>; setDevicePolicy: (id: string, mode: 'vpn' | 'direct', expectedRevision: number) => Promise<unknown>;
pingServers: (profileId: string, ids: string[]) => Promise<unknown>; pingServers: (profileId: string, ids: string[]) => Promise<unknown>;
runConnectivityDiagnostics: (target?: unknown) => Promise<unknown>; runConnectivityDiagnostics: (target?: unknown) => Promise<unknown>;
loadActivityJournal: (cursor?: string | null) => Promise<unknown>;
} }
interface ClientViewState extends StateSnapshot { interface ClientViewState extends StateSnapshot {
@@ -119,6 +151,10 @@ interface ClientOverviewPageProps {
onSetGatewayAuto: (enabled: boolean) => Promise<unknown>; onSetGatewayAuto: (enabled: boolean) => Promise<unknown>;
onSaveRouteRules: (rules: RouteRule[], expectedRevision: number) => Promise<unknown>; onSaveRouteRules: (rules: RouteRule[], expectedRevision: number) => Promise<unknown>;
onUpdateDiagnosticsSettings: (settings: unknown) => Promise<unknown>; onUpdateDiagnosticsSettings: (settings: unknown) => Promise<unknown>;
onSaveFailover: (policy: FailoverPolicy) => Promise<unknown>;
onPauseFailover: (paused: boolean) => Promise<unknown>;
onSwitchFailover: (role: 'primary' | 'reserve') => Promise<unknown>;
onCheckFailover: () => Promise<unknown>;
onDismissError: () => void; onDismissError: () => void;
} }
@@ -252,6 +288,7 @@ const operationProgress: Partial<Record<keyof OperationRegistrySnapshot, readonl
profileRefresh: ['subscription', 'Обновляем подписку…'], profileRefresh: ['subscription', 'Обновляем подписку…'],
profileDelete: ['subscription', 'Удаляем подписку…'], profileDelete: ['subscription', 'Удаляем подписку…'],
routeRules: ['routing', 'Применяем локальные правила…'], routeRules: ['routing', 'Применяем локальные правила…'],
failover: ['failover', 'Применяем настройки резерва…'],
}; };
const canonicalOperationKeys: Record<string, OperationKey> = { const canonicalOperationKeys: Record<string, OperationKey> = {
@@ -266,6 +303,10 @@ const canonicalOperationKeys: Record<string, OperationKey> = {
'profile-delete': 'profileDelete', 'profile-delete': 'profileDelete',
'gateway-auto': 'gatewayAuto', 'gateway-auto': 'gatewayAuto',
'route-rules': 'routeRules', 'route-rules': 'routeRules',
'failover-save': 'failover',
'failover-pause': 'failover',
'failover-resume': 'failover',
'failover-switch': 'failover',
'subscription-import': 'profileAdd', 'subscription-import': 'profileAdd',
'subscription-refresh': 'profileRefresh', 'subscription-refresh': 'profileRefresh',
'subscription-forget': 'profileDelete', 'subscription-forget': 'profileDelete',
@@ -425,6 +466,10 @@ export function ClientOverviewPage({
onSetGatewayAuto, onSetGatewayAuto,
onSaveRouteRules, onSaveRouteRules,
onUpdateDiagnosticsSettings, onUpdateDiagnosticsSettings,
onSaveFailover,
onPauseFailover,
onSwitchFailover,
onCheckFailover,
onDismissError, onDismissError,
}: ClientOverviewPageProps) { }: ClientOverviewPageProps) {
const isGateway = state?.mode === 'gateway'; const isGateway = state?.mode === 'gateway';
@@ -533,6 +578,8 @@ export function ClientOverviewPage({
port: state?.clientRuntime?.proxyPort || (isGateway ? 8080 : 8082), port: state?.clientRuntime?.proxyPort || (isGateway ? 8080 : 8082),
controlHost, controlHost,
}); });
const failoverFeature = useFailoverFeature();
const activityJournalFeature = useActivityJournalFeature();
const diagnosticsAvailable = hasSubscription; const diagnosticsAvailable = hasSubscription;
const drawerControls = { const drawerControls = {
subscription: { subscription: {
@@ -541,6 +588,12 @@ export function ClientOverviewPage({
show: subscriptionFeature.toggle, show: subscriptionFeature.toggle,
close: subscriptionFeature.close, close: subscriptionFeature.close,
}, },
failover: {
isOpen: failoverFeature.isOpen,
panelRef: failoverFeature.panelRef,
show: failoverFeature.toggle,
close: failoverFeature.close,
},
instructions: { instructions: {
isOpen: instructionsFeature.isOpen, isOpen: instructionsFeature.isOpen,
panelRef: instructionsFeature.panelRef, panelRef: instructionsFeature.panelRef,
@@ -565,10 +618,16 @@ export function ClientOverviewPage({
show: routingFeature.open, show: routingFeature.open,
close: routingFeature.forceClose, close: routingFeature.forceClose,
}, },
journal: {
isOpen: activityJournalFeature.isOpen,
panelRef: activityJournalFeature.panelRef,
show: activityJournalFeature.toggle,
close: activityJournalFeature.close,
},
}; };
const drawerOrder = isGateway const drawerOrder = isGateway
? DRAWER_ORDER ? DRAWER_ORDER
: DRAWER_ORDER.filter((drawer) => drawer !== 'devices'); : DRAWER_ORDER.filter((drawer) => !['devices', 'failover', 'journal'].includes(drawer));
const activeRailDrawer = drawerSwitchTarget && drawerControls[drawerSwitchTarget].isOpen const activeRailDrawer = drawerSwitchTarget && drawerControls[drawerSwitchTarget].isOpen
? drawerSwitchTarget ? drawerSwitchTarget
: drawerOrder.find((drawer) => drawerControls[drawer].isOpen) || null; : drawerOrder.find((drawer) => drawerControls[drawer].isOpen) || null;
@@ -587,6 +646,8 @@ export function ClientOverviewPage({
instructionsFeature.close(); instructionsFeature.close();
devicesFeature.close(); devicesFeature.close();
diagnosticsFeature.close(); diagnosticsFeature.close();
failoverFeature.close();
activityJournalFeature.close();
} }
}, [hasSubscription, isGateway]); }, [hasSubscription, isGateway]);
@@ -656,6 +717,7 @@ export function ClientOverviewPage({
routingFeature.requestClose(); routingFeature.requestClose();
return; return;
} }
if (current === 'failover' && !failoverFeature.beforeCloseRef.current()) return;
if (!current) { if (!current) {
drawerControls[target].show(); drawerControls[target].show();
return; return;
@@ -733,6 +795,13 @@ export function ClientOverviewPage({
: switchingServer && operationProfile && operationServer : switchingServer && operationProfile && operationServer
? `Переключаем на ${subscriptionDomain(operationProfile.subscription.host)} · ${operationServer.label}` ? `Переключаем на ${subscriptionDomain(operationProfile.subscription.host)} · ${operationServer.label}`
: ''; : '';
const failoverIdentity = isGateway && state.failover.enabled
? `${state.failover.currentRole === 'reserve'
? 'Резервный канал'
: state.failover.currentRole === 'primary'
? 'Основной канал'
: 'Текущий канал вне резервной пары'} · ${failoverReasonLabel(state.failover.reason)}`
: '';
return ( return (
<div <div
@@ -752,6 +821,11 @@ export function ClientOverviewPage({
open={activeRailDrawer === 'subscription'} open={activeRailDrawer === 'subscription'}
onToggle={() => switchDrawer('subscription')} onToggle={() => switchDrawer('subscription')}
/> />
{isGateway && <FailoverToggle
feature={failoverFeature}
open={activeRailDrawer === 'failover'}
onToggle={() => switchDrawer('failover')}
/>}
<InstructionsToggle <InstructionsToggle
feature={instructionsFeature} feature={instructionsFeature}
open={activeRailDrawer === 'instructions'} open={activeRailDrawer === 'instructions'}
@@ -775,6 +849,11 @@ export function ClientOverviewPage({
hasSubscription={hasSubscription} hasSubscription={hasSubscription}
onOpen={() => switchDrawer('routing')} onOpen={() => switchDrawer('routing')}
/> />
{isGateway && <ActivityJournalToggle
feature={activityJournalFeature}
open={activeRailDrawer === 'journal'}
onToggle={() => switchDrawer('journal')}
/>}
</nav>} </nav>}
<main className={`client-panel${showPower ? '' : ' is-setup'}${!isGateway && hasSubscription ? ' has-subscription' : ''}${isGateway && hasSubscription ? ' is-gateway-home' : ''}`}> <main className={`client-panel${showPower ? '' : ' is-setup'}${!isGateway && hasSubscription ? ' has-subscription' : ''}${isGateway && hasSubscription ? ' is-gateway-home' : ''}`}>
<ConnectionPanel <ConnectionPanel
@@ -807,7 +886,7 @@ export function ClientOverviewPage({
blocked={connectionBlocked} blocked={connectionBlocked}
onRestart={onRestart} onRestart={onRestart}
/>} />}
serverSlot={<AppliedIdentity identity={mainIdentity} operation={switchIdentity} />} serverSlot={<AppliedIdentity identity={mainIdentity} operation={switchIdentity || failoverIdentity} />}
statusSlot={<> statusSlot={<>
<InlineError error={error} context="connection" /> <InlineError error={error} context="connection" />
<InlineProgress operations={visibleOperations} context="connection" /> <InlineProgress operations={visibleOperations} context="connection" />
@@ -857,6 +936,22 @@ export function ClientOverviewPage({
<InlineProgress operations={visibleOperations} context="routing" /> <InlineProgress operations={visibleOperations} context="routing" />
</>} </>}
/>} />}
{isGateway && hasSubscription && <FailoverPanel
feature={failoverFeature}
snapshot={state.failover}
profiles={profiles}
diagnostics={state.diagnostics}
blocked={operationBlocked(visibleOperations, 'failover')}
onSave={onSaveFailover}
onPause={onPauseFailover}
onSwitch={onSwitchFailover}
onCheck={onCheckFailover}
onUpdateDiagnostics={onUpdateDiagnosticsSettings}
/>}
{isGateway && hasSubscription && <ActivityJournalPanel
feature={activityJournalFeature}
loadPage={actions.loadActivityJournal}
/>}
<RoutingDiscardDialog feature={routingFeature} /> <RoutingDiscardDialog feature={routingFeature} />
<SubscriptionDeleteDialog feature={subscriptionFeature} /> <SubscriptionDeleteDialog feature={subscriptionFeature} />
</div> </div>
@@ -0,0 +1,167 @@
import { useEffect, useRef, useState } from 'react';
import { assertActivityJournalPage, type ActivityJournalEvent } from '../../../shared/activityJournal.js';
import { Drawer } from '../../ui/Drawer.js';
import { RailAction } from '../../ui/RailAction.js';
export function useActivityJournalFeature() {
const [isOpen, setIsOpen] = useState(false);
const panelRef = useRef<HTMLElement>(null);
const toggleRef = useRef<HTMLButtonElement>(null);
const closeRef = useRef<HTMLButtonElement>(null);
useEffect(() => {
if (!isOpen) return undefined;
const frame = requestAnimationFrame(() => closeRef.current?.focus());
const close = (event: PointerEvent | KeyboardEvent) => {
if (event.type === 'keydown' && (event as KeyboardEvent).key !== 'Escape') return;
if (event.type !== 'keydown' && (
panelRef.current?.contains(event.target as Node) || toggleRef.current?.contains(event.target as Node)
)) return;
setIsOpen(false);
};
document.addEventListener('pointerdown', close);
document.addEventListener('keydown', close);
return () => {
cancelAnimationFrame(frame);
document.removeEventListener('pointerdown', close);
document.removeEventListener('keydown', close);
requestAnimationFrame(() => {
if (panelRef.current?.contains(document.activeElement)) toggleRef.current?.focus();
});
};
}, [isOpen]);
return { isOpen, panelRef, toggleRef, closeRef, close: () => setIsOpen(false), toggle: () => setIsOpen((value) => !value) };
}
export type ActivityJournalFeature = ReturnType<typeof useActivityJournalFeature>;
export function ActivityJournalToggle({ feature, open, onToggle }: {
feature: ActivityJournalFeature;
open: boolean;
onToggle: () => void;
}) {
return <RailAction
buttonRef={feature.toggleRef}
className="client-journal-toggle"
open={open}
controls="client-activity-journal"
ariaLabel={open ? 'Закрыть журнал событий' : 'Открыть журнал событий'}
label="Журнал"
onClick={onToggle}
>
<svg viewBox="0 0 24 24" aria-hidden="true">
<path d="M5 8V4m0 4h4M5.6 7.1A8 8 0 1 1 4 12M12 7.5V12l3 2" />
<circle cx="12" cy="12" r=".8" />
</svg>
</RailAction>;
}
function eventCopy(event: ActivityJournalEvent) {
const value = event.data;
const copies: Record<string, [string, string]> = {
'connection.started': ['VPN включён', [value.profileLabel, value.serverLabel].filter(Boolean).join(' · ')],
'connection.stopped': ['VPN выключен', 'Остановлен пользователем'],
'connection.failed': ['VPN не запущен', String(value.errorCode || '')],
'subscription.added': ['Подписка добавлена', `${value.profileLabel || ''} · серверов: ${value.serverCount || 0}`],
'subscription.refreshed': ['Подписка обновлена', `${value.profileLabel || ''} · серверов: ${value.serverCount || 0} · +${value.added || 0} / ${value.removed || 0}`],
'subscription.refresh_failed': ['Подписка не обновлена', `${value.profileLabel || ''} · ${value.errorCode || ''}`],
'subscription.deleted': ['Подписка удалена', String(value.profileLabel || '')],
'failover.enabled': ['Резервный канал включён', 'Мониторинг начнётся после активации dual-config'],
'failover.disabled': ['Резервный канал выключен', 'Автоматика полностью остановлена'],
'failover.paused': ['Автопереключение на паузе', 'Проверки продолжаются'],
'failover.resumed': ['Автопереключение возобновлено', ''],
'failover.waiting_for_idle': ['Переключение отложено', 'Обнаружен активный трафик'],
'failover.switched': ['Новые соединения переключены', `${value.fromRole || ''}${value.toRole || ''} · ${value.reason || ''}`],
'failover.switch_failed': ['Переключение не выполнено', String(value.errorCode || '')],
'failover.both_unhealthy': ['Оба канала недоступны', 'Текущий маршрут сохранён'],
'failover.recovered': ['Основной канал восстановлен', String(value.reason || '')],
'journal.recovered': ['Журнал восстановлен', 'Повреждённый файл сохранён отдельно'],
};
return copies[event.type] || ['Системное событие', ''];
}
function dayLabel(value: string) {
const date = new Date(value);
const today = new Date();
const startDate = new Date(today.getFullYear(), today.getMonth(), today.getDate());
const start = startDate.getTime();
const yesterday = new Date(startDate);
yesterday.setDate(yesterday.getDate() - 1);
const day = new Date(date.getFullYear(), date.getMonth(), date.getDate()).getTime();
if (day === start) return 'Сегодня';
if (day === yesterday.getTime()) return 'Вчера';
return new Intl.DateTimeFormat('ru-RU', { day: 'numeric', month: 'long' }).format(date);
}
export function ActivityJournalPanel({ feature, loadPage }: {
feature: ActivityJournalFeature;
loadPage: (cursor?: string | null) => Promise<unknown>;
}) {
const [events, setEvents] = useState<ActivityJournalEvent[]>([]);
const [nextCursor, setNextCursor] = useState<string | null>(null);
const [status, setStatus] = useState<'idle' | 'loading' | 'ready' | 'refreshing' | 'older' | 'error'>('idle');
const [announcement, setAnnouncement] = useState('');
async function load(cursor: string | null = null, refresh = false) {
setStatus(cursor ? 'older' : refresh ? 'refreshing' : 'loading');
try {
const page = assertActivityJournalPage(await loadPage(cursor));
if (page.storage.status === 'error') throw new Error('journal unavailable');
if (refresh) {
const known = new Set(events.map(({ id }) => id));
setAnnouncement(`Журнал обновлён, новых событий: ${page.events.filter(({ id }) => !known.has(id)).length}`);
}
setEvents((current) => cursor ? [...current, ...page.events] : page.events);
setNextCursor(page.nextCursor);
setStatus('ready');
} catch {
setStatus('error');
}
}
useEffect(() => {
if (feature.isOpen) void load(null, status !== 'idle');
}, [feature.isOpen]);
const groups = events.reduce<Array<{ label: string; events: ActivityJournalEvent[] }>>((result, event) => {
const label = dayLabel(event.occurredAt);
const group = result.at(-1);
if (group?.label === label) group.events.push(event);
else result.push({ label, events: [event] });
return result;
}, []);
return <Drawer
panelRef={feature.panelRef}
closeRef={feature.closeRef}
id="client-activity-journal"
open={feature.isOpen}
label="Журнал Harbor"
closeLabel="Закрыть журнал"
onClose={feature.close}
className="client-journal-drawer"
>
<header className="client-journal-header">
<span>Важные события хранятся 30 дней</span>
<div><h2>Журнал</h2><button type="button" aria-label="Обновить журнал" disabled={status === 'refreshing'} onClick={() => void load(null, true)}>
<svg viewBox="0 0 24 24" aria-hidden="true"><path d="M20 11a8 8 0 1 0-2.3 6.7M20 5v6h-6" /></svg>
</button></div>
</header>
<span className="client-live-region" role="status" aria-live="polite">{announcement}</span>
{status === 'error' && <div className="client-journal-error" role="status">Журнал временно недоступен <button type="button" onClick={() => void load()}>Повторить</button></div>}
{status === 'loading' && !events.length ? <div className="client-journal-skeleton" aria-label="Загружаем журнал">{[0, 1, 2, 3].map((value) => <span key={value} />)}</div>
: !events.length && status === 'ready' ? <p className="client-journal-empty">За последние 30 дней важных событий пока нет</p>
: <div className="client-journal-groups">{groups.map((group) => <section key={group.label}>
<h3>{group.label}</h3>
<ol>{group.events.map((event) => {
const [title, details] = eventCopy(event);
return <li key={event.id} className={event.severity === 'error' ? 'is-error' : event.severity === 'warning' ? 'is-warning' : ''}>
<div className="client-journal-time"><time dateTime={event.occurredAt}>{new Intl.DateTimeFormat('ru-RU', { hour: '2-digit', minute: '2-digit' }).format(new Date(event.occurredAt))}</time><span>{event.source}</span></div>
<div><strong>{title}</strong>{details && <span>{details}</span>}{event.severity !== 'info' && <em>{event.severity === 'error' ? 'Ошибка' : 'Внимание'}</em>}</div>
</li>;
})}</ol>
</section>)}</div>}
<footer className="client-journal-footer">
{nextCursor ? <button type="button" disabled={status === 'older'} onClick={() => void load(nextCursor)}>{status === 'older' ? 'Загружаем…' : 'Показать ещё'}</button> : <span>{events.length ? 'Это вся история за последние 30 дней' : 'Храним события 30 дней'}</span>}
</footer>
</Drawer>;
}
@@ -0,0 +1 @@
export { ActivityJournalPanel, ActivityJournalToggle, useActivityJournalFeature } from './ActivityJournalFeature.js';
@@ -23,6 +23,7 @@ import {
type DiagnosticSiteResult, type DiagnosticSiteResult,
} from './connectivityResult.js'; } from './connectivityResult.js';
import type { DiagnosticsFeature } from './DiagnosticsFeature.js'; import type { DiagnosticsFeature } from './DiagnosticsFeature.js';
import { saveCustomDiagnosticService } from './customServiceAction.js';
const CUSTOM_SERVICES_KEY = 'harbor-diagnostic-services'; const CUSTOM_SERVICES_KEY = 'harbor-diagnostic-services';
const HIDDEN_SERVICES_KEY = 'harbor-hidden-diagnostic-services'; const HIDDEN_SERVICES_KEY = 'harbor-hidden-diagnostic-services';
@@ -334,19 +335,15 @@ export function ConnectivityDiagnosticsPanel({
async function addService(event: FormEvent<HTMLFormElement>) { async function addService(event: FormEvent<HTMLFormElement>) {
event.preventDefault(); event.preventDefault();
try { try {
if (customServices.length >= MAX_CUSTOM_DIAGNOSTIC_SERVICES) return;
const parsed = new URL(serviceUrl.trim());
if (parsed.protocol !== 'https:') throw new Error('Нужен публичный HTTPS-адрес.');
setSettingsSaving(true); setSettingsSaving(true);
const saved = await updateSettings({ const saved = await saveCustomDiagnosticService({
customServices: [...customServices, { name: serviceName,
id: `custom-${globalThis.crypto?.randomUUID?.() || Date.now()}`, url: serviceUrl,
label: serviceName.trim() || parsed.hostname, customServices,
url: parsed.href,
}],
hiddenServiceIds, hiddenServiceIds,
updateSettings,
}); });
if (saved === false) throw new Error('Не удалось сохранить сервис.'); if (!saved) return;
setServiceName(''); setServiceName('');
setServiceUrl(''); setServiceUrl('');
setFormError(''); setFormError('');
@@ -0,0 +1,33 @@
import {
MAX_CUSTOM_DIAGNOSTIC_SERVICES,
type DiagnosticService,
type DiagnosticSettings,
} from '../../../shared/connectivityDiagnostics.js';
export async function saveCustomDiagnosticService({
name,
url,
customServices,
hiddenServiceIds,
updateSettings,
}: {
name: string;
url: string;
customServices: DiagnosticService[];
hiddenServiceIds: string[];
updateSettings: (settings: Pick<DiagnosticSettings, 'customServices' | 'hiddenServiceIds'>) => Promise<unknown>;
}) {
if (customServices.length >= MAX_CUSTOM_DIAGNOSTIC_SERVICES) return false;
const parsed = new URL(url.trim());
if (parsed.protocol !== 'https:') throw new Error('Нужен публичный HTTPS-адрес.');
const saved = await updateSettings({
customServices: [...customServices, {
id: `custom-${globalThis.crypto?.randomUUID?.() || Date.now()}`,
label: name.trim() || parsed.hostname,
url: parsed.href,
}],
hiddenServiceIds,
});
if (saved === false) throw new Error('Не удалось сохранить сервис.');
return true;
}
+1
View File
@@ -4,3 +4,4 @@ export {
useDiagnosticsFeature, useDiagnosticsFeature,
type DiagnosticsFeature, type DiagnosticsFeature,
} from './DiagnosticsFeature.js'; } from './DiagnosticsFeature.js';
export { saveCustomDiagnosticService } from './customServiceAction.js';
@@ -0,0 +1,331 @@
import { useEffect, useMemo, useRef, useState } from 'react';
import { CONNECTIVITY_SITES, MAX_CUSTOM_DIAGNOSTIC_SERVICES, type DiagnosticSettings } from '../../../shared/connectivityDiagnostics.js';
import { normalizeFailoverPolicy, type FailoverPolicy, type FailoverSnapshot } from '../../../shared/failover.js';
import type { ProfileSnapshot } from '../../../shared/contracts/state.js';
import { Drawer } from '../../ui/Drawer.js';
import { RailAction } from '../../ui/RailAction.js';
import { saveCustomDiagnosticService } from '../diagnostics/index.js';
export function useFailoverFeature() {
const [isOpen, setIsOpen] = useState(false);
const panelRef = useRef<HTMLElement>(null);
const toggleRef = useRef<HTMLButtonElement>(null);
const closeRef = useRef<HTMLButtonElement>(null);
const beforeCloseRef = useRef<() => boolean>(() => true);
const close = () => {
if (beforeCloseRef.current()) setIsOpen(false);
};
useEffect(() => {
if (!isOpen) return undefined;
const frame = requestAnimationFrame(() => closeRef.current?.focus());
const handleClose = (event: PointerEvent | KeyboardEvent) => {
if (event.type === 'keydown' && (event as KeyboardEvent).key !== 'Escape') return;
if (event.type !== 'keydown' && (
panelRef.current?.contains(event.target as Node) || toggleRef.current?.contains(event.target as Node)
)) return;
close();
};
document.addEventListener('pointerdown', handleClose);
document.addEventListener('keydown', handleClose);
return () => {
cancelAnimationFrame(frame);
document.removeEventListener('pointerdown', handleClose);
document.removeEventListener('keydown', handleClose);
requestAnimationFrame(() => {
if (panelRef.current?.contains(document.activeElement)) toggleRef.current?.focus();
});
};
}, [isOpen]);
return {
isOpen, panelRef, toggleRef, closeRef, beforeCloseRef, close,
toggle: () => isOpen ? close() : setIsOpen(true),
};
}
export type FailoverFeature = ReturnType<typeof useFailoverFeature>;
export function FailoverToggle({ feature, open, onToggle }: {
feature: FailoverFeature;
open: boolean;
onToggle: () => void;
}) {
return <RailAction
buttonRef={feature.toggleRef}
className="client-failover-toggle"
open={open}
controls="client-failover"
ariaLabel={open ? 'Закрыть резервный канал' : 'Настроить резервный канал'}
label="Резерв"
onClick={onToggle}
>
<svg viewBox="0 0 24 24" aria-hidden="true">
<path d="M5 7h11m-3-3 3 3-3 3M19 17H8m3-3-3 3 3 3" />
<circle cx="4" cy="7" r="1" /><circle cx="20" cy="17" r="1" />
</svg>
</RailAction>;
}
const seconds = (milliseconds: number) => Math.round(milliseconds / 1000);
const milliseconds = (value: string, fallback: number) => {
const parsed = Number(value);
return Number.isFinite(parsed) ? Math.round(parsed * 1000) : fallback;
};
const reasonLabel = (reason: string | null) => ({
'primary-healthy': 'Основной канал работает',
'health-unknown': 'Ожидаем результаты проверки',
'failure-window': 'Подтверждаем сбой основного канала',
'reserve-not-healthy': 'Резервный канал ещё не подтверждён',
'both-unhealthy': 'Оба канала недоступны',
'primary-not-recovered': 'Основной канал восстанавливается',
'recovery-hold': 'Проверяем стабильность основного канала',
'activity-unknown': 'Не удалось определить активность',
'active-traffic': 'Ждём завершения активной работы',
'quiet-window': 'Проверяем тишину перед переключением',
'primary-failed': 'Основной канал недоступен',
'primary-recovered': 'Основной канал восстановился',
'pending-activation': 'Изменения ожидают следующего запуска VPN',
'vpn-stopped': 'VPN выключен',
paused: 'Автоматика на паузе',
disabled: 'Резерв выключен',
'switch-failed': 'Не удалось переключить канал',
'selector-unknown': 'Не удалось подтвердить текущий канал',
'reconcile-failed': 'Настройки сохранены, мониторинг временно недоступен',
'revalidation-required': 'Условия переключения проверяются заново',
'manual-check': 'Оба канала проверены',
}[reason || ''] || 'Наблюдаем за каналами');
function targetLabel(profiles: ProfileSnapshot[], target: { profileId: string; serverId: string }) {
const profile = profiles.find(({ id }) => id === target.profileId);
const server = profile?.servers.find(({ id }) => id === target.serverId);
return profile && server ? `${profile.label} · ${server.label}` : 'Не выбран';
}
export function FailoverPanel({
feature,
snapshot,
profiles,
diagnostics,
blocked,
onSave,
onPause,
onSwitch,
onCheck,
onUpdateDiagnostics,
}: {
feature: FailoverFeature;
snapshot: FailoverSnapshot;
profiles: ProfileSnapshot[];
diagnostics: DiagnosticSettings;
blocked: boolean;
onSave: (policy: FailoverPolicy) => Promise<unknown>;
onPause: (paused: boolean) => Promise<unknown>;
onSwitch: (role: 'primary' | 'reserve') => Promise<unknown>;
onCheck: () => Promise<unknown>;
onUpdateDiagnostics: (settings: unknown) => Promise<unknown>;
}) {
const [draft, setDraft] = useState(() => snapshot.policy);
const [dirty, setDirty] = useState(false);
const [confirmDiscard, setConfirmDiscard] = useState(false);
const [addingService, setAddingService] = useState(false);
const [serviceName, setServiceName] = useState('');
const [serviceUrl, setServiceUrl] = useState('');
const [serviceError, setServiceError] = useState('');
useEffect(() => {
if (!dirty) setDraft(snapshot.policy);
}, [snapshot.policy, dirty]);
useEffect(() => {
feature.beforeCloseRef.current = () => {
if (!dirty) return true;
setConfirmDiscard(true);
return false;
};
const beforeUnload = (event: BeforeUnloadEvent) => {
if (!dirty) return;
event.preventDefault();
};
window.addEventListener('beforeunload', beforeUnload);
return () => {
feature.beforeCloseRef.current = () => true;
window.removeEventListener('beforeunload', beforeUnload);
};
}, [dirty, feature.beforeCloseRef]);
const services = useMemo(() => [
...CONNECTIVITY_SITES,
...diagnostics.customServices,
], [diagnostics.customServices]);
const update = (value: Partial<FailoverPolicy>) => {
setDraft((current) => normalizeFailoverPolicy({ ...current, ...value }));
setDirty(true);
};
const updateTarget = (role: 'primary' | 'reserve', patch: Partial<FailoverPolicy[typeof role]>) => {
const next = { ...draft[role], ...patch };
const profile = profiles.find(({ id }) => id === next.profileId);
if (patch.profileId !== undefined) next.serverId = profile?.desiredServerId || profile?.servers[0]?.id || '';
update({ [role]: next } as Partial<FailoverPolicy>);
};
const role = snapshot.currentRole === 'primary' || snapshot.currentRole === 'reserve'
? snapshot.currentRole
: null;
const switchRole = role === 'primary' ? 'reserve' : role === 'reserve' ? 'primary' : null;
const activity = snapshot.trafficActivity;
const targetExists = (target: FailoverPolicy['primary']) => profiles
.find(({ id }) => id === target.profileId)?.servers.some(({ id }) => id === target.serverId);
const targetsValid = targetExists(draft.primary) && targetExists(draft.reserve)
&& (draft.primary.profileId !== draft.reserve.profileId || draft.primary.serverId !== draft.reserve.serverId);
const draftValid = Boolean(targetsValid && draft.checks.length);
const quietElapsed = activity?.quietSince
? Math.max(0, Date.now() - Date.parse(activity.quietSince))
: 0;
const addService = async () => {
try {
const saved = await saveCustomDiagnosticService({
name: serviceName,
url: serviceUrl,
customServices: diagnostics.customServices,
hiddenServiceIds: diagnostics.hiddenServiceIds,
updateSettings: onUpdateDiagnostics,
});
if (!saved) return;
setServiceName('');
setServiceUrl('');
setServiceError('');
setAddingService(false);
} catch (error) {
setServiceError(error instanceof Error ? error.message : 'Проверьте адрес.');
}
};
return <Drawer
panelRef={feature.panelRef}
closeRef={feature.closeRef}
id="client-failover"
open={feature.isOpen}
label="Резервный канал"
closeLabel="Закрыть резервный канал"
onClose={feature.close}
className="client-failover-drawer"
>
<header className="client-failover-header">
<span>Gateway</span>
<h2>Резервный канал</h2>
<p>Переключает только новые соединения. Уже открытые соединения Harbor не закрывает.</p>
</header>
<form className="client-failover-form" onSubmit={(event) => {
event.preventDefault();
void onSave(draft).then((result) => { if (result !== false) setDirty(false); });
}}>
<label className="client-failover-master">
<span><strong>Использовать резерв</strong><small>{draft.enabled ? 'Мониторинг включён' : 'Полностью пассивен'}</small></span>
<input type="checkbox" checked={draft.enabled} onChange={(event) => update({ enabled: event.target.checked })} />
</label>
{(['primary', 'reserve'] as const).map((channel) => {
const profile = profiles.find(({ id }) => id === draft[channel].profileId);
const health = snapshot[channel].health;
const missing = Boolean(snapshot[channel].target.profileId) && !targetExists(snapshot[channel].target);
const healthLabel = missing
? 'Цель недоступна'
: channel === 'primary' && snapshot.reason === 'failure-window'
? `Нестабилен · ${seconds(Math.max(0, Date.now() - Date.parse(snapshot.primary.stateSince || new Date().toISOString())))} из ${seconds(draft.failureWindowMs)} с`
: channel === 'primary' && snapshot.currentRole === 'reserve' && health === 'healthy' && snapshot.reason === 'recovery-hold'
? 'Восстанавливается'
: health === 'healthy' ? 'Работает' : health === 'unhealthy' ? 'Недоступен' : health === 'not-monitoring' ? 'Не проверяется' : 'Нет данных';
return <section className="client-failover-channel" key={channel}>
<div className="client-failover-channel-title">
<span>{channel === 'primary' ? 'Основной' : 'Резервный'}</span>
<strong className={health === 'healthy' ? 'is-healthy' : health === 'unhealthy' || missing ? 'is-unhealthy' : ''}>{healthLabel}</strong>
</div>
<label><span>Подписка</span><select value={draft[channel].profileId} onChange={(event) => updateTarget(channel, { profileId: event.target.value })}>
<option value="">Не выбрана</option>
{profiles.map((item) => <option value={item.id} key={item.id}>{item.label}</option>)}
</select></label>
<label><span>Сервер</span><select value={draft[channel].serverId} onChange={(event) => updateTarget(channel, { serverId: event.target.value })} disabled={!profile}>
<option value="">Не выбран</option>
{profile?.servers.map((server) => <option value={server.id} key={server.id}>{server.label}</option>)}
</select></label>
</section>;
})}
<fieldset className="client-failover-services">
<legend>Что проверять</legend>
{services.map((service) => {
const check = draft.checks.find(({ serviceId }) => serviceId === service.id);
return <div className="client-failover-service" key={service.id}>
<label>
<input type="checkbox" checked={Boolean(check)} onChange={(event) => update({
checks: event.target.checked
? [...draft.checks, { serviceId: service.id, timeoutMs: 6_000 }]
: draft.checks.filter(({ serviceId }) => serviceId !== service.id),
})} />
<span>{service.label}</span>
</label>
{check && <label className="client-failover-service-timeout">
<span>таймаут, сек</span>
<input
type="number"
min="2"
max="30"
aria-label={`Таймаут проверки: ${service.label}`}
value={seconds(check.timeoutMs)}
onChange={(event) => update({ checks: draft.checks.map((item) => item.serviceId === service.id
? { ...item, timeoutMs: milliseconds(event.target.value, item.timeoutMs) }
: item) })}
/>
</label>}
</div>;
})}
{!addingService && <button type="button" className="client-failover-add-service" disabled={diagnostics.customServices.length >= MAX_CUSTOM_DIAGNOSTIC_SERVICES} onClick={() => setAddingService(true)}>+ Добавить HTTPS-сервис</button>}
{addingService && <div className="client-failover-service-editor">
<input aria-label="Название HTTPS-сервиса" placeholder="Название" value={serviceName} onChange={(event) => setServiceName(event.target.value)} />
<input aria-label="HTTPS-адрес сервиса" placeholder="https://example.com/health" value={serviceUrl} onChange={(event) => setServiceUrl(event.target.value)} />
{serviceError && <span role="alert">{serviceError}</span>}
<button type="button" onClick={() => void addService()}>Добавить</button>
<button type="button" onClick={() => { setAddingService(false); setServiceError(''); }}>Отмена</button>
</div>}
</fieldset>
<section className="client-failover-timing" aria-label="Пороги переключения">
<label><span>Проверять каждые, сек</span><input type="number" min="15" max="900" value={seconds(draft.intervalMs)} onChange={(event) => update({ intervalMs: milliseconds(event.target.value, draft.intervalMs) })} /></label>
<label><span>Сбой должен длиться, сек</span><input type="number" min={seconds(draft.intervalMs * 2)} max="1800" value={seconds(draft.failureWindowMs)} onChange={(event) => update({ failureWindowMs: milliseconds(event.target.value, draft.failureWindowMs) })} /></label>
<label><span>Восстановление, сек</span><input type="number" min="60" max="86400" value={seconds(draft.recoveryWindowMs)} onChange={(event) => update({ recoveryWindowMs: milliseconds(event.target.value, draft.recoveryWindowMs) })} /></label>
<label><span>Не переключать во время работы</span><input type="checkbox" checked={draft.trafficGuard.enabled} onChange={(event) => update({ trafficGuard: { ...draft.trafficGuard, enabled: event.target.checked } })} /></label>
<label><span>Тишина перед переключением, сек</span><input type="number" min="5" max="600" value={seconds(draft.trafficGuard.quietWindowMs)} onChange={(event) => update({ trafficGuard: { ...draft.trafficGuard, quietWindowMs: milliseconds(event.target.value, draft.trafficGuard.quietWindowMs) } })} /></label>
<label><span>Активный трафик, КБ/с</span><input type="number" min="1" max="102400" value={Math.round(draft.trafficGuard.thresholdBytesPerSecond / 1024)} onChange={(event) => update({ trafficGuard: { ...draft.trafficGuard, thresholdBytesPerSecond: Number(event.target.value) * 1024 } })} /></label>
</section>
<details className="client-failover-advanced"><summary>Защита от повторных сбоев</summary>
<label><span>Минимум на резерве, сек</span><input type="number" min="60" max="86400" value={seconds(draft.minimumReserveMs)} onChange={(event) => update({ minimumReserveMs: milliseconds(event.target.value, draft.minimumReserveMs) })} /></label>
<label><span>Падений до карантина</span><input type="number" min="2" max="10" value={draft.flapProtection.count} onChange={(event) => update({ flapProtection: { ...draft.flapProtection, count: Number(event.target.value) } })} /></label>
<label><span>Окно повторных сбоев, ч</span><input type="number" min="1" max="72" value={Math.round(draft.flapProtection.windowMs / 3_600_000)} onChange={(event) => update({ flapProtection: { ...draft.flapProtection, windowMs: Number(event.target.value) * 3_600_000 } })} /></label>
<label><span>Карантин основного, мин</span><input type="number" min="10" max="10080" value={Math.round(draft.flapProtection.quarantineMs / 60_000)} onChange={(event) => update({ flapProtection: { ...draft.flapProtection, quarantineMs: Number(event.target.value) * 60_000 } })} /></label>
</details>
<div className="client-failover-runtime">
<strong role="status" aria-live="polite">{blocked ? 'Harbor выполняет действие…' : snapshot.activation === 'pending' ? 'Включится при следующем запуске VPN' : reasonLabel(snapshot.reason)}</strong>
<span>Новые соединения: {role ? targetLabel(profiles, snapshot[role].target) : 'текущий канал вне настроенной пары'}</span>
{snapshot.nextDecisionAt && <span>Следующее решение не раньше чем через {seconds(Math.max(0, Date.parse(snapshot.nextDecisionAt) - Date.now()))} с</span>}
{snapshot.currentRole === 'other' && <span>Чтобы запустить автоматику, выключите VPN и включите снова на основном канале.</span>}
{activity && <span>{activity.state === 'active' ? `Активный трафик · ${Math.round(activity.totalBytesPerSecond / 1024)} КБ/с · соединений: ${activity.transmittingConnections}` : activity.state === 'quiet' ? `Проверяем тишину · ${Math.min(seconds(quietElapsed), seconds(draft.trafficGuard.quietWindowMs))} из ${seconds(draft.trafficGuard.quietWindowMs)} с` : 'Не удалось определить активность · автоматическое переключение остановлено'}</span>}
{activity?.blockers.slice(0, 2).map((blocker) => <small key={`${blocker.device}:${blocker.service}`}>{blocker.device} · {blocker.service} · {Math.round((blocker.uploadBytesPerSecond + blocker.downloadBytesPerSecond) / 1024)} КБ/с</small>)}
{activity && activity.blockers.length > 2 && <small>Ещё {activity.blockers.length - 2}</small>}
</div>
{!draftValid && <p className="client-failover-validation">Выберите два разных сервера. Они могут быть из одной подписки.</p>}
{confirmDiscard && <div className="client-failover-discard" role="alert">
<span>Отменить несохранённые изменения?</span>
<button type="button" onClick={() => { setDraft(snapshot.policy); setDirty(false); setConfirmDiscard(false); feature.beforeCloseRef.current = () => true; feature.close(); }}>Отменить изменения</button>
<button type="button" onClick={() => setConfirmDiscard(false)}>Продолжить настройку</button>
</div>}
<div className="client-failover-actions">
<button type="button" disabled={blocked || !snapshot.enabled || snapshot.activation !== 'active'} onClick={() => void onCheck()}>Проверить оба канала</button>
<button type="submit" disabled={blocked || !dirty || !draftValid}>Сохранить</button>
{snapshot.enabled && <button type="button" disabled={blocked} onClick={() => void onPause(!snapshot.paused)}>{snapshot.paused ? 'Возобновить' : 'Пауза'}</button>}
{switchRole && snapshot.enabled && snapshot.activation === 'active' && <button type="button" disabled={blocked} aria-label={`Переключить новые соединения на ${switchRole === 'reserve' ? 'резервный' : 'основной'} сейчас`} onClick={() => void onSwitch(switchRole)}>Переключить новые сейчас</button>}
{(snapshot.currentRole === 'primary' || snapshot.currentRole === 'reserve') && <small>Текущие соединения Harbor не закроет</small>}
</div>
</form>
</Drawer>;
}
+1
View File
@@ -0,0 +1 @@
export { FailoverPanel, FailoverToggle, useFailoverFeature } from './FailoverFeature.js';
+34 -1
View File
@@ -4,6 +4,10 @@ export type SyncErrorKind = 'incompatible-api' | 'control-unreachable' | 'fatal'
export interface HarborReducerState { export interface HarborReducerState {
snapshot: HarborClientState | null; snapshot: HarborClientState | null;
failoverTransport: {
activeEpoch: string;
retiredEpochs: string[];
};
transport: { transport: {
bootStatus: 'loading' | 'ready' | SyncErrorKind; bootStatus: 'loading' | 'ready' | SyncErrorKind;
lastSuccessfulSyncAt: string | null; lastSuccessfulSyncAt: string | null;
@@ -20,6 +24,7 @@ export type HarborAction =
export const initialHarborState: HarborReducerState = { export const initialHarborState: HarborReducerState = {
snapshot: null, snapshot: null,
failoverTransport: { activeEpoch: '', retiredEpochs: [] },
transport: { transport: {
bootStatus: 'loading', bootStatus: 'loading',
lastSuccessfulSyncAt: null, lastSuccessfulSyncAt: null,
@@ -31,6 +36,14 @@ export const initialHarborState: HarborReducerState = {
export const STALE_FAILURE_THRESHOLD = 3; export const STALE_FAILURE_THRESHOLD = 3;
function failoverEpoch(snapshot: HarborClientState | null) {
return snapshot?.failover?.observationEpoch || '';
}
function failoverSequence(snapshot: HarborClientState | null) {
return snapshot?.failover?.observationSequence || 0;
}
export function classifySyncError(error: unknown): SyncErrorKind { export function classifySyncError(error: unknown): SyncErrorKind {
const candidate = error && typeof error === 'object' ? error as Record<string, unknown> : {}; const candidate = error && typeof error === 'object' ? error as Record<string, unknown> : {};
const status = Number(candidate.status) || 0; const status = Number(candidate.status) || 0;
@@ -71,9 +84,29 @@ export function harborReducer(current: HarborReducerState, action: HarborAction)
const snapshot = action.snapshot; const snapshot = action.snapshot;
const newer = !current.snapshot || snapshot.revision > current.snapshot.revision; const newer = !current.snapshot || snapshot.revision > current.snapshot.revision;
const equal = Boolean(current.snapshot) && snapshot.revision === current.snapshot?.revision;
const incomingEpoch = failoverEpoch(snapshot);
const activeEpoch = current.failoverTransport.activeEpoch || failoverEpoch(current.snapshot);
const unseenEpoch = Boolean(incomingEpoch)
&& incomingEpoch !== activeEpoch
&& !current.failoverTransport.retiredEpochs.includes(incomingEpoch);
const newerObservation = equal && Boolean(snapshot.failover) && (
unseenEpoch
|| (incomingEpoch === activeEpoch && failoverSequence(snapshot) > failoverSequence(current.snapshot))
);
const nextSnapshot = newer
? snapshot
: newerObservation && current.snapshot
? { ...current.snapshot, failover: snapshot.failover }
: current.snapshot;
const nextActiveEpoch = newer || unseenEpoch ? incomingEpoch : activeEpoch;
const retiredEpochs = unseenEpoch && activeEpoch
? [...current.failoverTransport.retiredEpochs, activeEpoch]
: current.failoverTransport.retiredEpochs;
return { return {
snapshot: newer ? snapshot : current.snapshot, snapshot: nextSnapshot,
failoverTransport: { activeEpoch: nextActiveEpoch, retiredEpochs },
transport: { transport: {
bootStatus: 'ready', bootStatus: 'ready',
lastSuccessfulSyncAt: action.receivedAt, lastSuccessfulSyncAt: action.receivedAt,
+2 -1
View File
@@ -1,6 +1,6 @@
export type OperationKey = 'connection' | 'serverApply' | 'profileAdd' | 'profileRename' export type OperationKey = 'connection' | 'serverApply' | 'profileAdd' | 'profileRename'
| 'profileSelect' | 'profileActivate' | 'profileRefresh' | 'profileDelete' | 'profileSelect' | 'profileActivate' | 'profileRefresh' | 'profileDelete'
| 'gatewayAuto' | 'routeRules' | 'diagnosticsSettings'; | 'gatewayAuto' | 'routeRules' | 'diagnosticsSettings' | 'failover';
export interface OperationState { export interface OperationState {
status: 'running'; status: 'running';
@@ -22,6 +22,7 @@ const OPERATION_KEYS: readonly OperationKey[] = [
'gatewayAuto', 'gatewayAuto',
'routeRules', 'routeRules',
'diagnosticsSettings', 'diagnosticsSettings',
'failover',
]; ];
// The backend has one canonical revision and one mutation queue, so the UI mirrors that lock. // The backend has one canonical revision and one mutation queue, so the UI mirrors that lock.
@@ -0,0 +1,27 @@
.client-journal-header { display: grid; gap: 7px; margin: 0 8px 30px; }
.client-journal-header > span, .client-journal-groups h3 { color: var(--client-muted); font: var(--type-label); letter-spacing: var(--type-label-tracking); text-transform: var(--type-label-transform); }
.client-journal-header > div { display: flex; align-items: center; gap: 12px; }
.client-journal-header h2 { margin: 0; font: var(--type-drawer-title); letter-spacing: var(--type-drawer-title-tracking); text-transform: var(--type-drawer-title-transform); }
.client-journal-header button { width: 34px; height: 34px; display: grid; place-items: center; border: 0; background: transparent; color: var(--client-muted); cursor: pointer; }
.client-journal-header svg { width: 17px; fill: none; stroke: currentColor; stroke-width: 1.7; }
.client-journal-header button:hover, .client-journal-header button:focus-visible { color: var(--client-accent); outline: 0; filter: drop-shadow(0 0 7px var(--client-accent)); }
.client-journal-header button:disabled svg { animation: client-spin 900ms linear infinite; }
.client-journal-groups { display: grid; gap: 26px; margin: 0 8px; }
.client-journal-groups section { display: grid; gap: 7px; }
.client-journal-groups h3 { margin: 0; }
.client-journal-groups ol { display: grid; margin: 0; padding: 0; }
.client-journal-groups li { min-height: 58px; display: grid; grid-template-columns: 52px minmax(0, 1fr); gap: 14px; align-items: start; padding: 10px 0; border-top: 1px solid color-mix(in oklch, var(--client-border) 48%, transparent); }
.client-journal-time, .client-journal-groups li > div:last-child { min-width: 0; display: grid; gap: 3px; }
.client-journal-time time { color: var(--client-text); font: var(--type-data); font-variant-numeric: var(--numeric-tabular); letter-spacing: var(--type-data-tracking); text-transform: var(--type-data-transform); }
.client-journal-time span, .client-journal-groups li span { color: var(--client-muted); font: var(--type-label); letter-spacing: var(--type-label-tracking); text-transform: var(--type-label-transform); overflow-wrap: anywhere; }
.client-journal-time span { text-transform: var(--type-label-transform); }
.client-journal-groups li strong { font: var(--type-body); letter-spacing: var(--type-body-tracking); text-transform: var(--type-body-transform); }
.client-journal-groups li em { width: fit-content; color: oklch(0.68 0.15 28); font: var(--type-label); font-style: normal; letter-spacing: var(--type-label-tracking); text-transform: var(--type-label-transform); }
.client-journal-error, .client-journal-empty { min-height: 80px; margin: 0 8px; color: var(--client-muted); font: var(--type-body); letter-spacing: var(--type-body-tracking); text-transform: var(--type-body-transform); }
.client-journal-error button, .client-journal-footer button { border: 0; background: transparent; color: var(--client-accent); font: var(--type-control); letter-spacing: var(--type-control-tracking); text-transform: var(--type-control-transform); cursor: pointer; }
.client-journal-skeleton { display: grid; gap: 14px; margin: 0 8px; }
.client-journal-skeleton span { height: 48px; background: color-mix(in oklch, var(--client-border) 24%, transparent); opacity: .55; }
.client-journal-footer { min-height: 72px; display: grid; place-items: center; margin: 12px 8px 0; color: var(--client-muted); font: var(--type-label); letter-spacing: var(--type-label-tracking); text-transform: var(--type-label-transform); text-align: center; }
.client-journal-footer button:focus-visible, .client-journal-error button:focus-visible { outline: 2px solid var(--client-accent); outline-offset: 2px; }
@media (max-width: 560px) { .client-journal-groups li { grid-template-columns: 1fr; gap: 5px; } .client-journal-time { display: flex; gap: 8px; } }
@media (prefers-reduced-motion: reduce) { .client-journal-header button:disabled svg { animation: none; } }
+45
View File
@@ -0,0 +1,45 @@
.client-failover-header { display: grid; gap: 8px; margin: 0 8px 30px; }
.client-failover-header > span, .client-failover-channel-title > span, .client-failover-services legend { color: var(--client-muted); font: var(--type-label); letter-spacing: var(--type-label-tracking); text-transform: var(--type-label-transform); }
.client-failover-header h2 { margin: 0; font: var(--type-drawer-title); letter-spacing: var(--type-drawer-title-tracking); text-transform: var(--type-drawer-title-transform); }
.client-failover-header p { margin: 0; color: var(--client-muted); font: var(--type-body); letter-spacing: var(--type-body-tracking); text-transform: var(--type-body-transform); }
.client-failover-form { display: grid; gap: 24px; margin: 0 8px; }
.client-failover-master { min-height: 54px; display: flex; align-items: center; justify-content: space-between; gap: 20px; border-bottom: 1px solid color-mix(in oklch, var(--client-border) 48%, transparent); }
.client-failover-master > span { display: grid; gap: 3px; }
.client-failover-master strong { font: var(--type-body); letter-spacing: var(--type-body-tracking); text-transform: var(--type-body-transform); }
.client-failover-master small { color: var(--client-muted); font: var(--type-label); letter-spacing: var(--type-label-tracking); text-transform: var(--type-label-transform); }
.client-failover-master input { width: 42px; height: 22px; }
.client-failover-channel { display: grid; gap: 10px; }
.client-failover-channel-title { min-height: 28px; display: flex; align-items: center; justify-content: space-between; gap: 12px; }
.client-failover-channel-title strong { color: var(--client-muted); font: var(--type-label); letter-spacing: var(--type-label-tracking); text-transform: var(--type-label-transform); transition: color 600ms ease, filter 600ms ease; }
.client-failover-channel-title strong.is-healthy { color: var(--client-accent); }
.client-failover-channel-title strong.is-unhealthy { color: oklch(0.68 0.15 28); }
.client-failover-channel label, .client-failover-timing label, .client-failover-advanced label { display: grid; grid-template-columns: minmax(0, 1fr) minmax(150px, 48%); align-items: center; gap: 16px; min-height: 38px; color: var(--client-muted); font: var(--type-label); letter-spacing: var(--type-label-tracking); text-transform: var(--type-label-transform); }
.client-failover-channel select, .client-failover-timing input, .client-failover-advanced input { min-width: 0; min-height: 34px; border: 0; border-bottom: 1px solid var(--client-border); background: transparent; color: var(--client-text); font: var(--type-body); letter-spacing: var(--type-body-tracking); text-transform: var(--type-body-transform); }
.client-failover-services { display: grid; gap: 9px; margin: 0; padding: 0; border: 0; }
.client-failover-services legend { margin-bottom: 10px; }
.client-failover-service { display: grid; grid-template-columns: minmax(0, 1fr) auto; align-items: center; gap: 12px; min-height: 34px; }
.client-failover-service > label { display: flex; align-items: center; gap: 10px; font: var(--type-body); letter-spacing: var(--type-body-tracking); text-transform: var(--type-body-transform); }
.client-failover-service > .client-failover-service-timeout { color: var(--client-muted); font: var(--type-label); letter-spacing: var(--type-label-tracking); text-transform: var(--type-label-transform); }
.client-failover-service-timeout input { width: 52px; min-height: 30px; border: 0; border-bottom: 1px solid var(--client-border); background: transparent; color: var(--client-text); font: var(--type-body); letter-spacing: var(--type-body-tracking); text-transform: var(--type-body-transform); }
.client-failover-add-service { justify-self: start; padding: 7px 0; border: 0; background: transparent; color: var(--client-accent); font: var(--type-control); letter-spacing: var(--type-control-tracking); text-transform: var(--type-control-transform); cursor: pointer; }
.client-failover-add-service:disabled { color: var(--client-muted); cursor: default; opacity: .45; }
.client-failover-service-editor { display: grid; grid-template-columns: minmax(0, 1fr) minmax(0, 1.6fr); gap: 8px 12px; padding: 8px 0; }
.client-failover-service-editor input { min-width: 0; min-height: 34px; border: 0; border-bottom: 1px solid var(--client-border); background: transparent; color: var(--client-text); font: var(--type-body); letter-spacing: var(--type-body-tracking); text-transform: var(--type-body-transform); }
.client-failover-service-editor span { grid-column: 1 / -1; color: oklch(0.68 0.15 28); font: var(--type-label); letter-spacing: var(--type-label-tracking); text-transform: var(--type-label-transform); }
.client-failover-service-editor button { justify-self: start; padding: 4px 0; border: 0; background: transparent; color: var(--client-accent); font: var(--type-control); letter-spacing: var(--type-control-tracking); text-transform: var(--type-control-transform); cursor: pointer; }
.client-failover-timing { display: grid; gap: 5px; }
.client-failover-advanced summary { min-height: 38px; color: var(--client-accent); font: var(--type-control); letter-spacing: var(--type-control-tracking); text-transform: var(--type-control-transform); cursor: pointer; }
.client-failover-runtime { min-height: 132px; display: grid; align-content: center; gap: 5px; padding: 14px 0; border-top: 1px solid color-mix(in oklch, var(--client-border) 48%, transparent); border-bottom: 1px solid color-mix(in oklch, var(--client-border) 48%, transparent); font-variant-numeric: var(--numeric-tabular); }
.client-failover-runtime strong { font: var(--type-body); letter-spacing: var(--type-body-tracking); text-transform: var(--type-body-transform); }
.client-failover-runtime span, .client-failover-runtime small { color: var(--client-muted); font: var(--type-label); letter-spacing: var(--type-label-tracking); text-transform: var(--type-label-transform); }
.client-failover-validation { margin: -12px 0 0; color: oklch(0.68 0.15 28); font: var(--type-label); letter-spacing: var(--type-label-tracking); text-transform: var(--type-label-transform); }
.client-failover-discard { min-height: 74px; display: flex; flex-wrap: wrap; align-items: center; gap: 8px 16px; padding: 10px 0; border-top: 1px solid var(--client-border); border-bottom: 1px solid var(--client-border); }
.client-failover-discard span { flex-basis: 100%; color: var(--client-text); font: var(--type-body); letter-spacing: var(--type-body-tracking); text-transform: var(--type-body-transform); }
.client-failover-discard button { padding: 4px 0; border: 0; background: transparent; color: var(--client-accent); font: var(--type-control); letter-spacing: var(--type-control-tracking); text-transform: var(--type-control-transform); cursor: pointer; }
.client-failover-actions { min-height: 42px; display: flex; flex-wrap: wrap; gap: 18px; }
.client-failover-actions button { padding: 7px 0; border: 0; background: transparent; color: var(--client-accent); font: var(--type-control); letter-spacing: var(--type-control-tracking); text-transform: var(--type-control-transform); cursor: pointer; }
.client-failover-actions small { flex-basis: 100%; margin-top: -14px; color: var(--client-muted); font: var(--type-label); letter-spacing: var(--type-label-tracking); text-transform: var(--type-label-transform); }
.client-failover-actions button:disabled { color: var(--client-muted); cursor: default; opacity: .45; }
.client-failover-actions button:focus-visible, .client-failover-form input:focus-visible, .client-failover-form select:focus-visible, .client-failover-advanced summary:focus-visible, .client-failover-add-service:focus-visible, .client-failover-service-editor button:focus-visible, .client-failover-discard button:focus-visible { outline: 2px solid var(--client-accent); outline-offset: 2px; }
@media (max-width: 560px) { .client-failover-channel label, .client-failover-timing label, .client-failover-advanced label { grid-template-columns: 1fr; gap: 4px; } .client-failover-service, .client-failover-service-editor { grid-template-columns: 1fr; gap: 2px; } }
@media (prefers-reduced-motion: reduce) { .client-failover-channel-title strong { transition: none; } }
+2
View File
@@ -8,5 +8,7 @@
@import './features/servers.css'; @import './features/servers.css';
@import './primitives.css'; @import './primitives.css';
@import './features/diagnostics.css'; @import './features/diagnostics.css';
@import './features/failover.css';
@import './features/activity-journal.css';
@import './layout.css'; @import './layout.css';
@import './themes.css'; @import './themes.css';
@@ -0,0 +1,25 @@
import assert from 'node:assert/strict';
import test from 'node:test';
import { createActivityJournalRoute } from '../../dist/server/http/routes/activityJournalRoute.js';
test('journal route is read-only and forwards bounded cursor pagination', async () => {
const calls = [];
const route = createActivityJournalRoute({
journal: { page: (...args) => {
calls.push(args);
return { events: [], nextCursor: null, retentionDays: 30, generatedAt: 'now', storage: { status: 'ready', errorCode: null } };
} },
});
let status;
let payload;
const response = {
writeHead: (value) => { status = value; },
end: (value) => { payload = JSON.parse(value); },
};
assert.equal(await route.handle({ method: 'GET', url: '/api/activity-journal?limit=25&cursor=evt' }, response), true);
assert.equal(status, 200);
assert.equal(payload.retentionDays, 30);
assert.deepEqual(calls, [[25, 'evt']]);
assert.equal(await route.handle({ method: 'POST', url: '/api/activity-journal' }, response), false);
});
@@ -0,0 +1,145 @@
import assert from 'node:assert/strict';
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import test from 'node:test';
import { createActivityJournalService } from '../../dist/server/services/activityJournalService.js';
import { assertActivityJournalPage } from '../../dist/shared/activityJournal.js';
function fixture(t) {
const directory = fs.mkdtempSync(path.join(os.tmpdir(), 'harbor-journal-'));
t.after(() => fs.rmSync(directory, { recursive: true, force: true }));
return path.join(directory, 'activity-journal.json');
}
const event = (dedupeKey, profileLabel = 'Home') => ({
type: 'subscription.refreshed',
severity: 'info',
source: 'subscription',
dedupeKey: `subscription.refreshed:${dedupeKey}`,
data: { profileId: 'profile-1', profileLabel, host: 'provider.example', serverCount: 12, added: 2, removed: 1 },
});
test('journal appends typed events, deduplicates and keeps stable newest-first cursors', (t) => {
let clock = new Date('2026-08-19T10:00:00.000Z');
const filePath = fixture(t);
const service = createActivityJournalService({ filePath, now: () => clock });
service.append(event('refresh:1', 'One'));
clock = new Date('2026-08-19T10:01:00.000Z');
service.append(event('refresh:2', 'Two'));
service.append(event('refresh:2', 'Duplicate'));
const first = service.page(1);
assert.equal(first.events[0].data.profileLabel, 'Two');
assert.ok(first.nextCursor);
clock = new Date('2026-08-19T10:02:00.000Z');
service.append(event('refresh:3', 'Three'));
const older = service.page(10, first.nextCursor);
assert.deepEqual(older.events.map(({ data }) => data.profileLabel), ['One']);
assert.equal(first.events[0].dedupeKey, null);
const inode = fs.statSync(filePath).ino;
assert.equal(service.page(10).events.length, 3);
assert.equal(fs.statSync(filePath).ino, inode);
assert.deepEqual(service.page(10, 'expired-cursor').events, []);
});
test('journal prunes events older than 30 days and rejects unsafe payloads', (t) => {
let clock = new Date('2026-07-01T00:00:00.000Z');
const filePath = fixture(t);
const service = createActivityJournalService({ filePath, now: () => clock });
service.append(event('old'));
clock = new Date('2026-08-19T00:00:00.000Z');
assert.deepEqual(service.page().events, []);
assert.deepEqual(JSON.parse(fs.readFileSync(filePath, 'utf8')).events, []);
service.append(event('new'));
assert.throws(() => service.append({ ...event('unsafe'), data: { rawUrl: 'https://secret' } }), /Unsafe/);
service.append({ ...event('ip'), data: { ...event('ip').data, host: '192.168.1.1' } });
service.append(event('credential-label', 'https://user:pass@example.test/private?token=secret'));
service.append(event('path-label', '192.168.1.7/private'));
service.append(event('ipv6-label', '2001:db8::1'));
const page = service.page();
const labels = page.events.slice(0, 3).map(({ data }) => data.profileLabel);
assert.deepEqual(labels, ['Подписка', 'Подписка', 'Подписка']);
assert.equal(page.events.find(({ data }) => data.host === 'Провайдер')?.data.host, 'Провайдер');
for (const dedupeKey of [
'subscription.refreshed:192.168.1.1',
'subscription.refreshed:2001:db8::1',
'subscription.refreshed:user:pass',
]) service.append({ ...event('safe'), dedupeKey });
service.append({ ...event('safe'), dedupeKey: 'subscription.refreshed:user:pass' });
const persisted = fs.readFileSync(filePath, 'utf8');
assert.doesNotMatch(persisted, /user:pass|192\.168\.1\.1|192\.168\.1\.7|2001:db8|token=secret/);
assert.match(persisted, /subscription\.refreshed:sha256:[a-f0-9]{64}/);
assert.throws(() => service.append({ ...event('safe'), dedupeKey: 'https://user:pass@example.test/private?token=x' }), /dedupe/i);
assert.throws(() => service.append({ ...event('safe'), dedupeKey: 'subscription.refreshed:private/path' }), /dedupe/i);
});
test('journal persists the 10,000 event cap when opening an oversized store', (t) => {
const filePath = fixture(t);
const occurredAt = '2026-08-19T00:00:00.000Z';
const events = Array.from({ length: 10_001 }, (_, index) => ({
id: `00000000-0000-4000-8000-${String(index).padStart(12, '0')}`,
occurredAt,
...event(`event:${index}`),
}));
fs.writeFileSync(filePath, JSON.stringify({ schemaVersion: 1, events }));
const service = createActivityJournalService({
filePath,
now: () => new Date('2026-08-19T01:00:00.000Z'),
});
assert.equal(JSON.parse(fs.readFileSync(filePath, 'utf8')).events.length, 10_000);
assert.equal(service.page(1).events.length, 1);
});
test('corrupt journal is isolated and recovery becomes a safe event', (t) => {
const filePath = fixture(t);
fs.writeFileSync(filePath, '{broken');
const service = createActivityJournalService({
filePath,
now: () => new Date('2026-08-19T12:00:00.000Z'),
});
const page = service.page();
assert.equal(page.storage.status, 'ready');
assert.equal(page.events[0].type, 'journal.recovered');
assert.ok(fs.readdirSync(path.dirname(filePath)).some((name) => name.includes('.corrupt-')));
});
test('journal exposes a latched write failure until a later append succeeds', (t) => {
const filePath = fixture(t);
const directory = path.dirname(filePath);
const service = createActivityJournalService({ filePath });
service.append(event('before-error'));
fs.chmodSync(directory, 0o555);
try {
assert.throws(() => service.append(event('lost')));
const failed = service.page();
assert.equal(failed.storage.status, 'error');
assert.equal(failed.storage.errorCode, 'JOURNAL_UNAVAILABLE');
assert.equal(failed.events.length, 1);
assert.equal(failed.events[0].dedupeKey, null);
} finally {
fs.chmodSync(directory, 0o755);
}
service.append(event('recovered'));
assert.equal(service.page().storage.status, 'ready');
});
test('journal page parser rejects malformed wire data and strips unknown event fields', (t) => {
const service = createActivityJournalService({ filePath: fixture(t) });
service.append(event('wire'));
const page = service.page();
const parsed = assertActivityJournalPage({
...page,
events: page.events.map((item) => ({ ...item, ignored: 'value' })),
});
assert.equal(Object.hasOwn(parsed.events[0], 'ignored'), false);
assert.throws(() => assertActivityJournalPage({ ...page, retentionDays: 31 }), TypeError);
assert.throws(() => assertActivityJournalPage({ ...page, events: [{ broken: true }] }), TypeError);
const future = assertActivityJournalPage({
...page,
events: [{ ...page.events[0], type: 'future.safe_event', data: { raw: 'not exposed' } }],
});
assert.equal(future.events[0].type, 'unknown');
assert.deepEqual(future.events[0].data, {});
});
+193
View File
@@ -52,6 +52,7 @@ function createHarness(overrides = {}) {
let stateUpdates = 0; let stateUpdates = 0;
let startFailure = overrides.failStart || null; let startFailure = overrides.failStart || null;
const events = []; const events = [];
const journalEvents = [];
const serialize = (operation) => { const serialize = (operation) => {
const run = async () => { const run = async () => {
@@ -119,13 +120,16 @@ function createHarness(overrides = {}) {
restartCommand: () => captureRuntimeCommand(restart, { preMutationErrorCodes: ['CONFIG_INVALID'] }), restartCommand: () => captureRuntimeCommand(restart, { preMutationErrorCodes: ['CONFIG_INVALID'] }),
}, },
route: { isGatewayDirect: () => overrides.gatewayDirect === true }, route: { isGatewayDirect: () => overrides.gatewayDirect === true },
failover: overrides.failover,
serialize, serialize,
now: () => new Date('2026-08-08T12:00:00.000Z'), now: () => new Date('2026-08-08T12:00:00.000Z'),
onEvent: (event) => journalEvents.push(event),
}); });
return { return {
service, service,
events, events,
journalEvents,
serialize, serialize,
snapshot: () => structuredClone({ state, config, running, peak }), snapshot: () => structuredClone({ state, config, running, peak }),
}; };
@@ -167,6 +171,8 @@ test('failed candidate config or runtime restores the old profile, config and ru
const before = harness.snapshot(); const before = harness.snapshot();
await assert.rejects(harness.service.apply('work', 'b')); await assert.rejects(harness.service.apply('work', 'b'));
assert.deepEqual(domain(harness.snapshot()), domain(before)); assert.deepEqual(domain(harness.snapshot()), domain(before));
assert.equal(harness.journalEvents.at(-1).type, 'connection.failed');
assert.match(harness.journalEvents.at(-1).data.errorCode, /^[A-Z0-9_]+$/);
} }
}); });
@@ -256,6 +262,193 @@ test('running restart preserves a pending desired profile while restoring the ap
assert.equal(harness.snapshot().state.appliedProfileId, 'personal'); assert.equal(harness.snapshot().state.appliedProfileId, 'personal');
}); });
test('running dual restart rebuilds only the applied failover pair', async () => {
const state = initialState();
state.failoverPolicy = { enabled: true };
state.appliedFailoverPolicy = {
primary: { profileId: 'personal', serverId: 'a' },
reserve: { profileId: 'work', serverId: 'b' },
};
const sources = [];
const prepared = [];
const harness = createHarness({
state,
failover: {
build: (_state, source) => {
sources.push(source);
return {
config: { source },
applied: state.appliedFailoverPolicy,
primaryProfile: state.profiles[0],
primaryServer: serverA,
};
},
prepareActivation: async (role) => { prepared.push(role); harness.events.push(`selector.${role}`); },
restoreAppliedActivation: async () => {},
reconcile: async () => {},
},
});
await harness.service.restart();
assert.deepEqual(sources, ['applied']);
assert.deepEqual(prepared, ['primary']);
assert.deepEqual(harness.events.slice(0, 4), ['config.write', 'runtime.restart', 'selector.primary', 'state.update']);
assert.match(harness.snapshot().config, /"source":"applied"/);
});
test('running single-channel keeps server edits pending after failover is enabled', async () => {
const state = initialState();
state.failoverPolicy = { enabled: true };
state.appliedFailoverPolicy = null;
const harness = createHarness({ state });
await harness.service.apply('work', 'b');
assert.equal(harness.snapshot().state.desiredProfileId, 'work');
assert.equal(harness.snapshot().state.appliedProfileId, 'personal');
assert.equal(harness.events.includes('config.write'), false);
assert.equal(harness.events.includes('runtime.start'), false);
});
test('running dual restart preserves reserve and commits only after selector read-back', async () => {
const state = initialState();
state.failoverPolicy = { enabled: true };
state.appliedFailoverPolicy = {
primary: { profileId: 'personal', serverId: 'a' },
reserve: { profileId: 'work', serverId: 'b' },
};
state.appliedProfileId = 'work';
state.appliedServerId = 'b';
state.appliedServerSnapshot = serverB;
const harness = createHarness({
state,
failover: {
build: () => ({
config: { dual: true },
applied: state.appliedFailoverPolicy,
primaryProfile: state.profiles[0],
primaryServer: serverA,
}),
prepareActivation: async (role) => { harness.events.push(`selector.${role}`); },
restoreAppliedActivation: async () => { harness.events.push('selector.primary'); },
reconcile: async () => {},
},
});
await harness.service.restart();
assert.equal(harness.snapshot().state.appliedProfileId, 'work');
assert.equal(harness.snapshot().state.appliedServerId, 'b');
assert.deepEqual(harness.events.slice(0, 4), ['config.write', 'runtime.restart', 'selector.reserve', 'state.update']);
});
test('selector activation failure rolls runtime and config back before applied truth changes', async () => {
const state = initialState();
state.failoverPolicy = { enabled: true };
state.appliedFailoverPolicy = {
primary: { profileId: 'personal', serverId: 'a' },
reserve: { profileId: 'work', serverId: 'b' },
};
const harness = createHarness({
state,
failover: {
build: () => ({
config: { dual: true },
applied: state.appliedFailoverPolicy,
primaryProfile: state.profiles[0],
primaryServer: serverA,
}),
prepareActivation: async (role) => {
harness.events.push(`selector.${role}`);
if (harness.events.filter((event) => event.startsWith('selector.')).length === 1) {
throw new Error('selector failed');
}
},
restoreAppliedActivation: async () => { harness.events.push('selector.primary'); },
reconcile: async () => {},
},
});
const before = harness.snapshot();
await assert.rejects(harness.service.restart(), /selector failed/);
assert.deepEqual(domain(harness.snapshot()), domain(before));
assert.equal(harness.events.includes('state.update'), false);
assert.deepEqual(harness.events.filter((event) => event.startsWith('selector.')), ['selector.primary', 'selector.primary']);
});
test('state commit rollback restores the previously selected reserve role', async () => {
const state = initialState();
state.failoverPolicy = { enabled: true };
state.appliedFailoverPolicy = {
primary: { profileId: 'personal', serverId: 'a' },
reserve: { profileId: 'work', serverId: 'b' },
};
state.appliedProfileId = 'work';
state.appliedServerId = 'b';
state.appliedServerSnapshot = serverB;
const harness = createHarness({
state,
failStateAt: 1,
failover: {
build: () => ({
config: { dual: true },
applied: state.appliedFailoverPolicy,
primaryProfile: state.profiles[0],
primaryServer: serverA,
}),
prepareActivation: async (role) => { harness.events.push(`selector.${role}`); },
restoreAppliedActivation: async () => { harness.events.push('selector.reserve'); },
reconcile: async () => {},
},
});
const before = harness.snapshot();
await assert.rejects(harness.service.restart(), /state failed/);
assert.deepEqual(domain(harness.snapshot()), domain(before));
assert.deepEqual(harness.events.filter((event) => event.startsWith('selector.')), ['selector.reserve', 'selector.reserve']);
});
test('failed stop restores the selected reserve role before restoring state', async () => {
const state = initialState();
state.failoverPolicy = { enabled: true };
state.appliedFailoverPolicy = {
primary: { profileId: 'personal', serverId: 'a' },
reserve: { profileId: 'work', serverId: 'b' },
};
state.appliedProfileId = 'work';
state.appliedServerId = 'b';
state.appliedServerSnapshot = serverB;
const harness = createHarness({
state,
failStateAt: 1,
failover: {
restoreAppliedActivation: async () => { harness.events.push('selector.reserve'); },
reconcile: async () => {},
},
});
const before = harness.snapshot();
await assert.rejects(harness.service.stop(), /state failed/);
assert.deepEqual(domain(harness.snapshot()), domain(before));
assert.deepEqual(harness.events.slice(0, 4), ['runtime.stop', 'state.update', 'runtime.start', 'selector.reserve']);
});
test('disabled passive dual rollback restores the selected reserve role', async () => {
const state = initialState();
state.failoverPolicy = { enabled: false };
state.appliedFailoverPolicy = {
primary: { profileId: 'personal', serverId: 'a' },
reserve: { profileId: 'work', serverId: 'b' },
};
state.appliedProfileId = 'work';
state.appliedServerId = 'b';
state.appliedServerSnapshot = serverB;
const harness = createHarness({
state,
failStateAt: 1,
failover: {
restoreAppliedActivation: async () => { harness.events.push('selector.reserve'); },
reconcile: async () => {},
},
});
const before = harness.snapshot();
await assert.rejects(harness.service.restart(), /state failed/);
assert.deepEqual(domain(harness.snapshot()), domain(before));
assert.equal(harness.events.includes('selector.reserve'), true);
});
test('restart fails closed when runtime status is unknown', async () => { test('restart fails closed when runtime status is unknown', async () => {
const state = initialState(); const state = initialState();
state.desiredProfileId = 'work'; state.desiredProfileId = 'work';
@@ -506,3 +506,18 @@ test('a targeted custom row validates and samples only that service', async () =
assert.deepEqual(calls, ['https://second.example/', 'https://second.example/', 'https://second.example/']); assert.deepEqual(calls, ['https://second.example/', 'https://second.example/', 'https://second.example/']);
assert.equal(result.direct.sites[0].latencyMs, 120); assert.equal(result.direct.sites[0].latencyMs, 120);
}); });
test('failover VPN checks apply their per-service timeout to curl', async () => {
const calls = [];
await createConnectivityDiagnosticsService({
proxyPort: 18080,
execute: async (args) => { calls.push(args); return response(); },
}).runVpn({ target: 'site:youtube', timeoutMs: 9_000 });
assert.equal(calls.length, 3);
for (const args of calls) {
assert.equal(args[args.indexOf('--max-time') + 1], '9');
assert.equal(args[args.indexOf('--connect-timeout') + 1], '3');
assert.ok(args.includes('http://127.0.0.1:18080'));
}
});
+16
View File
@@ -32,6 +32,12 @@ test('control uses the dataplane socket protocol', async () => {
[{ id: 'custom-test', url: 'https://example.com' }], [{ id: 'custom-test', url: 'https://example.com' }],
'site:custom-test', 'site:custom-test',
); );
await client.checkConfig({ outbounds: [] });
await client.runFailoverProbe('primary', [], 'site:youtube', 9_000);
await client.readFailoverSelector();
await client.selectFailoverRole('reserve');
await client.setFailoverActivityEnabled(true);
await client.readFailoverActivity(1024);
assert.equal(client.running, true); assert.equal(client.running, true);
await client.restart(); await client.restart();
assert.equal((await client.stop()).running, false); assert.equal((await client.stop()).running, false);
@@ -44,6 +50,12 @@ test('control uses the dataplane socket protocol', async () => {
'GET /device-policy /run/dataplane.sock', 'GET /device-policy /run/dataplane.sock',
'PUT /device-policy /run/dataplane.sock', 'PUT /device-policy /run/dataplane.sock',
'POST /diagnostics/connectivity /run/dataplane.sock', 'POST /diagnostics/connectivity /run/dataplane.sock',
'POST /config/check /run/dataplane.sock',
'POST /failover/probe /run/dataplane.sock',
'GET /failover/selector /run/dataplane.sock',
'PUT /failover/selector /run/dataplane.sock',
'PUT /failover/activity /run/dataplane.sock',
'POST /failover/activity/read /run/dataplane.sock',
'POST /restart /run/dataplane.sock', 'POST /restart /run/dataplane.sock',
'POST /stop /run/dataplane.sock', 'POST /stop /run/dataplane.sock',
]); ]);
@@ -53,6 +65,10 @@ test('control uses the dataplane socket protocol', async () => {
target: 'site:custom-test', target: 'site:custom-test',
}); });
assert.equal(requests[7].timeoutMs, 25_000); assert.equal(requests[7].timeoutMs, 25_000);
assert.deepEqual(requests[8].body, { config: { outbounds: [] } });
assert.deepEqual(requests[9].body, { role: 'primary', services: [], target: 'site:youtube', timeoutMs: 9_000 });
assert.equal(requests[9].timeoutMs, 19_000);
assert.deepEqual(requests[11].body, { role: 'reserve' });
}); });
test('connectivity diagnostics expose a retryable domain error', async () => { test('connectivity diagnostics expose a retryable domain error', async () => {
+34 -1
View File
@@ -8,7 +8,7 @@ import { deviceId } from '../../dist/server/services/deviceInventoryService.js';
const mac = '00:11:22:33:44:55'; const mac = '00:11:22:33:44:55';
const id = deviceId(mac); const id = deviceId(mac);
const device = { ip: '192.168.50.7', mac }; const device = { ip: '192.168.50.7', mac, alias: 'MacBook' };
const connection = (connectionId, type, host, upload, download, sourceIP = device.ip, chains = ['vpn-out']) => ({ const connection = (connectionId, type, host, upload, download, sourceIP = device.ip, chains = ['vpn-out']) => ({
id: connectionId, id: connectionId,
metadata: { type, host, sourceIP }, metadata: { type, host, sourceIP },
@@ -205,3 +205,36 @@ test('domain classification normalizes known services and rejects IP or malforme
assert.equal(classifyDomain('192.0.2.1'), null); assert.equal(classifyDomain('192.0.2.1'), null);
assert.equal(classifyDomain('broken_label.example'), null); assert.equal(classifyDomain('broken_label.example'), null);
}); });
test('failover activity is zero-work while disabled and uses the existing connection poll', async () => {
let observedAt = new Date('2026-08-19T10:00:00.000Z');
let response = { connections: [
connection('work', 'tproxy/tproxy-in', 'r1.googlevideo.com', 0, 0, device.ip, ['channel-primary', 'channel-selector']),
connection('probe', 'mixed/diagnostics-primary-in', 'youtube.com', 0, 10_000, device.ip, ['channel-primary']),
] };
const service = createDomainTrafficService({
observe: async () => response,
devices: () => [device],
now: () => observedAt,
});
await service.refresh();
assert.equal(service.activitySnapshot(100), null);
service.enableActivity();
observedAt = new Date('2026-08-19T10:00:02.000Z');
response.connections[0].download = 2_000;
response.connections[1].download = 99_000;
await service.refresh();
const active = service.activitySnapshot(500);
assert.equal(active.state, 'active');
assert.equal(active.totalBytesPerSecond, 1_000);
assert.equal(active.transmittingConnections, 1);
assert.equal(active.blockers[0].device, 'MacBook');
assert.equal(active.blockers[0].service, 'YouTube');
observedAt = new Date('2026-08-19T10:00:14.000Z');
await service.refresh();
assert.equal(service.activitySnapshot(500).state, 'quiet');
service.disableActivity();
assert.equal(service.activitySnapshot(500), null);
});
+54
View File
@@ -0,0 +1,54 @@
import assert from 'node:assert/strict';
import test from 'node:test';
import { createFailoverRoute } from '../../dist/server/http/routes/failoverRoute.js';
function request(method, url = '/api/failover') {
return { method, url };
}
test('Gateway failover route serializes save, pause, checks and manual switch mutations', async () => {
const calls = [];
let body = { policy: { enabled: true }, expectedRevision: 7 };
const route = createFailoverRoute({
appMode: 'gateway',
failover: {
save: async (value) => calls.push(['save', value]),
pause: async (value) => calls.push(['pause', value]),
manualSwitch: async (value) => calls.push(['switch', value]),
checkNow: async () => calls.push(['check']),
},
readBody: async () => body,
withOperation: async (kind, operation, options) => {
calls.push(['operation', kind, options]);
return operation();
},
sendState: async () => calls.push(['state']),
});
assert.equal(await route.handle(request('PUT'), {}), true);
body = { paused: true, expectedRevision: 8 };
assert.equal(await route.handle(request('POST', '/api/failover/pause'), {}), true);
body = { role: 'reserve', expectedRevision: 9 };
assert.equal(await route.handle(request('POST', '/api/failover/switch'), {}), true);
body = { expectedRevision: 10 };
assert.equal(await route.handle(request('POST', '/api/failover/check'), {}), true);
assert.deepEqual(calls, [
['operation', 'failover-save', { expectedRevision: 7 }], ['save', { enabled: true }], ['state'],
['operation', 'failover-pause', { expectedRevision: 8 }], ['pause', true], ['state'],
['operation', 'failover-switch', { expectedRevision: 9 }], ['switch', 'reserve'], ['state'],
['check'], ['state'],
]);
});
test('failover mutations stay Gateway-only and reject unknown roles', async () => {
const connectRoute = createFailoverRoute({
appMode: 'client', failover: {}, readBody: async () => ({}), withOperation: async () => {}, sendState: async () => {},
});
await assert.rejects(connectRoute.handle(request('PUT'), {}), { code: 'ENDPOINT_NOT_FOUND' });
const gatewayRoute = createFailoverRoute({
appMode: 'gateway', failover: {}, readBody: async () => ({ role: 'other' }), withOperation: async () => {}, sendState: async () => {},
});
await assert.rejects(gatewayRoute.handle(request('POST', '/api/failover/switch'), {}), { code: 'REQUEST_INVALID' });
});
+635
View File
@@ -0,0 +1,635 @@
import assert from 'node:assert/strict';
import fs from 'node:fs';
import test from 'node:test';
import {
DEFAULT_FAILOVER_POLICY,
isFailoverConfigured,
nextFailoverDecision,
normalizeFailoverPolicy,
} from '../../dist/shared/failover.js';
import { createFailoverService } from '../../dist/server/features/failover/failoverService.js';
const enabled = normalizeFailoverPolicy({
...DEFAULT_FAILOVER_POLICY,
enabled: true,
primary: { profileId: 'profile-1', serverId: 'server-1' },
reserve: { profileId: 'profile-2', serverId: 'server-2' },
intervalMs: 15_000,
failureWindowMs: 30_000,
recoveryWindowMs: 60_000,
trafficGuard: { enabled: true, thresholdBytesPerSecond: 1024, quietWindowMs: 5_000 },
minimumReserveMs: 60_000,
});
test('failover policy defaults to strict disabled zero-work state', () => {
const policy = normalizeFailoverPolicy(null);
const decision = nextFailoverDecision({
now: 0,
policy,
currentRole: 'primary',
primaryHealth: 'unhealthy',
reserveHealth: 'healthy',
activity: 'quiet',
});
assert.equal(policy.enabled, false);
assert.equal(decision.status, 'idle');
assert.equal(decision.switchTo, null);
});
test('failover save rejects policy values that would otherwise be silently clamped', async () => {
const testHarness = harness(serviceState());
await assert.rejects(testHarness.service.save({ ...enabled, intervalMs: -1 }), { code: 'REQUEST_INVALID' });
});
test('strict failover validation accepts semantically identical reordered JSON keys', () => {
const reordered = {
flapProtection: enabled.flapProtection,
minimumReserveMs: enabled.minimumReserveMs,
trafficGuard: enabled.trafficGuard,
recoveryWindowMs: enabled.recoveryWindowMs,
failureWindowMs: enabled.failureWindowMs,
intervalMs: enabled.intervalMs,
checks: enabled.checks,
reserve: enabled.reserve,
primary: enabled.primary,
paused: enabled.paused,
enabled: enabled.enabled,
version: enabled.version,
};
assert.deepEqual(normalizeFailoverPolicy(reordered, { strict: true }), enabled);
});
test('enabled failover requires two distinct existing targets', async () => {
const sameTarget = normalizeFailoverPolicy({ ...enabled, reserve: enabled.primary });
assert.equal(isFailoverConfigured(sameTarget), false);
await assert.rejects(harness(serviceState()).service.save(sameTarget), { code: 'REQUEST_INVALID' });
const state = serviceState({ ...enabled, paused: true });
state.profiles[1].servers = [];
const testHarness = harness(state);
await assert.rejects(testHarness.service.pause(false), { code: 'REQUEST_INVALID' });
assert.equal(testHarness.read().failoverPolicy.paused, true);
});
test('local Gateway activity uses the existing domain traffic collector only while enabled', () => {
const source = fs.readFileSync(new URL('../../src/server/index.ts', import.meta.url), 'utf8');
assert.match(source, /const localFailoverTraffic = !remoteDataplane && settings\.appMode === 'gateway'[\s\S]*createDomainTrafficService/);
assert.match(source, /setLocalFailoverActivityEnabled[\s\S]*enableActivity\(\)[\s\S]*setInterval\(refresh, 2_000\)/);
assert.match(source, /if \(!enabled\)[\s\S]*clearInterval\(localFailoverTrafficTimer\)[\s\S]*disableActivity\(\)/);
assert.doesNotMatch(source, /setFailoverActivityEnabled: async \(\) => \(\{\}\)/);
});
test('failure window and traffic guard delay reserve switch until continuous quiet', () => {
let decision = nextFailoverDecision({
now: 0,
policy: enabled,
currentRole: 'primary',
primaryHealth: 'unhealthy',
reserveHealth: 'healthy',
activity: 'active',
});
decision = nextFailoverDecision({
now: 30_000,
policy: enabled,
currentRole: 'primary',
primaryHealth: 'unhealthy',
reserveHealth: 'healthy',
activity: 'active',
memory: decision.memory,
});
assert.equal(decision.status, 'waiting-for-idle');
decision = nextFailoverDecision({
now: 31_000,
policy: enabled,
currentRole: 'primary',
primaryHealth: 'unhealthy',
reserveHealth: 'healthy',
activity: 'quiet',
memory: decision.memory,
});
assert.equal(decision.switchTo, null);
decision = nextFailoverDecision({
now: 36_000,
policy: enabled,
currentRole: 'primary',
primaryHealth: 'unhealthy',
reserveHealth: 'healthy',
activity: 'quiet',
memory: decision.memory,
});
assert.equal(decision.switchTo, 'reserve');
});
test('both unhealthy and unknown activity never cause a switch', () => {
const both = nextFailoverDecision({
now: 30_000,
policy: enabled,
currentRole: 'primary',
primaryHealth: 'unhealthy',
reserveHealth: 'unhealthy',
activity: 'quiet',
});
assert.equal(both.reason, 'both-unhealthy');
assert.equal(both.switchTo, null);
const unknown = nextFailoverDecision({
now: 30_000,
policy: enabled,
currentRole: 'primary',
primaryHealth: 'unhealthy',
reserveHealth: 'healthy',
activity: 'unknown',
memory: { primaryFailedSince: 0, primaryRecoveredSince: null, quietSince: null },
});
assert.equal(unknown.reason, 'activity-unknown');
assert.equal(unknown.switchTo, null);
});
test('failback waits for recovery, hold and quarantine deadlines', () => {
let decision = nextFailoverDecision({
now: 10_000,
policy: enabled,
currentRole: 'reserve',
primaryHealth: 'healthy',
reserveHealth: 'healthy',
activity: 'quiet',
holdUntil: 50_000,
primaryQuarantineUntil: 70_000,
});
assert.equal(decision.nextDecisionAt, 70_000);
decision = nextFailoverDecision({
now: 70_000,
policy: enabled,
currentRole: 'reserve',
primaryHealth: 'healthy',
reserveHealth: 'healthy',
activity: 'quiet',
holdUntil: 50_000,
primaryQuarantineUntil: 70_000,
memory: decision.memory,
});
assert.equal(decision.switchTo, null);
assert.equal(decision.nextDecisionAt, 130_000);
decision = nextFailoverDecision({
now: 130_000,
policy: enabled,
currentRole: 'reserve',
primaryHealth: 'healthy',
reserveHealth: 'healthy',
activity: 'quiet',
holdUntil: 50_000,
primaryQuarantineUntil: 70_000,
memory: { ...decision.memory, quietSince: 125_000 },
});
assert.equal(decision.switchTo, 'primary');
const deadReserve = nextFailoverDecision({
now: 70_000,
policy: enabled,
currentRole: 'reserve',
primaryHealth: 'healthy',
reserveHealth: 'unhealthy',
activity: 'quiet',
holdUntil: 500_000,
primaryQuarantineUntil: 700_000,
memory: { primaryFailedSince: null, primaryRecoveredSince: 10_000, quietSince: 65_000 },
});
assert.equal(deadReserve.switchTo, null);
assert.equal(deadReserve.nextDecisionAt, 700_000);
});
function serviceState(policy = enabled) {
return {
revision: 1,
failoverPolicy: policy,
failoverRuntimeState: { lastSwitchAt: null, holdUntil: null, primaryQuarantineUntil: null, failoverHistory: [], reasonCode: null },
appliedFailoverPolicy: {
primary: { profileId: 'profile-1', serverId: 'server-1' },
reserve: { profileId: 'profile-2', serverId: 'server-2' },
primaryConfigFingerprint: 'a'.repeat(64),
reserveConfigFingerprint: 'b'.repeat(64),
},
appliedProfileId: 'profile-1',
appliedServerId: 'server-1',
profiles: [
{ id: 'profile-1', servers: [{ id: 'server-1', label: 'Primary' }] },
{ id: 'profile-2', servers: [{ id: 'server-2', label: 'Reserve' }] },
],
diagnostics: { customServices: [] },
};
}
function harness(initial = serviceState(), overrides = {}) {
let state = structuredClone(initial);
const calls = [];
const dependencies = {
state: {
read: () => state,
update: (mutator) => {
state = { ...mutator(state), revision: state.revision + 1 };
calls.push('state:update');
return state;
},
},
runtime: { isRunning: async () => true },
dataplane: {
checkConfig: async () => ({}),
runFailoverProbe: async () => ({ vpn: { sites: [{ status: 'available' }] } }),
readFailoverSelector: async () => ({ role: 'primary' }),
selectFailoverRole: async (role) => { calls.push(`select:${role}`); return { role }; },
setFailoverActivityEnabled: async (value) => { calls.push(`activity:${value}`); },
readFailoverActivity: async () => ({ activity: { state: 'quiet', observedAt: new Date().toISOString() } }),
},
buildCandidate: () => ({ config: {}, applied: initial.appliedFailoverPolicy }),
serialize: (operation) => operation(),
scheduler: {
setTimeout: () => ({ unref() {} }),
clearTimeout: () => {},
},
...overrides,
};
return { service: createFailoverService(dependencies), calls, read: () => state };
}
test('disabled failover performs one restart-safe collector cleanup and no steady-state work', async () => {
const state = serviceState(normalizeFailoverPolicy(null));
state.appliedFailoverPolicy = null;
const testHarness = harness(state);
await testHarness.service.reconcile();
assert.deepEqual(testHarness.calls, ['activity:false']);
await testHarness.service.reconcile();
assert.deepEqual(testHarness.calls, ['activity:false']);
assert.equal(testHarness.service.snapshot().activation, 'inactive');
});
test('disabled failover reports an already loaded dual config without managing it', async () => {
const testHarness = harness(serviceState(normalizeFailoverPolicy(null)));
await testHarness.service.reconcile();
assert.deepEqual(testHarness.calls, ['activity:false']);
assert.equal(testHarness.service.snapshot().activation, 'passive-loaded');
assert.equal(testHarness.service.snapshot().currentRole, 'primary');
});
test('runtime rollback restores the role from applied truth even while disabled', async () => {
const state = serviceState(normalizeFailoverPolicy(null));
state.appliedProfileId = state.appliedFailoverPolicy.reserve.profileId;
state.appliedServerId = state.appliedFailoverPolicy.reserve.serverId;
const testHarness = harness(state);
await testHarness.service.restoreAppliedActivation(state);
assert.deepEqual(testHarness.calls, ['select:reserve']);
});
test('an in-flight observation is discarded after failover is disabled', async () => {
const pending = [];
const testHarness = harness(serviceState(), {
dataplane: {
checkConfig: async () => ({}),
runFailoverProbe: (role) => new Promise((resolve) => pending.push({ role, resolve })),
readFailoverSelector: async () => ({ role: 'primary' }),
selectFailoverRole: async (role) => { testHarness.calls.push(`select:${role}`); return { role }; },
setFailoverActivityEnabled: async (value) => { testHarness.calls.push(`activity:${value}`); },
readFailoverActivity: async () => ({ activity: { state: 'quiet', observedAt: new Date().toISOString() } }),
},
});
await testHarness.service.reconcile();
const round = testHarness.service.runRound();
await new Promise((resolve) => setImmediate(resolve));
await testHarness.service.save({ ...enabled, enabled: false });
for (const probe of pending) probe.resolve({ vpn: { sites: [{ status: probe.role === 'reserve' ? 'available' : 'unavailable' }] } });
await round;
assert.equal(testHarness.read().failoverPolicy.enabled, false);
assert.equal(testHarness.calls.some((call) => call.startsWith('select:reserve')), false);
assert.deepEqual(testHarness.calls.filter((call) => call.startsWith('activity:')), ['activity:true', 'activity:true', 'activity:false']);
});
test('one channel probe failure does not erase the other channel health', async () => {
const testHarness = harness(serviceState(), {
dataplane: {
checkConfig: async () => ({}),
runFailoverProbe: async (role) => {
if (role === 'primary') throw new Error('primary probe transport failed');
return { vpn: { sites: [{ status: 'available' }] } };
},
readFailoverSelector: async () => ({ role: 'primary' }),
selectFailoverRole: async (role) => ({ role }),
setFailoverActivityEnabled: async () => {},
readFailoverActivity: async () => ({ activity: { state: 'active', observedAt: new Date().toISOString() } }),
},
});
await testHarness.service.checkNow();
assert.equal(testHarness.service.snapshot().primary.health, 'unknown');
assert.equal(testHarness.service.snapshot().reserve.health, 'healthy');
});
test('monitoring wakes at an earlier decision deadline instead of waiting a full interval', async () => {
let clock = 0;
const delays = [];
const testHarness = harness(serviceState(), {
now: () => new Date(clock),
scheduler: {
setTimeout: (_callback, delay) => { delays.push(delay); return { unref() {} }; },
clearTimeout: () => {},
},
dataplane: {
checkConfig: async () => ({}),
runFailoverProbe: async (role) => ({ vpn: { sites: [{ status: role === 'primary' ? 'unavailable' : 'available' }] } }),
readFailoverSelector: async () => ({ role: 'primary' }),
selectFailoverRole: async (role) => ({ role }),
setFailoverActivityEnabled: async () => {},
readFailoverActivity: async () => ({ activity: { state: 'quiet', observedAt: new Date(clock).toISOString() } }),
},
});
await testHarness.service.reconcile();
await testHarness.service.runRound();
clock = 30_000;
await testHarness.service.runRound();
assert.equal(delays.at(-1), enabled.trafficGuard.quietWindowMs);
});
test('a late activity response cannot switch after failover is disabled', async () => {
let resolveActivity;
const activity = new Promise((resolve) => { resolveActivity = resolve; });
const testHarness = harness(serviceState(), {
dataplane: {
checkConfig: async () => ({}),
runFailoverProbe: async (role) => ({ vpn: { sites: [{ status: role === 'primary' ? 'unavailable' : 'available' }] } }),
readFailoverSelector: async () => ({ role: 'primary' }),
selectFailoverRole: async (role) => { testHarness.calls.push(`select:${role}`); return { role }; },
setFailoverActivityEnabled: async (value) => { testHarness.calls.push(`activity:${value}`); },
readFailoverActivity: async () => activity,
},
});
await testHarness.service.reconcile();
const round = testHarness.service.runRound();
await new Promise(setImmediate);
await testHarness.service.save({ ...enabled, enabled: false });
resolveActivity({ activity: { state: 'quiet', observedAt: new Date().toISOString() } });
await round;
assert.equal(testHarness.calls.some((value) => value.startsWith('select:')), false);
});
test('automatic switch revalidates activity immediately before selector mutation', async () => {
let clock = 0;
let activityReads = 0;
const testHarness = harness(serviceState(), {
now: () => new Date(clock),
dataplane: {
checkConfig: async () => ({}),
runFailoverProbe: async (role) => ({ vpn: { sites: [{ status: role === 'primary' ? 'unavailable' : 'available' }] } }),
readFailoverSelector: async () => ({ role: 'primary' }),
selectFailoverRole: async (role) => { testHarness.calls.push(`select:${role}`); return { role }; },
setFailoverActivityEnabled: async () => {},
readFailoverActivity: async () => ({
activity: {
state: ++activityReads >= 4 ? 'active' : 'quiet',
observedAt: new Date(clock).toISOString(),
},
}),
},
});
await testHarness.service.reconcile();
await testHarness.service.runRound();
clock = 30_000;
await testHarness.service.runRound();
clock = 35_000;
await testHarness.service.runRound();
assert.equal(testHarness.calls.some((value) => value.startsWith('select:')), false);
assert.equal(testHarness.service.snapshot().reason, 'revalidation-required');
});
test('selector read-back failure rolls an uncertain switch back to the canonical role', async () => {
const testHarness = harness(serviceState(), {
dataplane: {
checkConfig: async () => ({}),
runFailoverProbe: async () => ({}),
readFailoverSelector: async () => ({ role: 'primary' }),
selectFailoverRole: async (role) => {
testHarness.calls.push(`select:${role}`);
if (role === 'reserve') throw new Error('read-back failed');
return { role };
},
setFailoverActivityEnabled: async () => {},
readFailoverActivity: async () => ({ activity: null }),
},
});
await assert.rejects(testHarness.service.manualSwitch('reserve'), /read-back failed/);
assert.deepEqual(testHarness.calls, ['select:reserve', 'select:primary']);
assert.equal(testHarness.read().appliedServerId, 'server-1');
});
test('failed collector disable is retried because local state remains unknown', async () => {
let attempts = 0;
const testHarness = harness(serviceState(normalizeFailoverPolicy(null)), {
dataplane: {
checkConfig: async () => ({}),
runFailoverProbe: async () => ({}),
readFailoverSelector: async () => ({ role: 'primary' }),
selectFailoverRole: async () => ({}),
setFailoverActivityEnabled: async () => {
attempts += 1;
if (attempts === 1) throw new Error('dataplane unavailable');
},
readFailoverActivity: async () => ({ activity: null }),
},
});
await assert.rejects(testHarness.service.reconcile(), /dataplane unavailable/);
await testHarness.service.reconcile();
assert.equal(attempts, 2);
});
test('each monitoring round repairs selector and collector state after a dataplane restart', async () => {
const state = serviceState();
state.appliedProfileId = 'profile-2';
state.appliedServerId = 'server-2';
let selected = 'reserve';
let enables = 0;
const testHarness = harness(state, {
dataplane: {
checkConfig: async () => ({}),
runFailoverProbe: async () => ({ vpn: { sites: [{ status: 'available' }] } }),
readFailoverSelector: async () => ({ role: selected }),
selectFailoverRole: async (role) => { selected = role; return { role }; },
setFailoverActivityEnabled: async (value) => { if (value) enables += 1; },
readFailoverActivity: async () => ({ activity: { state: 'quiet', observedAt: new Date().toISOString() } }),
},
});
await testHarness.service.reconcile();
selected = 'primary';
await testHarness.service.runRound();
assert.equal(selected, 'reserve');
assert.equal(enables, 2);
});
test('selector acknowledgement precedes state commit and a failed commit rolls selector back', async () => {
const state = serviceState();
const calls = [];
const testHarness = harness(state, {
state: {
read: () => state,
update: (mutator) => {
const candidate = mutator(state);
calls.push('state:update');
if (candidate.appliedServerId === 'server-2') throw new Error('write failed');
Object.assign(state, candidate);
return state;
},
},
dataplane: {
checkConfig: async () => ({}),
runFailoverProbe: async () => ({}),
readFailoverSelector: async () => ({ role: 'primary' }),
selectFailoverRole: async (role) => { calls.push(`select:${role}`); return { role }; },
setFailoverActivityEnabled: async () => {},
readFailoverActivity: async () => ({ activity: null }),
},
});
await assert.rejects(testHarness.service.manualSwitch('reserve'), /write failed/);
assert.deepEqual(calls, ['select:reserve', 'state:update', 'select:primary']);
assert.equal(state.appliedServerId, 'server-1');
});
test('manual selector and pause publish in one canonical commit', async () => {
const testHarness = harness(serviceState());
await testHarness.service.manualSwitch('reserve');
assert.equal(testHarness.read().appliedServerId, 'server-2');
assert.equal(testHarness.read().failoverPolicy.paused, true);
assert.equal(testHarness.calls.filter((call) => call === 'state:update').length, 1);
});
test('post-commit monitoring failure does not report a committed pause as failed', async () => {
const warnings = [];
const testHarness = harness(serviceState(), {
runtime: { isRunning: async () => { throw new Error('runtime unavailable'); } },
onWarning: (error) => warnings.push(error.message),
});
await testHarness.service.pause(true);
assert.equal(testHarness.read().failoverPolicy.paused, true);
assert.equal(testHarness.service.snapshot().status, 'error');
assert.deepEqual(warnings, ['runtime unavailable']);
});
test('pause and resume discard stale failure-window evidence', async () => {
let clock = 1_000;
const testHarness = harness(serviceState(), {
now: () => new Date(clock),
dataplane: {
checkConfig: async () => ({}),
runFailoverProbe: async (role) => ({
vpn: { sites: [{ status: role === 'primary' ? 'unavailable' : 'available' }] },
}),
readFailoverSelector: async () => ({ role: 'primary' }),
selectFailoverRole: async (role) => { testHarness.calls.push(`select:${role}`); return { role }; },
setFailoverActivityEnabled: async (value) => { testHarness.calls.push(`activity:${value}`); },
readFailoverActivity: async () => ({
activity: { state: 'quiet', observedAt: new Date(clock).toISOString(), quietSince: new Date(clock - 10_000).toISOString() },
}),
},
});
await testHarness.service.runRound();
await testHarness.service.pause(true);
clock = 3_600_000;
await testHarness.service.pause(false);
await testHarness.service.runRound();
assert.equal(testHarness.calls.includes('select:reserve'), false);
assert.equal(testHarness.service.snapshot().reason, 'failure-window');
});
test('an unconfirmed selector rollback durably pauses automation', async () => {
const testHarness = harness(serviceState(), {
dataplane: {
checkConfig: async () => ({}),
runFailoverProbe: async () => ({}),
readFailoverSelector: async () => ({ role: 'primary' }),
selectFailoverRole: async () => { throw new Error('selector unavailable'); },
setFailoverActivityEnabled: async () => {},
readFailoverActivity: async () => ({ activity: null }),
},
});
await assert.rejects(testHarness.service.manualSwitch('reserve'), AggregateError);
assert.equal(testHarness.read().failoverPolicy.paused, true);
assert.equal(testHarness.read().failoverRuntimeState.reasonCode, 'selector-unknown');
assert.equal(testHarness.service.snapshot().reason, 'selector-unknown');
assert.equal(testHarness.service.snapshot().currentRole, 'other');
});
test('successful selector reconciliation clears an unknown-role latch', async () => {
const state = serviceState();
state.failoverRuntimeState.reasonCode = 'selector-unknown';
const testHarness = harness(state);
await testHarness.service.reconcile();
assert.equal(testHarness.read().failoverRuntimeState.reasonCode, null);
assert.equal(testHarness.service.snapshot().currentRole, 'primary');
});
test('disabling on reserve preserves it as the next ordinary single-channel target', async () => {
const state = serviceState();
state.appliedProfileId = 'profile-2';
state.appliedServerId = 'server-2';
const testHarness = harness(state);
await testHarness.service.save({ ...enabled, enabled: false });
assert.equal(testHarness.read().desiredProfileId, 'profile-2');
assert.equal(testHarness.read().profiles[1].desiredServerId, 'server-2');
assert.equal(testHarness.service.snapshot().activation, 'passive-loaded');
});
test('significant health states append once without logging every observation', async () => {
let clock = 0;
const health = { primary: 'unavailable', reserve: 'available' };
const events = [];
const testHarness = harness(serviceState(), {
now: () => new Date(clock),
onEvent: (event) => events.push(event),
dataplane: {
checkConfig: async () => ({}),
runFailoverProbe: async (role) => ({ vpn: { sites: [{ status: health[role] }] } }),
readFailoverSelector: async () => ({ role: 'primary' }),
selectFailoverRole: async (role) => ({ role }),
setFailoverActivityEnabled: async () => {},
readFailoverActivity: async () => ({ activity: { state: 'quiet', observedAt: new Date(clock).toISOString() } }),
},
});
await testHarness.service.reconcile();
await testHarness.service.runRound();
clock = 30_000;
await testHarness.service.runRound();
health.reserve = 'unavailable';
clock = 31_000;
await testHarness.service.runRound();
clock = 32_000;
await testHarness.service.runRound();
health.primary = 'available';
clock = 33_000;
await testHarness.service.runRound();
assert.deepEqual(events.map(({ type }) => type), [
'failover.waiting_for_idle',
'failover.both_unhealthy',
'failover.recovered',
]);
});
test('pending target edits keep selector commits bound to the loaded channel pair', async () => {
const state = serviceState();
state.failoverPolicy = normalizeFailoverPolicy({
...state.failoverPolicy,
reserve: { profileId: 'profile-3', serverId: 'server-3' },
});
state.profiles.push({ id: 'profile-3', servers: [{ id: 'server-3', label: 'Desired later' }] });
const testHarness = harness(state, {
buildCandidate: () => ({
config: {},
applied: {
...state.appliedFailoverPolicy,
reserve: { profileId: 'profile-3', serverId: 'server-3' },
reserveConfigFingerprint: 'c'.repeat(64),
},
}),
});
await testHarness.service.manualSwitch('reserve');
assert.equal(testHarness.read().appliedProfileId, 'profile-2');
assert.equal(testHarness.read().appliedServerId, 'server-2');
assert.equal(testHarness.service.snapshot().activation, 'pending');
assert.deepEqual(testHarness.service.snapshot().reserve.target, { profileId: 'profile-2', serverId: 'server-2' });
});
+22
View File
@@ -109,6 +109,9 @@ function createHarness(overrides = {}) {
events.push('operation'); events.push('operation');
return operation(); return operation();
}, },
restoreAppliedActivation: overrides.restoreAppliedActivation
? async (previousState) => overrides.restoreAppliedActivation(previousState, events)
: undefined,
}); });
return { return {
@@ -246,6 +249,24 @@ test('route rules rollback continues and classifies runtime restore failure', as
assert.ok(broken.events.filter((event) => event === 'state.update').length >= 1); assert.ok(broken.events.filter((event) => event === 'state.update').length >= 1);
}); });
test('route rules rollback restores the previous reserve selector', async () => {
const state = canonicalState();
state.appliedFailoverPolicy = {
primary: { profileId: 'primary', serverId: 'other' },
reserve: { profileId: 'primary', serverId: 'server' },
};
const harness = createHarness({
state,
failures: { stateUpdates: [{ after: new Error('state') }] },
restoreAppliedActivation: async (previousState, events) => {
assert.equal(previousState.appliedServerId, 'server');
events.push('selector.reserve');
},
});
await assert.rejects(harness.service.update(newRules, 2, 2), /state/);
assert.ok(harness.events.indexOf('selector.reserve') > harness.events.indexOf('runtime.restore'));
});
test('route rules route preserves one adapter and state-only response', async () => { test('route rules route preserves one adapter and state-only response', async () => {
const calls = []; const calls = [];
const route = createRouteRulesRoute({ const route = createRouteRulesRoute({
@@ -260,5 +281,6 @@ test('route rules route preserves one adapter and state-only response', async ()
const source = readFileSync(new URL('../../src/server/index.ts', import.meta.url), 'utf8'); const source = readFileSync(new URL('../../src/server/index.ts', import.meta.url), 'utf8');
assert.match(source, /createRouteRulesRoute\(\{/); assert.match(source, /createRouteRulesRoute\(\{/);
assert.match(source, /state\.appliedFailoverPolicy[\s\S]*buildFailoverCandidate\(\{ \.\.\.state, routeRules \}, 'applied'\)\.config/);
assert.doesNotMatch(source, /function applyRouteRules|req\.url === ['"]\/api\/route-rules['"]/); assert.doesNotMatch(source, /function applyRouteRules|req\.url === ['"]\/api\/route-rules['"]/);
}); });
+66 -1
View File
@@ -8,7 +8,13 @@ process.env.APP_MODE = 'gateway';
process.env.DATA_DIR = fs.mkdtempSync(path.join(os.tmpdir(), 'vpn-proxy-gateway-test-')); process.env.DATA_DIR = fs.mkdtempSync(path.join(os.tmpdir(), 'vpn-proxy-gateway-test-'));
process.env.SING_BOX_CACHE = path.join(process.env.DATA_DIR, 'cache.db'); process.env.SING_BOX_CACHE = path.join(process.env.DATA_DIR, 'cache.db');
const { buildGatewayConfig } = await import(`../../dist/server/singbox.js?gateway=${Date.now()}`); const {
buildDualChannelGatewayConfig,
buildGatewayConfig,
dualChannelConfigMatchesApplied,
fingerprintConfiguredOutbound,
fingerprintSelectedOutbound,
} = await import(`../../dist/server/singbox.js?gateway=${Date.now()}`);
const subscriptionConfig = { const subscriptionConfig = {
outbounds: [{ outbounds: [{
@@ -59,3 +65,62 @@ test('gateway preserves mixed user-rule order and dynamic VPN target', () => {
assert.deepEqual(config.experimental.clash_api, { external_controller: '127.0.0.1:19090' }); assert.deepEqual(config.experimental.clash_api, { external_controller: '127.0.0.1:19090' });
assert.equal(config.route.final, 'test-vpn'); assert.equal(config.route.final, 'test-vpn');
}); });
test('gateway dual-channel config fixes probes to role tags and keeps inbound connections', () => {
const reserveConfig = structuredClone(subscriptionConfig);
reserveConfig.outbounds[0].tag = 'same-provider-tag';
const primaryConfig = structuredClone(subscriptionConfig);
primaryConfig.outbounds[0].tag = 'same-provider-tag';
const config = buildDualChannelGatewayConfig({
primary: { subscriptionConfig: primaryConfig, selectedServerId: 'same-provider-tag' },
reserve: { subscriptionConfig: reserveConfig, selectedServerId: 'same-provider-tag' },
}, {
routeRules: [{ type: 'domain', value: 'api.example.com', enabled: true, outbound: 'vpn' }],
});
assert.deepEqual(config.outbounds.map(({ tag }) => tag), [
'channel-primary', 'channel-reserve', 'channel-selector', 'direct',
]);
assert.deepEqual(config.outbounds[2], {
type: 'selector',
tag: 'channel-selector',
outbounds: ['channel-primary', 'channel-reserve'],
default: 'channel-primary',
interrupt_exist_connections: false,
});
assert.deepEqual(config.route.rules.slice(1, 5), [
{ inbound: ['diagnostics-primary-in'], outbound: 'channel-primary' },
{ inbound: ['diagnostics-reserve-in'], outbound: 'channel-reserve' },
{ inbound: ['diagnostics-vpn-in'], outbound: 'channel-selector' },
{ domain: ['api.example.com'], outbound: 'channel-selector' },
]);
assert.equal(config.route.final, 'channel-selector');
const restoredReserve = buildDualChannelGatewayConfig({
primary: { subscriptionConfig: primaryConfig, selectedServerId: 'same-provider-tag' },
reserve: { subscriptionConfig: reserveConfig, selectedServerId: 'same-provider-tag' },
}, { defaultRole: 'reserve' });
assert.equal(restoredReserve.outbounds[2].default, 'channel-reserve');
});
test('cached dual-channel outbounds must match the applied provider fingerprints', () => {
const config = buildDualChannelGatewayConfig({
primary: { subscriptionConfig, selectedServerId: 'test-vpn' },
reserve: { subscriptionConfig, selectedServerId: 'test-vpn' },
});
const expected = fingerprintSelectedOutbound(subscriptionConfig, 'test-vpn');
const applied = {
primary: { profileId: 'primary', serverId: 'test-vpn' },
reserve: { profileId: 'reserve', serverId: 'test-vpn' },
primaryConfigFingerprint: expected,
reserveConfigFingerprint: expected,
};
assert.equal(dualChannelConfigMatchesApplied(config, applied, 'primary'), true);
assert.equal(dualChannelConfigMatchesApplied(config, applied, 'reserve'), false);
assert.equal(fingerprintConfiguredOutbound(config.outbounds[0], 'test-vpn'), expected);
config.outbounds[0].uuid = '11111111-1111-4111-8111-111111111111';
assert.notEqual(fingerprintConfiguredOutbound(config.outbounds[0], 'test-vpn'), expected);
assert.equal(dualChannelConfigMatchesApplied(config, applied, 'primary'), false);
config.outbounds[0].uuid = subscriptionConfig.outbounds[0].uuid;
config.outbounds[2].outbounds = ['channel-primary'];
assert.equal(dualChannelConfigMatchesApplied(config, applied, 'primary'), false);
});
@@ -0,0 +1,361 @@
import assert from 'node:assert/strict';
import { spawn, spawnSync } from 'node:child_process';
import dgram from 'node:dgram';
import fs from 'node:fs';
import net from 'node:net';
import os from 'node:os';
import path from 'node:path';
import test from 'node:test';
import { createFailoverService } from '../../dist/server/features/failover/failoverService.js';
import { createDomainTrafficService } from '../../dist/server/services/domainTrafficService.js';
import { DEFAULT_FAILOVER_POLICY, normalizeFailoverPolicy } from '../../dist/shared/failover.js';
const image = process.env.HARBOR_SINGBOX_IMAGE;
function listen(server, host = '127.0.0.1') {
return new Promise((resolve, reject) => {
server.once('error', reject);
server.listen(0, host, () => resolve(server.address().port));
});
}
function listenUdp(socket, host = '127.0.0.1') {
return new Promise((resolve, reject) => {
socket.once('error', reject);
socket.bind(0, host, () => resolve(socket.address().port));
});
}
function readExactly(socket, size) {
return new Promise((resolve, reject) => {
let value = Buffer.alloc(0);
const onData = (chunk) => {
value = Buffer.concat([value, chunk]);
if (value.length < size) return;
cleanup();
if (value.length > size) {
socket.pause();
socket.unshift(value.subarray(size));
}
resolve(value.subarray(0, size));
};
const cleanup = () => {
socket.off('data', onData);
socket.off('error', reject);
socket.off('end', onEnd);
};
const onEnd = () => {
cleanup();
reject(new Error('Socket ended early'));
};
socket.on('data', onData);
socket.once('error', reject);
socket.once('end', onEnd);
socket.resume();
});
}
async function openSocksConnection(proxyPort, targetPort, diagnostic = () => {}) {
const socket = net.connect(proxyPort, '127.0.0.1');
socket.setTimeout(3_000, () => socket.destroy(new Error('SOCKS fixture timed out')));
await new Promise((resolve, reject) => {
socket.once('connect', resolve);
socket.once('error', reject);
});
diagnostic('tcp connected');
socket.write(Buffer.from([5, 1, 0]));
assert.deepEqual(await readExactly(socket, 2), Buffer.from([5, 0]));
diagnostic('socks greeting accepted');
const host = Buffer.from('host.docker.internal');
socket.write(Buffer.from([5, 1, 0, 3, host.length, ...host, targetPort >> 8, targetPort & 255]));
const response = await readExactly(socket, 4);
diagnostic(`socks connect response ${response.toString('hex')}`);
assert.equal(response[1], 0);
const addressLength = response[3] === 0
? 0
: response[3] === 1
? 4
: response[3] === 4
? 16
: (await readExactly(socket, 1))[0];
await readExactly(socket, addressLength + 2);
return socket;
}
async function echo(socket, value) {
socket.write(value);
assert.equal((await readExactly(socket, Buffer.byteLength(value))).toString(), value);
}
async function openSocksUdpAssociation(proxyPort, targetPort, resolveRelayPort) {
const control = net.connect(proxyPort, '127.0.0.1');
control.setTimeout(3_000, () => control.destroy(new Error('SOCKS UDP fixture timed out')));
await new Promise((resolve, reject) => {
control.once('connect', resolve);
control.once('error', reject);
});
control.write(Buffer.from([5, 1, 0]));
assert.deepEqual(await readExactly(control, 2), Buffer.from([5, 0]));
control.write(Buffer.from([5, 3, 0, 1, 0, 0, 0, 0, 0, 0]));
const response = await readExactly(control, 4);
assert.equal(response[1], 0);
const addressLength = response[3] === 1 ? 4 : response[3] === 4 ? 16 : (await readExactly(control, 1))[0];
const addressAndPort = await readExactly(control, addressLength + 2);
const boundPort = addressAndPort.readUInt16BE(addressAndPort.length - 2);
const relayPort = resolveRelayPort(boundPort);
const socket = dgram.createSocket('udp4');
await listenUdp(socket);
return {
close() { socket.close(); control.destroy(); },
async echo(value) {
const host = Buffer.from('host.docker.internal');
const payload = Buffer.from(value);
const packet = Buffer.from([0, 0, 0, 3, host.length, ...host, targetPort >> 8, targetPort & 255, ...payload]);
const reply = new Promise((resolve, reject) => {
const timeout = setTimeout(() => reject(new Error(`SOCKS UDP echo timed out; relay ${boundPort}`)), 3_000);
socket.once('message', (message) => {
clearTimeout(timeout);
const headerLength = message[3] === 1 ? 10 : message[3] === 4 ? 22 : 7 + message[4];
resolve(message.subarray(headerLength).toString());
});
});
socket.send(packet, relayPort, '127.0.0.1');
assert.equal(await reply, value);
},
};
}
async function waitForApi(port) {
for (let attempt = 0; attempt < 80; attempt += 1) {
try {
const response = await fetch(`http://127.0.0.1:${port}/proxies/channel-selector`);
if (response.ok) return;
} catch {}
await new Promise((resolve) => setTimeout(resolve, 100));
}
throw new Error('Clash API did not start');
}
test('sing-box 1.13 selector preserves inbound TCP and UDP connections and routes new connections', {
skip: image ? false : 'Set HARBOR_SINGBOX_IMAGE to run the local Docker capability proof',
timeout: 30_000,
}, async (t) => {
const echoServer = net.createServer((socket) => socket.pipe(socket));
const echoPort = await listen(echoServer, '0.0.0.0');
const udpEchoServer = dgram.createSocket('udp4');
udpEchoServer.on('message', (message, remote) => udpEchoServer.send(message, remote.port, remote.address));
const udpEchoPort = await listenUdp(udpEchoServer, '0.0.0.0');
const proxyReservation = net.createServer();
const proxyPort = await listen(proxyReservation);
await new Promise((resolve) => proxyReservation.close(resolve));
const apiReservation = net.createServer();
const apiPort = await listen(apiReservation);
await new Promise((resolve) => apiReservation.close(resolve));
const fixtureDir = fs.mkdtempSync(path.join(os.tmpdir(), 'harbor-selector-'));
const configPath = path.join(fixtureDir, 'config.json');
const config = {
log: { level: 'error' },
experimental: {
cache_file: { enabled: true, path: '/config/cache.db' },
clash_api: { external_controller: '0.0.0.0:19091' },
},
inbounds: [
{ type: 'mixed', tag: 'mixed-in', listen: '0.0.0.0', listen_port: 18081 },
{ type: 'mixed', tag: 'diagnostics-primary-in', listen: '0.0.0.0', listen_port: 18082 },
{ type: 'mixed', tag: 'diagnostics-reserve-in', listen: '0.0.0.0', listen_port: 18083 },
],
outbounds: [
{ type: 'direct', tag: 'channel-primary' },
{ type: 'direct', tag: 'channel-reserve' },
{
type: 'selector',
tag: 'channel-selector',
outbounds: ['channel-primary', 'channel-reserve'],
default: 'channel-primary',
interrupt_exist_connections: false,
},
],
route: {
rules: [
{ inbound: ['diagnostics-primary-in'], outbound: 'channel-primary' },
{ inbound: ['diagnostics-reserve-in'], outbound: 'channel-reserve' },
{ inbound: ['mixed-in'], outbound: 'channel-selector' },
],
final: 'channel-selector',
},
};
fs.writeFileSync(configPath, JSON.stringify(config));
const mount = `${fixtureDir}:/config`;
const check = spawnSync('docker', [
'run', '--rm', '-v', mount, '--entrypoint', 'sing-box', image,
'check', '-c', '/config/config.json',
], { encoding: 'utf8' });
assert.equal(check.status, 0, check.stderr || check.stdout);
t.diagnostic('config accepted');
const containerName = `harbor-selector-${process.pid}-${Date.now()}`;
const udpRelayContainerPorts = Array.from({ length: 32 }, (_, index) => 18084 + index);
const runtime = spawn('docker', [
'run', '--rm', '--name', containerName,
'--sysctl', `net.ipv4.ip_local_port_range=${udpRelayContainerPorts[0]} ${udpRelayContainerPorts.at(-1)}`,
'-p', `${proxyPort}:18081/tcp`,
...udpRelayContainerPorts.flatMap((port) => ['-p', `127.0.0.1::${port}/udp`]),
'-p', `${apiPort}:19091`,
'-v', mount, '--entrypoint', 'sing-box', image,
'run', '-c', '/config/config.json',
], { stdio: ['ignore', 'pipe', 'pipe'] });
let runtimeOutput = '';
runtime.stdout.on('data', (chunk) => { runtimeOutput += chunk; });
runtime.stderr.on('data', (chunk) => { runtimeOutput += chunk; });
t.after(async () => {
if (runtime.exitCode == null) {
runtime.kill('SIGTERM');
await new Promise((resolve) => runtime.once('close', resolve));
}
echoServer.close();
udpEchoServer.close();
fs.rmSync(fixtureDir, { recursive: true, force: true });
});
try {
await waitForApi(apiPort);
const runtimePid = runtime.pid;
t.diagnostic('api ready');
const resolveUdpRelayPort = (containerPort) => {
const mapping = spawnSync('docker', ['port', containerName, `${containerPort}/udp`], { encoding: 'utf8' });
assert.equal(mapping.status, 0, mapping.stderr || mapping.stdout);
const port = Number(mapping.stdout.trim().split(':').at(-1));
assert.ok(Number.isSafeInteger(port) && port > 0, mapping.stdout);
return port;
};
const before = await (await fetch(`http://127.0.0.1:${apiPort}/proxies/channel-selector`)).json();
assert.equal(before.now, 'channel-primary');
const existing = await openSocksConnection(proxyPort, echoPort, (message) => t.diagnostic(message));
t.diagnostic('primary connection open');
t.after(() => existing.destroy());
await echo(existing, 'before-switch');
const existingUdp = await openSocksUdpAssociation(proxyPort, udpEchoPort, resolveUdpRelayPort);
t.after(() => existingUdp.close());
await existingUdp.echo('before-switch-udp');
const policy = normalizeFailoverPolicy({
...DEFAULT_FAILOVER_POLICY,
enabled: true,
primary: { profileId: 'primary-profile', serverId: 'primary-server' },
reserve: { profileId: 'reserve-profile', serverId: 'reserve-server' },
intervalMs: 15_000,
failureWindowMs: 30_000,
recoveryWindowMs: 60_000,
trafficGuard: { enabled: true, thresholdBytesPerSecond: 1024, quietWindowMs: 5_000 },
minimumReserveMs: 60_000,
});
const applied = {
primary: policy.primary,
reserve: policy.reserve,
primaryConfigFingerprint: 'a'.repeat(64),
reserveConfigFingerprint: 'b'.repeat(64),
};
let clock = 0;
let state = {
revision: 1,
failoverPolicy: policy,
failoverRuntimeState: { lastSwitchAt: null, holdUntil: null, primaryQuarantineUntil: null, failoverHistory: [], reasonCode: null },
appliedFailoverPolicy: applied,
appliedProfileId: policy.primary.profileId,
appliedServerId: policy.primary.serverId,
appliedServerSnapshot: { id: policy.primary.serverId, label: 'Primary' },
profiles: [
{ id: policy.primary.profileId, servers: [{ id: policy.primary.serverId, label: 'Primary' }] },
{ id: policy.reserve.profileId, servers: [{ id: policy.reserve.serverId, label: 'Reserve' }] },
],
diagnostics: { customServices: [] },
};
const readSelector = async () => {
const value = await (await fetch(`http://127.0.0.1:${apiPort}/proxies/channel-selector`)).json();
return { role: value.now === 'channel-primary' ? 'primary' : value.now === 'channel-reserve' ? 'reserve' : 'other' };
};
const selectRole = async (role) => {
const response = await fetch(`http://127.0.0.1:${apiPort}/proxies/channel-selector`, {
method: 'PUT',
headers: { 'content-type': 'application/json' },
body: JSON.stringify({ name: role === 'primary' ? 'channel-primary' : 'channel-reserve' }),
});
assert.equal(response.status, 204);
const selected = await readSelector();
assert.equal(selected.role, role);
return selected;
};
const traffic = createDomainTrafficService({
observe: async () => (await fetch(`http://127.0.0.1:${apiPort}/connections`)).json(),
devices: () => [],
now: () => new Date(clock),
});
await traffic.refresh();
const failover = createFailoverService({
state: {
read: () => state,
update: (mutator) => {
state = { ...mutator(state), revision: state.revision + 1 };
return state;
},
},
runtime: { isRunning: async () => true },
dataplane: {
checkConfig: async () => ({}),
runFailoverProbe: async (role) => ({ vpn: { sites: [{ status: role === 'primary' ? 'unavailable' : 'available' }] } }),
readFailoverSelector: readSelector,
selectFailoverRole: selectRole,
setFailoverActivityEnabled: async (value) => value ? traffic.enableActivity() : traffic.disableActivity(),
readFailoverActivity: async (threshold) => {
await traffic.refresh();
return { activity: traffic.activitySnapshot(threshold) };
},
},
buildCandidate: () => ({ config: {}, applied }),
serialize: (operation) => operation(),
scheduler: { setTimeout: () => ({ unref() {} }), clearTimeout: () => {} },
now: () => new Date(clock),
});
t.after(() => failover.shutdown());
await failover.reconcile();
for (clock of [0, 30_000, 35_000]) {
await echo(existing, `active-${clock}-${'x'.repeat(64 * 1024)}`);
await failover.runRound();
}
assert.equal((await readSelector()).role, 'primary');
assert.equal(failover.snapshot().status, 'waiting-for-idle');
t.diagnostic('active traffic delayed selector switch');
clock = 46_000;
await failover.runRound();
assert.equal((await readSelector()).role, 'primary');
clock = 51_000;
await failover.runRound();
const after = await (await fetch(`http://127.0.0.1:${apiPort}/proxies/channel-selector`)).json();
assert.equal(after.now, 'channel-reserve');
t.diagnostic('quiet window elapsed; selector switched and read back');
await echo(existing, 'after-switch-existing');
await existingUdp.echo('after-switch-existing-udp');
const createdAfterSwitch = await openSocksConnection(proxyPort, echoPort, (message) => t.diagnostic(message));
t.diagnostic('reserve connection open');
t.after(() => createdAfterSwitch.destroy());
await echo(createdAfterSwitch, 'after-switch-new');
const udpCreatedAfterSwitch = await openSocksUdpAssociation(proxyPort, udpEchoPort, resolveUdpRelayPort);
t.after(() => udpCreatedAfterSwitch.close());
await udpCreatedAfterSwitch.echo('after-switch-new-udp');
const connections = await (await fetch(`http://127.0.0.1:${apiPort}/connections`)).json();
const chains = connections.connections.map((connection) => connection.chains);
assert.ok(chains.some((chain) => chain.includes('channel-primary')), JSON.stringify(chains));
assert.ok(chains.some((chain) => chain.includes('channel-reserve')), JSON.stringify(chains));
assert.equal(runtime.pid, runtimePid);
assert.equal(runtime.exitCode, null);
} catch (cause) {
assert.fail(`${cause.stack || cause}\n${runtimeOutput}`);
}
});
+1
View File
@@ -320,6 +320,7 @@ setInterval(() => {}, 60_000);
'configExists', 'configExists',
'connection', 'connection',
'diagnostics', 'diagnostics',
'failover',
'fetchedAt', 'fetchedAt',
'gatewayAuto', 'gatewayAuto',
'generatedAt', 'generatedAt',
+33
View File
@@ -115,6 +115,39 @@ test('schema v5 migrates rules and diagnostics settings with an exact backup', (
assert.equal(fs.readFileSync(store.migration.backupPath, 'utf8'), bytes); assert.equal(fs.readFileSync(store.migration.backupPath, 'utf8'), bytes);
}); });
test('schema v7 migrates failover disabled without losing canonical state', (t) => {
const filePath = fixture(t);
const legacy = {
schemaVersion: 7,
revision: 19,
profiles: [],
desiredProfileId: '',
appliedProfileId: '',
appliedServerId: '',
appliedServerSnapshot: null,
routeRules: [],
appliedRouteRules: [],
routeRulesRevision: 4,
diagnostics: { configured: true, customServices: [], hiddenServiceIds: ['google'] },
};
fs.writeFileSync(filePath, JSON.stringify(legacy));
const store = createStateStore(filePath, {
now: () => new Date('2026-08-19T12:00:00.000Z'),
});
const migrated = store.read();
assert.equal(migrated.schemaVersion, 8);
assert.equal(migrated.revision, 19);
assert.equal(migrated.routeRulesRevision, 4);
assert.equal(migrated.failoverPolicy.enabled, false);
assert.equal(migrated.failoverRuntimeState.lastSwitchAt, null);
assert.equal(migrated.appliedFailoverPolicy, null);
assert.deepEqual(migrated.diagnostics.hiddenServiceIds, ['google']);
assert.equal(store.migration.fromVersion, 7);
assert.deepEqual(JSON.parse(fs.readFileSync(store.migration.backupPath, 'utf8')), legacy);
});
test('schema v6 rejects missing or unknown outbound without rewriting source bytes', (t) => { test('schema v6 rejects missing or unknown outbound without rewriting source bytes', (t) => {
for (const [name, rule] of [ for (const [name, rule] of [
['missing', { type: 'domain', value: 'example.com', enabled: true }], ['missing', { type: 'domain', value: 'example.com', enabled: true }],
+116
View File
@@ -124,6 +124,8 @@ function createHarness(overrides = {}) {
clearInterval: (timer) => { timer.clearCalls += 1; }, clearInterval: (timer) => { timer.clearCalls += 1; },
}, },
onRefreshError: overrides.onRefreshError || (() => {}), onRefreshError: overrides.onRefreshError || (() => {}),
onEvent: overrides.onEvent,
failover: overrides.failover,
now: () => new Date('2026-08-08T12:30:00.000Z'), now: () => new Date('2026-08-08T12:30:00.000Z'),
}); });
@@ -305,6 +307,36 @@ test('inactive delete is state-only; applied delete requires one stop-and-delete
assert.equal(after.config, null); assert.equal(after.config, null);
}); });
test('failed active delete restores the previous reserve selector', async () => {
const state = defaultState();
state.failoverPolicy = {
enabled: true,
paused: false,
primary: { profileId: 'work', serverId: 'shared' },
reserve: { profileId: 'personal', serverId: 'shared' },
};
state.appliedFailoverPolicy = {
primary: state.failoverPolicy.primary,
reserve: state.failoverPolicy.reserve,
};
const harness = createHarness({
state,
failStateAt: 1,
failover: {
reconcile: async () => {},
restoreAppliedActivation: async (previousState) => {
assert.equal(previousState.appliedProfileId, 'personal');
harness.calls.push('selector.reserve');
},
},
});
await assert.rejects(harness.service.deleteProfile('personal', 'stop-and-delete'), /state failed/);
assert.equal(harness.snapshot().running, true);
assert.equal(harness.snapshot().config, 'old-config');
assert.equal(harness.snapshot().state.appliedProfileId, 'personal');
assert.ok(harness.calls.indexOf('selector.reserve') > harness.calls.indexOf('runtime.start'));
});
test('deleting a pending desired profile leaves the running applied route untouched', async () => { test('deleting a pending desired profile leaves the running applied route untouched', async () => {
const state = defaultState(); const state = defaultState();
state.desiredProfileId = 'work'; state.desiredProfileId = 'work';
@@ -317,6 +349,90 @@ test('deleting a pending desired profile leaves the running applied route untouc
assert.equal(harness.calls.includes('gateway.set'), false); assert.equal(harness.calls.includes('gateway.set'), false);
}); });
test('deleting a desired failover target disables only the pending policy', async () => {
const state = defaultState();
state.failoverPolicy = {
enabled: true,
paused: false,
primary: { profileId: 'work', serverId: 'shared' },
reserve: { profileId: 'personal', serverId: 'shared' },
};
const harness = createHarness({ state });
await harness.service.deleteProfile('work', 'delete', 3);
const after = harness.snapshot();
assert.equal(after.running, true);
assert.equal(after.config, 'old-config');
assert.equal(after.state.failoverPolicy.enabled, false);
assert.deepEqual(after.state.failoverPolicy.primary, { profileId: '', serverId: '' });
assert.deepEqual(after.state.failoverPolicy.reserve, { profileId: 'personal', serverId: 'shared' });
});
test('refresh pauses failover when a loaded channel target disappears', async () => {
const state = defaultState();
state.failoverPolicy = {
enabled: true,
paused: false,
primary: { profileId: 'personal', serverId: 'shared' },
reserve: { profileId: 'work', serverId: 'shared' },
};
state.appliedFailoverPolicy = {
primary: state.failoverPolicy.primary,
reserve: state.failoverPolicy.reserve,
primaryConfigFingerprint: 'a'.repeat(64),
reserveConfigFingerprint: 'b'.repeat(64),
};
const events = [];
const harness = createHarness({ state, onEvent: (event) => events.push(event) });
await harness.service.refreshProfile('personal');
assert.equal(harness.snapshot().state.failoverPolicy.paused, true);
assert.equal(harness.snapshot().running, true);
assert.equal(harness.snapshot().config, 'old-config');
assert.ok(events.some(({ type }) => type === 'failover.paused'));
});
test('scheduled refresh failures use one stable key for the same failure streak', async () => {
const failure = Object.assign(new Error('down'), { code: 'PROVIDER_UNAVAILABLE' });
const events = [];
const harness = createHarness({
fetchSubscription: async () => { throw failure; },
onEvent: (event) => events.push(event),
});
await assert.rejects(harness.service.refreshProfile('personal', undefined, 'scheduled'));
await assert.rejects(harness.service.refreshProfile('personal', undefined, 'scheduled'));
const keys = events.filter(({ type }) => type === 'subscription.refresh_failed').map(({ dedupeKey }) => dedupeKey);
assert.equal(keys.length, 2);
assert.equal(keys[0], keys[1]);
});
test('scheduled success logs only content changes or recovery while manual refresh always logs', async () => {
const unchanged = (id) => ({
config: { profile: id },
servers: [oldServer],
userInfo: { total: 100 },
fetchedAt: '2026-08-08T12:00:00.000Z',
});
const events = [];
const harness = createHarness({
fetchSubscription: async () => unchanged('work'),
onEvent: (event) => events.push(event),
});
await harness.service.refreshProfile('work', undefined, 'scheduled');
assert.equal(events.length, 0);
await harness.service.refreshProfile('work');
assert.deepEqual(events.map(({ type }) => type), ['subscription.refreshed']);
const recoveryState = defaultState();
recoveryState.profiles[1].lastRefreshErrorCode = 'PROVIDER_UNAVAILABLE';
const recoveryEvents = [];
const recovery = createHarness({
state: recoveryState,
fetchSubscription: async () => unchanged('work'),
onEvent: (event) => recoveryEvents.push(event),
});
await recovery.service.refreshProfile('work', undefined, 'scheduled');
assert.deepEqual(recoveryEvents.map(({ type }) => type), ['subscription.refreshed']);
});
test('auto refresh iterates profiles once, reports scoped failures, and stops idempotently', async () => { test('auto refresh iterates profiles once, reports scoped failures, and stops idempotently', async () => {
const errors = []; const errors = [];
const failure = Object.assign(new Error('down'), { code: 'PROVIDER_UNAVAILABLE' }); const failure = Object.assign(new Error('down'), { code: 'PROVIDER_UNAVAILABLE' });
@@ -0,0 +1,27 @@
import assert from 'node:assert/strict';
import fs from 'node:fs';
import path from 'node:path';
import test from 'node:test';
const root = path.resolve(import.meta.dirname, '../..');
const page = fs.readFileSync(path.join(root, 'src/web/components/ClientOverviewPage.tsx'), 'utf8');
const feature = fs.readFileSync(path.join(root, 'src/web/features/activity-journal/ActivityJournalFeature.tsx'), 'utf8');
const styles = fs.readFileSync(path.join(root, 'src/web/styles/features/activity-journal.css'), 'utf8');
test('journal is the last Gateway-only drawer and refreshes whenever it opens', () => {
assert.match(page, /<RoutingToggle[\s\S]*\{isGateway && <ActivityJournalToggle/);
assert.match(page, /DRAWER_ORDER = \[[^\]]*'journal'\]/);
assert.match(feature, /if \(feature\.isOpen\) void load\(null, status !== 'idle'\)/);
assert.match(feature, /assertActivityJournalPage\(await loadPage\(cursor\)\)/);
});
test('journal presents 30-day grouped history with refresh, pagination and stable states', () => {
assert.match(feature, /Важные события хранятся 30 дней/);
assert.match(feature, /Сегодня[\s\S]*Вчера/);
assert.match(feature, /Обновить журнал/);
assert.match(feature, /M5 8V4m0 4h4/);
assert.match(feature, /Показать ещё/);
assert.match(feature, /Журнал временно недоступен[\s\S]*Повторить/);
assert.match(styles, /\.client-journal-skeleton/);
assert.match(styles, /@media \(prefers-reduced-motion: reduce\)/);
});
+1 -1
View File
@@ -34,7 +34,7 @@ test('connection feature preserves actions, local preference and opaque neighbor
assert.match(panel, /client-state-detail[\s\S]*\{routingSlot\}[\s\S]*client-state-copy[\s\S]*\{serverSlot\}[\s\S]*client-proxies[\s\S]*\{statusSlot\}/); assert.match(panel, /client-state-detail[\s\S]*\{routingSlot\}[\s\S]*client-state-copy[\s\S]*\{serverSlot\}[\s\S]*client-proxies[\s\S]*\{statusSlot\}/);
assert.match(page, /routingSlot=\{<RoutingPendingStatus[\s\S]*blocked=\{connectionBlocked\}[\s\S]*onRestart=\{onRestart\}/); assert.match(page, /routingSlot=\{<RoutingPendingStatus[\s\S]*blocked=\{connectionBlocked\}[\s\S]*onRestart=\{onRestart\}/);
assert.match(routing, /client-route-rules-pending[\s\S]*Перезапустить VPN/); assert.match(routing, /client-route-rules-pending[\s\S]*Перезапустить VPN/);
assert.match(page, /serverSlot=\{<AppliedIdentity identity=\{mainIdentity\} operation=\{switchIdentity\} \/>\}/); assert.match(page, /serverSlot=\{<AppliedIdentity identity=\{mainIdentity\} operation=\{switchIdentity \|\| failoverIdentity\} \/>\}/);
assert.match(page, /mainIdentity[\s\S]*appliedProfile[\s\S]*appliedServer/); assert.match(page, /mainIdentity[\s\S]*appliedProfile[\s\S]*appliedServer/);
assert.match(page, /statusSlot=\{<>[\s\S]*InlineError[\s\S]*InlineProgress/); assert.match(page, /statusSlot=\{<>[\s\S]*InlineError[\s\S]*InlineProgress/);
}); });
+1 -1
View File
@@ -233,7 +233,7 @@ test('Gateway Home reuses the canonical device snapshot for applied route and gl
assert.match(overview, /const appliedServer = appliedProfile\?\.servers\.find[\s\S]*state\.selection\.appliedServerSnapshot/); assert.match(overview, /const appliedServer = appliedProfile\?\.servers\.find[\s\S]*state\.selection\.appliedServerSnapshot/);
assert.match(overview, /const mainIdentity = gatewayDirect[\s\S]*: connected[\s\S]*appliedProfile && appliedServer[\s\S]*`\$\{subscriptionDomain\(appliedProfile\.subscription\.host\)\} · \$\{appliedServer\.label\}`/); assert.match(overview, /const mainIdentity = gatewayDirect[\s\S]*: connected[\s\S]*appliedProfile && appliedServer[\s\S]*`\$\{subscriptionDomain\(appliedProfile\.subscription\.host\)\} · \$\{appliedServer\.label\}`/);
assert.match(overview, /const switchIdentity = gatewayDirect[\s\S]*switchingServer && operationProfile && operationServer[\s\S]*`Переключаем на \$\{subscriptionDomain\(operationProfile\.subscription\.host\)\} · \$\{operationServer\.label\}`/); assert.match(overview, /const switchIdentity = gatewayDirect[\s\S]*switchingServer && operationProfile && operationServer[\s\S]*`Переключаем на \$\{subscriptionDomain\(operationProfile\.subscription\.host\)\} · \$\{operationServer\.label\}`/);
assert.match(overview, /serverSlot=\{<AppliedIdentity identity=\{mainIdentity\} operation=\{switchIdentity\} \/>\}/); assert.match(overview, /serverSlot=\{<AppliedIdentity identity=\{mainIdentity\} operation=\{switchIdentity \|\| failoverIdentity\} \/>\}/);
assert.match(feature, /formatByteString\(globalTraffic\?\.totalBytes \|\| '0'\)/); assert.match(feature, /formatByteString\(globalTraffic\?\.totalBytes \|\| '0'\)/);
assert.match(feature, /const history = globalTraffic\?\.history \|\| \[\][\s\S]*samples=\{history\}[\s\S]*routeLabel="Gateway"[\s\S]*series="speed"/); assert.match(feature, /const history = globalTraffic\?\.history \|\| \[\][\s\S]*samples=\{history\}[\s\S]*routeLabel="Gateway"[\s\S]*series="speed"/);
assert.match(connection, /<section className=\{`client-power-section[\s\S]*client-state-detail[\s\S]*client-connection-title[\s\S]*\{serverSlot\}[\s\S]*client-proxies/); assert.match(connection, /<section className=\{`client-power-section[\s\S]*client-state-detail[\s\S]*client-connection-title[\s\S]*\{serverSlot\}[\s\S]*client-proxies/);
@@ -10,6 +10,7 @@ const page = fs.readFileSync(path.join(root, 'src/web/components/ClientOverviewP
const feature = fs.readFileSync(path.join(root, 'src/web/features/diagnostics/DiagnosticsFeature.tsx'), 'utf8'); const feature = fs.readFileSync(path.join(root, 'src/web/features/diagnostics/DiagnosticsFeature.tsx'), 'utf8');
const panel = fs.readFileSync(path.join(root, 'src/web/features/diagnostics/ConnectivityDiagnosticsPanel.tsx'), 'utf8'); const panel = fs.readFileSync(path.join(root, 'src/web/features/diagnostics/ConnectivityDiagnosticsPanel.tsx'), 'utf8');
const model = fs.readFileSync(path.join(root, 'src/web/features/diagnostics/connectivityResult.ts'), 'utf8'); const model = fs.readFileSync(path.join(root, 'src/web/features/diagnostics/connectivityResult.ts'), 'utf8');
const customServiceAction = fs.readFileSync(path.join(root, 'src/web/features/diagnostics/customServiceAction.ts'), 'utf8');
const boundary = fs.readFileSync(path.join(root, 'src/web/features/diagnostics/index.ts'), 'utf8'); const boundary = fs.readFileSync(path.join(root, 'src/web/features/diagnostics/index.ts'), 'utf8');
const ip = { source: 'cloudflare', address: '198.51.100.10', extra: true }; const ip = { source: 'cloudflare', address: '198.51.100.10', extra: true };
@@ -82,7 +83,7 @@ test('unknown target and legacy-full results pass one identity-preserving parser
assert.doesNotMatch(model, /\sas\s(?:ConnectivityResult|Record<string, unknown>)/); assert.doesNotMatch(model, /\sas\s(?:ConnectivityResult|Record<string, unknown>)/);
}); });
test('serial probes and editor stay panel-owned while backend state owns the service set', () => { test('serial probes stay panel-owned while custom service validation is shared by both editors', () => {
assert.match(panel, /const targets = onlyTarget \? \[onlyTarget\] : \[[\s\S]*CONNECTIVITY_NETWORK_SOURCE\.id[\s\S]*CONNECTIVITY_IP_SOURCES\.map[\s\S]*sites\.map/); assert.match(panel, /const targets = onlyTarget \? \[onlyTarget\] : \[[\s\S]*CONNECTIVITY_NETWORK_SOURCE\.id[\s\S]*CONNECTIVITY_IP_SOURCES\.map[\s\S]*sites\.map/);
assert.match(panel, /for \(const target of targets\) \{[\s\S]*setActiveTarget\(target\)[\s\S]*await runConnectivityDiagnostics\(target\)[\s\S]*if \(legacyFullResult\) break/); assert.match(panel, /for \(const target of targets\) \{[\s\S]*setActiveTarget\(target\)[\s\S]*await runConnectivityDiagnostics\(target\)[\s\S]*if \(legacyFullResult\) break/);
assert.match(panel, /CUSTOM_SERVICES_KEY = 'harbor-diagnostic-services'/); assert.match(panel, /CUSTOM_SERVICES_KEY = 'harbor-diagnostic-services'/);
@@ -92,7 +93,8 @@ test('serial probes and editor stay panel-owned while backend state owns the ser
assert.doesNotMatch(panel, /localStorage\.setItem/); assert.doesNotMatch(panel, /localStorage\.setItem/);
assert.doesNotMatch(panel, /useState\(read(?:Custom|Hidden)Services\)/); assert.doesNotMatch(panel, /useState\(read(?:Custom|Hidden)Services\)/);
assert.match(panel, /slice\(0, MAX_CUSTOM_DIAGNOSTIC_SERVICES\)/); assert.match(panel, /slice\(0, MAX_CUSTOM_DIAGNOSTIC_SERVICES\)/);
assert.match(panel, /parsed\.protocol !== 'https:'/); assert.match(panel, /saveCustomDiagnosticService/);
assert.match(customServiceAction, /parsed\.protocol !== 'https:'/);
assert.match(panel, /document\.startViewTransition\(update\)/); assert.match(panel, /document\.startViewTransition\(update\)/);
assert.match(panel, /retryable: Boolean\(Reflect\.get\(value, 'retryable'\)\)/); assert.match(panel, /retryable: Boolean\(Reflect\.get\(value, 'retryable'\)\)/);
assert.match(panel, /requestDetails\(error\)[\s\S]*requestError\.retryable/); assert.match(panel, /requestDetails\(error\)[\s\S]*requestError\.retryable/);
@@ -0,0 +1,41 @@
import assert from 'node:assert/strict';
import fs from 'node:fs';
import path from 'node:path';
import test from 'node:test';
const root = path.resolve(import.meta.dirname, '../..');
const page = fs.readFileSync(path.join(root, 'src/web/components/ClientOverviewPage.tsx'), 'utf8');
const feature = fs.readFileSync(path.join(root, 'src/web/features/failover/FailoverFeature.tsx'), 'utf8');
const styles = fs.readFileSync(path.join(root, 'src/web/styles/features/failover.css'), 'utf8');
test('reserve is a Gateway-only drawer immediately after subscriptions', () => {
assert.match(page, /<SubscriptionToggle[\s\S]*\{isGateway && <FailoverToggle[\s\S]*<InstructionsToggle/);
assert.match(page, /\{isGateway && hasSubscription && <FailoverPanel/);
assert.match(page, /DRAWER_ORDER = \['subscription', 'failover'/);
assert.match(feature, /label="Резерв"/);
assert.match(feature, /Уже открытые соединения Harbor не закрывает/);
});
test('failover settings expose an off switch, channel targets, checks and per-service timeouts', () => {
assert.match(feature, /Использовать резерв[\s\S]*Полностью пассивен/);
assert.match(feature, /\['primary', 'reserve'\][\s\S]*Подписка[\s\S]*Сервер/);
assert.match(feature, /Что проверять/);
assert.match(feature, /min="2"[\s\S]*max="30"[\s\S]*Таймаут проверки:/);
assert.match(feature, /Проверять каждые, сек[\s\S]*Сбой должен длиться, сек[\s\S]*Восстановление, сек/);
assert.match(feature, /Не переключать во время работы[\s\S]*Тишина перед переключением[\s\S]*Активный трафик, КБ\/с/);
assert.match(feature, /Защита от повторных сбоев[\s\S]*Окно повторных сбоев[\s\S]*Карантин основного/);
assert.match(feature, /Добавить HTTPS-сервис/);
assert.match(feature, /Проверить оба канала/);
assert.match(feature, /beforeunload/);
});
test('runtime status keeps fixed geometry and explains active traffic without motion dependence', () => {
assert.match(feature, /Ждём завершения активной работы/);
assert.match(feature, /Активный трафик ·[\s\S]*transmittingConnections/);
assert.match(feature, /activity\?\.blockers\.slice\(0, 2\)\.map/);
assert.match(feature, /Ещё \{activity\.blockers\.length - 2\}/);
assert.match(feature, /Нестабилен ·[\s\S]*Следующее решение не раньше чем через/);
assert.match(feature, /switchRole && snapshot\.enabled && snapshot\.activation === 'active'/);
assert.match(styles, /\.client-failover-runtime \{[^}]*min-height:\s*132px/);
assert.match(styles, /@media \(prefers-reduced-motion: reduce\)[\s\S]*transition:\s*none/);
});
+19
View File
@@ -56,6 +56,10 @@ test('typed Harbor client validates unknown state and isolates wire compatibilit
gatewayAvailable: true, gatewayAvailable: true,
}); });
assert.equal(Object.hasOwn(parsed, 'proxyPort'), false); assert.equal(Object.hasOwn(parsed, 'proxyPort'), false);
assert.throws(() => parseHarborState({
...snapshot,
failover: { ...snapshot.failover, policy: { ...snapshot.failover.policy, intervalMs: 'fast' } },
}), { code: 'INCOMPATIBLE_API' });
let incompatible; let incompatible;
try { try {
parseHarborState({ ...snapshot, revision: -1 }); parseHarborState({ ...snapshot, revision: -1 });
@@ -116,6 +120,21 @@ test('an equal revision keeps the current snapshot identity', () => {
assert.equal(next.snapshot.selection.desiredServerId, 'one'); assert.equal(next.snapshot.selection.desiredServerId, 'one');
}); });
test('equal revision accepts only newer failover observations and retires old epochs', () => {
const value = (epoch, sequence, status) => ({
...snapshot(4, 'one'),
failover: { observationEpoch: epoch, observationSequence: sequence, status },
});
let state = receive(initialHarborState, value('epoch-a', 1, 'primary'));
state = receive(state, value('epoch-a', 2, 'waiting-for-idle'));
assert.equal(state.snapshot.failover.status, 'waiting-for-idle');
state = receive(state, value('epoch-b', 1, 'reserve'));
assert.equal(state.snapshot.failover.status, 'reserve');
state = receive(state, value('epoch-a', 99, 'primary'));
assert.equal(state.snapshot.failover.status, 'reserve');
assert.deepEqual(state.failoverTransport.retiredEpochs, ['epoch-a']);
});
test('selection has no client-side shadow and follows canonical profile snapshots', () => { test('selection has no client-side shadow and follows canonical profile snapshots', () => {
const state = receive(initialHarborState, snapshot(2, 'two')); const state = receive(initialHarborState, snapshot(2, 'two'));
assert.equal(Object.hasOwn(state, 'pendingServerId'), false); assert.equal(Object.hasOwn(state, 'pendingServerId'), false);
+2 -2
View File
@@ -166,13 +166,13 @@ test('secondary menus share one right rail and switch equal drawers as a vertica
assert.match(instructions, /<Drawer[\s\S]*className="client-instructions"/); assert.match(instructions, /<Drawer[\s\S]*className="client-instructions"/);
assert.match(routing, /<Drawer[\s\S]*className="client-local-rules"/); assert.match(routing, /<Drawer[\s\S]*className="client-local-rules"/);
assert.match(subscription, /<Drawer[\s\S]*className="client-subscription-drawer"/); assert.match(subscription, /<Drawer[\s\S]*className="client-subscription-drawer"/);
assert.match(component, /const DRAWER_ORDER = \['subscription', 'instructions', 'devices', 'diagnostics', 'routing'\]/); assert.match(component, /const DRAWER_ORDER = \['subscription', 'failover', 'instructions', 'devices', 'diagnostics', 'routing', 'journal'\]/);
assert.match(component, /function switchDrawer\(target: DrawerKey\)[\s\S]*from\.inert = true[\s\S]*translateY\(\$\{direction \* 100\}%\)[\s\S]*translateY\(\$\{-direction \* 100\}%\)/); assert.match(component, /function switchDrawer\(target: DrawerKey\)[\s\S]*from\.inert = true[\s\S]*translateY\(\$\{direction \* 100\}%\)[\s\S]*translateY\(\$\{-direction \* 100\}%\)/);
assert.match(component, /const \[drawerSwitchTarget, setDrawerSwitchTarget\] = useState<DrawerKey \| null>\(null\)/); assert.match(component, /const \[drawerSwitchTarget, setDrawerSwitchTarget\] = useState<DrawerKey \| null>\(null\)/);
assert.match(component, /const activeRailDrawer = drawerSwitchTarget && drawerControls\[drawerSwitchTarget\]\.isOpen[\s\S]*drawerControls\[drawer\]\.isOpen/); assert.match(component, /const activeRailDrawer = drawerSwitchTarget && drawerControls\[drawerSwitchTarget\]\.isOpen[\s\S]*drawerControls\[drawer\]\.isOpen/);
assert.match(component, /setDrawerSwitchTarget\(target\);[\s\S]*flushSync\(\(\) => toControl\.show\(\)\)/); assert.match(component, /setDrawerSwitchTarget\(target\);[\s\S]*flushSync\(\(\) => toControl\.show\(\)\)/);
assert.match(component, /fromControl\.close\(\);[\s\S]*setDrawerSwitchTarget\(null\)/); assert.match(component, /fromControl\.close\(\);[\s\S]*setDrawerSwitchTarget\(null\)/);
assert.match(component, /<SubscriptionToggle[\s\S]*open=\{activeRailDrawer === 'subscription'\}[\s\S]*<RoutingToggle[\s\S]*open=\{activeRailDrawer === 'routing'\}/); assert.match(component, /<SubscriptionToggle[\s\S]*open=\{activeRailDrawer === 'subscription'\}[\s\S]*<FailoverToggle[\s\S]*<RoutingToggle[\s\S]*open=\{activeRailDrawer === 'routing'\}[\s\S]*<ActivityJournalToggle/);
assert.match(component, /const cancel = \(\) => \{[\s\S]*from\.inert = false;[\s\S]*from\.removeAttribute\('aria-hidden'\)/); assert.match(component, /const cancel = \(\) => \{[\s\S]*from\.inert = false;[\s\S]*from\.removeAttribute\('aria-hidden'\)/);
assert.match(component, /DRAWER_SWITCH_MS = 620[\s\S]*prefers-reduced-motion: reduce[\s\S]*flushSync/); assert.match(component, /DRAWER_SWITCH_MS = 620[\s\S]*prefers-reduced-motion: reduce[\s\S]*flushSync/);
assert.match(component, /current === 'routing' && routingFeature\.dirty[\s\S]*routingFeature\.requestClose\(\)/); assert.match(component, /current === 'routing' && routingFeature\.dirty[\s\S]*routingFeature\.requestClose\(\)/);
+18 -16
View File
@@ -31,36 +31,38 @@ const expectedImports = [
'./features/servers.css', './features/servers.css',
'./primitives.css', './primitives.css',
'./features/diagnostics.css', './features/diagnostics.css',
'./features/failover.css',
'./features/activity-journal.css',
'./layout.css', './layout.css',
'./themes.css', './themes.css',
]; ];
const sha256 = (value) => crypto.createHash('sha256').update(value).digest('hex'); const sha256 = (value) => crypto.createHash('sha256').update(value).digest('hex');
const acceptedLedger = { const acceptedLedger = {
counts: { counts: {
cascadeEdges: 954, cascadeEdges: 984,
customProperties: 106, customProperties: 106,
declarations: 3487, declarations: 3805,
important: 0, important: 0,
keyframes: 48, keyframes: 48,
media: 13, media: 17,
rules: 971, rules: 1045,
variableReferences: 837, variableReferences: 977,
}, },
hashes: { hashes: {
cascadeEdges: 'd53f6a2236717d6bc27d486fc19f0f80a33169fa7f68a6df2a19536fd30be3bb', cascadeEdges: '423d29f846b45c3522fdc101b5aa1399264f2abe5953b9687b380d5fb865bd07',
customProperties: 'fd6f16069f9fe3526f09d4d574431ddae67150d8e7a8a40e04c9fd2d179ec7ac', customProperties: 'fd6f16069f9fe3526f09d4d574431ddae67150d8e7a8a40e04c9fd2d179ec7ac',
declarations: 'e1401e80ed2ebc5d3601d4d61a3b44c2adbf91db451907ef95a0998170ac5de5', declarations: '52b5bcd795dbea19950afee0259c62cb0fb1aaca094d553cfaaae046a5f747f9',
duplicateKeyframes: '4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945', duplicateKeyframes: '4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945',
duplicateSelectors: '8982145dba05b33bf1c93b2d54cfbe76305b276ff3c657aa020144016ada5848', duplicateSelectors: '8982145dba05b33bf1c93b2d54cfbe76305b276ff3c657aa020144016ada5848',
keyframes: 'a0d69c5b3e5f235d3a76ed04bdd64f67a73fafc9c59fe2635ffda603709c0ad9', keyframes: 'a0d69c5b3e5f235d3a76ed04bdd64f67a73fafc9c59fe2635ffda603709c0ad9',
ruleDeclarationSequences: '89f5f3b302f3a10df0ba90dfbbc63a7ccbc1d8a0c784be26a475321f44dfa8e8', ruleDeclarationSequences: 'bd8d10966c15df58eb76d67fc5a21acaeec524a64805797d5fb33af3b942292e',
selectors: 'dc1a2687ec872d1922a39e8258977949a9b775168764a59119ae6ea93a9f88a1', selectors: '76d2d373aa9ecfd5cac3f95a47ed0fa2c44a659ac98f02f6bfd2f4505de23226',
variableReferences: '843a5f71b0fb74691ed623dbc7ebb7ddddc34e5736e1f419f17cf54e22d43073', variableReferences: 'bea2b082cf2366041936c2da22ac3e0ab83b3e5cead1326a0f65a015daedbb3b',
witnesses: '81676a3fdb6f3bbd63e86b275b6cb6d40e930651b9d5cbde208408f3163ab2b4', witnesses: '605555d8bae9b8d64b1c960c398b14cf910e624d547b72e4fcfced85eec30991',
}, },
}; };
test('public stylesheet exposes exactly twelve flat semantic owners', () => { test('public stylesheet exposes exactly fourteen flat semantic owners', () => {
const imports = Array.from(index.matchAll(/^@import ['"](\.\/[^'"]+\.css)['"];$/gm), ([, file]) => file); const imports = Array.from(index.matchAll(/^@import ['"](\.\/[^'"]+\.css)['"];$/gm), ([, file]) => file);
assert.deepEqual(imports, expectedImports); assert.deepEqual(imports, expectedImports);
assert.equal(index, `${expectedImports.map((file) => `@import '${file}';`).join('\n')}\n`); assert.equal(index, `${expectedImports.map((file) => `@import '${file}';`).join('\n')}\n`);
@@ -209,7 +211,7 @@ test('client typography uses the shared semantic scale outside the token owner',
test('accepted stylesheet has pinned declaration, selector, keyframe, variable, and cascade ledgers', () => { test('accepted stylesheet has pinned declaration, selector, keyframe, variable, and cascade ledgers', () => {
const witnesses = readStyleWitnesses(root); const witnesses = readStyleWitnesses(root);
assert.equal(witnesses.length, 884); assert.equal(witnesses.length, 1021);
assert.equal(witnesses.filter((witness) => witness.unknown || witness.ancestorUnknown).length, 0); assert.equal(witnesses.filter((witness) => witness.unknown || witness.ancestorUnknown).length, 0);
const ledger = createStyleLedger(readStyleSource(root), { witnesses }); const ledger = createStyleLedger(readStyleSource(root), { witnesses });
assert.deepEqual(ledger.counts, acceptedLedger.counts); assert.deepEqual(ledger.counts, acceptedLedger.counts);
@@ -405,8 +407,8 @@ test('main owns one public stylesheet and the regrouped production CSS is determ
assert.equal((main.match(/import ['"][^'"]+\.css['"]/g) || []).length, 1); assert.equal((main.match(/import ['"][^'"]+\.css['"]/g) || []).length, 1);
const assets = fs.readdirSync(path.join(root, 'dist/assets')).filter((file) => file.endsWith('.css')); const assets = fs.readdirSync(path.join(root, 'dist/assets')).filter((file) => file.endsWith('.css'));
assert.deepEqual(assets, ['index-CdbGat4L.css']); assert.deepEqual(assets, ['index-DaioW5qf.css']);
const built = fs.readFileSync(path.join(root, 'dist/assets', assets[0])); const built = fs.readFileSync(path.join(root, 'dist/assets', assets[0]));
assert.equal(built.byteLength, 133670); assert.equal(built.byteLength, 144728);
assert.equal(sha256(built), '1aaa222692046eb166e5bfb8a195e53cf0ef61692a82ae53ca0119b4c8f878a2'); assert.equal(sha256(built), '7d175321821b38be6a947a76457e310cf519e2af4b9ee95fbb0097933ca70a28');
}); });
+2 -2
View File
@@ -34,7 +34,7 @@ test('all repeated client primitive consumers use the shared owners', () => {
assert.equal((production.match(/<Tooltip\b/g) || []).length, 14); assert.equal((production.match(/<Tooltip\b/g) || []).length, 14);
assert.equal((production.match(/<CopyButton\b/g) || []).length, 2); assert.equal((production.match(/<CopyButton\b/g) || []).length, 2);
assert.equal((production.match(/<RailAction\b/g) || []).length, 5); assert.equal((production.match(/<RailAction\b/g) || []).length, 7);
assert.equal((production.match(/<Drawer\b/g) || []).length, 5); assert.equal((production.match(/<Drawer\b/g) || []).length, 7);
assert.doesNotMatch(production, /className="client-tooltip"|className="client-copy-label"|className="client-drawer-close"/); assert.doesNotMatch(production, /className="client-tooltip"|className="client-copy-label"|className="client-drawer-close"/);
}); });