Improve L2 market parsing and data handling
@@ -3,3 +3,7 @@ dist/
|
||||
.DS_Store
|
||||
npm-debug.log*
|
||||
.env*.local
|
||||
playwright-report/
|
||||
test-results/
|
||||
test-output/
|
||||
.pnpm-store/
|
||||
|
||||
@@ -2,6 +2,18 @@
|
||||
|
||||
Кроссплатформенный локальный прототип для захвата нескольких окон Lineage II и сбора предметов из tooltip торгового окна.
|
||||
|
||||
## Прогресс иконок магазина
|
||||
|
||||
Для жёлто-зелёного прогресса дополнительно выдели области `Сетка предметов Sell` и `Сетка предметов Buy`. Нужен только прямоугольник 6 × 3 с ячейками предметов, без заголовка и кнопок прокрутки. Поля необязательны, поэтому старые калибровки продолжают распознавать текст без оверлея иконок.
|
||||
|
||||
На первом чистом кадре магазина занятые ячейки становятся жёлтыми. После успешного OCR и однозначного сравнения с иконкой из каталога соответствующая ячейка становится зелёной. При одинаково подходящих иконках ячейка остаётся жёлтой; приложение не угадывает слот по положению tooltip или порядку обхода. Выбранный источник разворачивается минимум до 640 CSS px, а экран калибровки показывает исходный скриншот в масштабе 2×, сохраняя координаты в исходных пикселях.
|
||||
|
||||
Проверка включает обычные unit-тесты, 17 подписанных игровых скриншотов в Chromium, production build и проверку diff:
|
||||
|
||||
```bash
|
||||
npm run check
|
||||
```
|
||||
|
||||
## Запуск
|
||||
|
||||
Нужен Node.js 20 или новее и Chrome либо Edge.
|
||||
@@ -35,7 +47,7 @@ npm run dev
|
||||
└─ да → OCR точной строки названия и точной строки цены
|
||||
```
|
||||
|
||||
Заголовок и зона tooltip хранятся относительно маркера магазина. Поэтому всё торговое окно можно перемещать. Внутри зоны tooltip отдельно ищется `Price :`, поэтому сам tooltip тоже может двигаться при наведении на разные предметы.
|
||||
Заголовок и зона tooltip хранятся относительно маркера магазина. Поэтому всё торговое окно можно перемещать. Широкая зона tooltip используется только для поиска `Price :` и не передаётся целиком в OCR. После нахождения якоря приложение отдельно определяет фактическую ширину строк названия и цены по их светлым пикселям.
|
||||
|
||||
Из заголовка `Private Store(Buy) - Hikvision` получится `side: "buy"`, `merchant: "Hikvision"`; из `Private Store(Sell) - Dwa` — `side: "sell"`, `merchant: "Dwa"`.
|
||||
|
||||
@@ -47,9 +59,9 @@ npm run dev
|
||||
2. В разделе «Калибровка» сделай снимок или загрузи готовый скриншот.
|
||||
3. Выдели постоянный нижний элемент торгового окна, например `Adena + Confirm`, как маркер магазина. По возможности не захватывай изменяемые числа.
|
||||
4. Выдели заголовок магазина одной строкой: от `Private Store(` до конца имени торговца. Рамку и кнопку закрытия не включай.
|
||||
5. Выдели широкую область всех возможных положений tooltip, не захватывая нижний `Price` основного окна.
|
||||
5. Выдели широкую горизонтальную полосу всех возможных положений tooltip. Она может выходить далеко за пределы торгового окна, но не должна захватывать нижний `Price` основного окна.
|
||||
6. Внутри tooltip обведи только постоянную надпись `Price :`.
|
||||
7. Отдельно выдели точную строку названия и точную строку цены. Для названия лучше использовать длинный предмет и оставить запас справа.
|
||||
7. Отдельно выдели левый край, высоту и начальную ширину строки названия и строки цены. Runtime сам продлит каждую строку вправо до фактического конца текста.
|
||||
8. Сохрани калибровку, нажми «Начать сбор» и вернись в игру.
|
||||
9. Сначала оставь курсор вне предметов, пока в результате не появятся тип сделки и имя торговца.
|
||||
10. Наводи курсор на каждый предмет. Даже если tooltip закроет заголовок, предмет будет записан в последний найденный магазин.
|
||||
@@ -62,7 +74,7 @@ npm run dev
|
||||
2. Добавь второй снимок с tooltip. Калибратор сам найдёт нижний маркер по всему новому кадру и перенесёт относительно него остальные области.
|
||||
3. На втором снимке выдели зону tooltip, `Price :`, название и цену. Уже выбранные области сохраняются.
|
||||
|
||||
Размер игрового окна и положение торгового окна могут отличаться. Масштаб интерфейса Lineage II должен оставаться тем же: шаблоны ищутся в исходном пиксельном размере.
|
||||
Размер игрового окна и положение торгового окна могут отличаться. В калибраторе снимок показывается в масштабе 2× для точной ручной разметки, но шаблоны и сохранённые координаты остаются в исходном пиксельном размере Lineage II.
|
||||
|
||||
После выбора всех шести областей нажми «Проверить снимки». Они обрабатываются по порядку: сначала кадр с видимым заголовком запоминает магазин, затем кадр с tooltip добавляет предмет. Подключать игровое окно для этого не нужно.
|
||||
|
||||
@@ -80,12 +92,12 @@ npm run dev
|
||||
| --- | --- | --- |
|
||||
| Маркер магазина | Небольшой постоянный фрагмент внизу, например `Adena` и `Confirm` | Захватить баланс Adena или другое изменяемое число |
|
||||
| Заголовок магазина | Вся строка `Private Store(Sell) - Dwa` или `Private Store(Buy) - Hikvision` | Обрезать `Buy/Sell` или имя; захватить рамку и кнопку закрытия |
|
||||
| Зона tooltip | Вся полоса возможных положений всплывающего окна над сеткой предметов | Включить нижнюю строку `Price` основного окна |
|
||||
| Зона tooltip | Широкая полоса возможных положений всплывающего окна, в том числе за пределами окна трейда | Включить нижнюю строку `Price` основного окна |
|
||||
| Якорь Price | Только постоянные символы `Price :` внутри tooltip | Добавить цену, которая меняется у каждого предмета |
|
||||
| Название предмета | Одна верхняя строка tooltip с запасом справа | Захватить иконки, фон игры или строку цены |
|
||||
| Строка цены | `Price : 3,000,000 Adena`, `For Each 400 Adena` или только число | Захватить дополнительную строку `(3 Million Adena)` |
|
||||
| Название предмета | Левый край и точная высота верхней строки tooltip; ширина подгоняется автоматически | Захватить иконки, фон игры или строку цены |
|
||||
| Строка цены | Левый край и точная высота `Price : 3,000,000 Adena`, `For Each 400 Adena` или числа | Захватить дополнительную строку `(3 Million Adena)` |
|
||||
|
||||
Лучше калиброваться на предмете с длинным названием и большой ценой. Тогда прямоугольники не обрежут более короткие варианты.
|
||||
Калиброваться на самом длинном предмете больше не требуется: после поиска `Price :` границы названия и цены подгоняются отдельно и останавливаются до удалённого текста интерфейса.
|
||||
|
||||
## Диагностика
|
||||
|
||||
|
||||
|
After Width: | Height: | Size: 473 KiB |
|
After Width: | Height: | Size: 513 KiB |
|
After Width: | Height: | Size: 404 KiB |
|
After Width: | Height: | Size: 458 KiB |
|
After Width: | Height: | Size: 434 KiB |
|
After Width: | Height: | Size: 477 KiB |
|
After Width: | Height: | Size: 433 KiB |
|
After Width: | Height: | Size: 520 KiB |
|
After Width: | Height: | Size: 433 KiB |
|
After Width: | Height: | Size: 520 KiB |
|
After Width: | Height: | Size: 476 KiB |
|
After Width: | Height: | Size: 405 KiB |
|
After Width: | Height: | Size: 431 KiB |
|
After Width: | Height: | Size: 434 KiB |
|
After Width: | Height: | Size: 519 KiB |
|
After Width: | Height: | Size: 408 KiB |
|
After Width: | Height: | Size: 474 KiB |
@@ -0,0 +1,14 @@
|
||||
{
|
||||
"source": "reviewed screenshot crops; replace with real /api/l2/items icon bytes before accepting live catalog-match evidence",
|
||||
"items": {
|
||||
"soulshot-c-grade": { "id": "fixture-soulshot-c-grade", "name": "Soulshot C grade", "iconUrl": "/data/fixture-assets/catalog/soulshot-c-grade.png", "matchScore": 1 },
|
||||
"blessed-spiritshot-d-grade": { "id": "fixture-blessed-spiritshot-d-grade", "name": "Blessed Spiritshot D Grade", "iconUrl": "/data/fixture-assets/catalog/blessed-spiritshot-d-grade.png", "matchScore": 1 },
|
||||
"spellbook-prominence": { "id": "fixture-spellbook-prominence", "name": "Spellbook Prominence", "iconUrl": "/data/fixture-assets/catalog/spellbook-prominence.png", "matchScore": 1 },
|
||||
"brigandine-helmet": { "id": "fixture-brigandine-helmet", "name": "Brigandine Helmet", "iconUrl": "/data/fixture-assets/catalog/brigandine-helmet.png", "matchScore": 1 },
|
||||
"tutorial-guide": { "id": "fixture-tutorial-guide", "name": "Tutorial Guide", "iconUrl": "/data/fixture-assets/catalog/tutorial-guide.png", "matchScore": 1 },
|
||||
"dimensional-fragment": { "id": "fixture-dimensional-fragment", "name": "Dimensional Fragment", "iconUrl": "/data/fixture-assets/catalog/dimensional-fragment.png", "matchScore": 1 },
|
||||
"ancient-adena": { "id": "fixture-ancient-adena", "name": "Ancient Adena", "iconUrl": "/data/fixture-assets/catalog/ancient-adena.png", "matchScore": 1 },
|
||||
"scroll-enchant-weapon-d": { "id": "fixture-scroll-enchant-weapon-d", "name": "Scroll Enchant Weapon D", "iconUrl": "/data/fixture-assets/catalog/scroll-enchant-weapon-d.png", "matchScore": 1 },
|
||||
"coarse-bone-powder": { "id": "fixture-coarse-bone-powder", "name": "Coarse Bone Powder", "iconUrl": "/data/fixture-assets/catalog/coarse-bone-powder.png", "matchScore": 1 }
|
||||
}
|
||||
}
|
||||
|
After Width: | Height: | Size: 2.7 KiB |
|
After Width: | Height: | Size: 2.5 KiB |
|
After Width: | Height: | Size: 2.3 KiB |
|
After Width: | Height: | Size: 2.2 KiB |
|
After Width: | Height: | Size: 2.5 KiB |
|
After Width: | Height: | Size: 1.8 KiB |
|
After Width: | Height: | Size: 2.6 KiB |
|
After Width: | Height: | Size: 2.6 KiB |
|
After Width: | Height: | Size: 2.5 KiB |
|
After Width: | Height: | Size: 3.4 KiB |
|
After Width: | Height: | Size: 1.5 KiB |
@@ -0,0 +1,51 @@
|
||||
{
|
||||
"version": 1,
|
||||
"frame": { "width": 1560, "height": 1360 },
|
||||
"saleAnchor": {
|
||||
"image": "fixture-assets/templates/sale-anchor.png",
|
||||
"width": 79,
|
||||
"height": 29
|
||||
},
|
||||
"storeHeaderRegion": {
|
||||
"offsetX": -162,
|
||||
"offsetY": -356,
|
||||
"width": 222,
|
||||
"height": 18
|
||||
},
|
||||
"tooltipSearchRegion": {
|
||||
"offsetX": -300,
|
||||
"offsetY": -430,
|
||||
"width": 650,
|
||||
"height": 220
|
||||
},
|
||||
"tooltipAnchor": {
|
||||
"image": "fixture-assets/templates/tooltip-anchor.png",
|
||||
"width": 42,
|
||||
"height": 17
|
||||
},
|
||||
"itemNameRegion": {
|
||||
"offsetX": -3,
|
||||
"offsetY": -28,
|
||||
"width": 260,
|
||||
"height": 17
|
||||
},
|
||||
"itemPriceRegion": {
|
||||
"offsetX": 0,
|
||||
"offsetY": 0,
|
||||
"width": 260,
|
||||
"height": 17
|
||||
},
|
||||
"sellItemGridRegion": {
|
||||
"offsetX": -164,
|
||||
"offsetY": -305,
|
||||
"width": 222,
|
||||
"height": 111
|
||||
},
|
||||
"buyItemGridRegion": {
|
||||
"offsetX": -164,
|
||||
"offsetY": -305,
|
||||
"width": 222,
|
||||
"height": 111
|
||||
},
|
||||
"threshold": 0.76
|
||||
}
|
||||
@@ -0,0 +1,360 @@
|
||||
{
|
||||
"version": 1,
|
||||
"viewerCharacter": "Deela",
|
||||
"frame": { "width": 1560, "height": 1360 },
|
||||
"calibration": "fixture-calibration.json",
|
||||
"cases": [
|
||||
{
|
||||
"id": "sell-domestos-clean",
|
||||
"file": "Discord_7xmx8OSuCH.jpg",
|
||||
"groupId": "sell-domestos",
|
||||
"order": 0,
|
||||
"scene": "clean",
|
||||
"mode": "standalone",
|
||||
"expected": {
|
||||
"side": "sell",
|
||||
"merchant": "Domestos",
|
||||
"occupiedSlots": [0],
|
||||
"pendingSlots": [0],
|
||||
"foundSlots": []
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "sell-crom-clean",
|
||||
"file": "Discord_K9RkTrU6B2.jpg",
|
||||
"groupId": "sell-crom",
|
||||
"order": 0,
|
||||
"scene": "clean",
|
||||
"mode": "standalone",
|
||||
"expected": {
|
||||
"side": "sell",
|
||||
"merchant": "Crom",
|
||||
"occupiedSlots": [0, 1],
|
||||
"pendingSlots": [0, 1],
|
||||
"foundSlots": []
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "sell-gnumli-clean",
|
||||
"file": "Discord_w3Smu4bOQK.jpg",
|
||||
"groupId": "sell-gnumli",
|
||||
"order": 0,
|
||||
"scene": "clean",
|
||||
"mode": "sequence",
|
||||
"expected": {
|
||||
"side": "sell",
|
||||
"merchant": "Gnumli",
|
||||
"occupiedSlots": [0, 1],
|
||||
"pendingSlots": [0, 1],
|
||||
"foundSlots": []
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "sell-gnumli-soulshot",
|
||||
"file": "Discord_ARWzEIaayl.jpg",
|
||||
"groupId": "sell-gnumli",
|
||||
"order": 1,
|
||||
"scene": "tooltip",
|
||||
"mode": "sequence",
|
||||
"catalogAssetId": "soulshot-c-grade",
|
||||
"expected": {
|
||||
"side": "sell",
|
||||
"merchant": "Gnumli",
|
||||
"occupiedSlots": [0, 1],
|
||||
"hoveredSlot": 0,
|
||||
"pendingSlots": [1],
|
||||
"foundSlots": [0],
|
||||
"item": {
|
||||
"displayName": "Soulshot: C-grade",
|
||||
"name": "Soulshot C grade",
|
||||
"quantity": 36370,
|
||||
"priceAdena": 20,
|
||||
"priceMode": "price"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "sell-gnumli-blessed-spiritshot",
|
||||
"file": "Discord_qy3EYIEcMV.jpg",
|
||||
"groupId": "sell-gnumli",
|
||||
"order": 2,
|
||||
"scene": "tooltip",
|
||||
"mode": "sequence",
|
||||
"catalogAssetId": "blessed-spiritshot-d-grade",
|
||||
"expected": {
|
||||
"side": "sell",
|
||||
"merchant": "Gnumli",
|
||||
"occupiedSlots": [0, 1],
|
||||
"hoveredSlot": 1,
|
||||
"pendingSlots": [],
|
||||
"foundSlots": [0, 1],
|
||||
"item": {
|
||||
"displayName": "Blessed Spiritshot: D-Grade",
|
||||
"name": "Blessed Spiritshot D Grade",
|
||||
"quantity": 7245,
|
||||
"priceAdena": 58,
|
||||
"priceMode": "price"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "sell-boroda4-clean",
|
||||
"file": "Discord_sSJAGZLHvm.jpg",
|
||||
"groupId": "sell-boroda4",
|
||||
"order": 0,
|
||||
"scene": "clean",
|
||||
"mode": "sequence",
|
||||
"expected": {
|
||||
"side": "sell",
|
||||
"merchant": "Boroda4",
|
||||
"occupiedSlots": [0, 1],
|
||||
"pendingSlots": [0, 1],
|
||||
"foundSlots": []
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "sell-boroda4-spellbook",
|
||||
"file": "Discord_KL0fnDcvr7.jpg",
|
||||
"groupId": "sell-boroda4",
|
||||
"order": 1,
|
||||
"scene": "tooltip",
|
||||
"mode": "sequence",
|
||||
"catalogAssetId": "spellbook-prominence",
|
||||
"expected": {
|
||||
"side": "sell",
|
||||
"merchant": "Boroda4",
|
||||
"occupiedSlots": [0, 1],
|
||||
"hoveredSlot": 0,
|
||||
"pendingSlots": [1],
|
||||
"foundSlots": [0],
|
||||
"item": {
|
||||
"displayName": "Spellbook: Prominence",
|
||||
"name": "Spellbook Prominence",
|
||||
"quantity": 2,
|
||||
"priceAdena": 50000,
|
||||
"priceMode": "price"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "sell-boroda4-helmet",
|
||||
"file": "Discord_dxpCzaJEWD.jpg",
|
||||
"groupId": "sell-boroda4",
|
||||
"order": 2,
|
||||
"scene": "tooltip",
|
||||
"mode": "sequence",
|
||||
"catalogAssetId": "brigandine-helmet",
|
||||
"expected": {
|
||||
"side": "sell",
|
||||
"merchant": "Boroda4",
|
||||
"occupiedSlots": [0, 1],
|
||||
"hoveredSlot": 1,
|
||||
"pendingSlots": [],
|
||||
"foundSlots": [0, 1],
|
||||
"item": {
|
||||
"displayName": "Brigandine Helmet",
|
||||
"name": "Brigandine Helmet",
|
||||
"quantity": 1,
|
||||
"priceAdena": 800000,
|
||||
"priceMode": "price"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "sell-shotd-clean",
|
||||
"file": "Discord_uqGcQhxIWc.jpg",
|
||||
"groupId": "sell-shotd",
|
||||
"order": 0,
|
||||
"scene": "clean",
|
||||
"mode": "sequence",
|
||||
"expected": {
|
||||
"side": "sell",
|
||||
"merchant": "shotD",
|
||||
"occupiedSlots": [0, 1],
|
||||
"pendingSlots": [0, 1],
|
||||
"foundSlots": []
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "sell-shotd-tutorial-guide",
|
||||
"file": "Discord_fb9f61z8Ek.jpg",
|
||||
"groupId": "sell-shotd",
|
||||
"order": 1,
|
||||
"scene": "tooltip",
|
||||
"mode": "sequence",
|
||||
"catalogAssetId": "tutorial-guide",
|
||||
"expected": {
|
||||
"side": "sell",
|
||||
"merchant": "shotD",
|
||||
"occupiedSlots": [0, 1],
|
||||
"hoveredSlot": 0,
|
||||
"pendingSlots": [1],
|
||||
"foundSlots": [0],
|
||||
"item": {
|
||||
"displayName": "Tutorial Guide",
|
||||
"name": "Tutorial Guide",
|
||||
"quantity": 1,
|
||||
"priceAdena": 5000000,
|
||||
"priceMode": "price"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "sell-shotd-dimensional-fragment",
|
||||
"file": "Discord_ezvlzpNHRg.jpg",
|
||||
"groupId": "sell-shotd",
|
||||
"order": 2,
|
||||
"scene": "tooltip",
|
||||
"mode": "sequence",
|
||||
"catalogAssetId": "dimensional-fragment",
|
||||
"expected": {
|
||||
"side": "sell",
|
||||
"merchant": "shotD",
|
||||
"occupiedSlots": [0, 1],
|
||||
"hoveredSlot": 1,
|
||||
"pendingSlots": [],
|
||||
"foundSlots": [0, 1],
|
||||
"item": {
|
||||
"displayName": "Dimensional Fragment",
|
||||
"name": "Dimensional Fragment",
|
||||
"quantity": 687,
|
||||
"priceAdena": 3500,
|
||||
"priceMode": "price"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "sell-1shop-clean",
|
||||
"file": "Discord_tJFxvFi7DM.jpg",
|
||||
"groupId": "sell-1shop",
|
||||
"order": 0,
|
||||
"scene": "clean",
|
||||
"mode": "sequence",
|
||||
"expected": {
|
||||
"side": "sell",
|
||||
"merchant": "1SHOP",
|
||||
"occupiedSlots": [0],
|
||||
"pendingSlots": [0],
|
||||
"foundSlots": []
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "sell-1shop-ancient-adena",
|
||||
"file": "Discord_fFbzX9vDhW.jpg",
|
||||
"groupId": "sell-1shop",
|
||||
"order": 1,
|
||||
"scene": "tooltip",
|
||||
"mode": "sequence",
|
||||
"catalogAssetId": "ancient-adena",
|
||||
"expected": {
|
||||
"side": "sell",
|
||||
"merchant": "1SHOP",
|
||||
"occupiedSlots": [0],
|
||||
"hoveredSlot": 0,
|
||||
"pendingSlots": [],
|
||||
"foundSlots": [0],
|
||||
"item": {
|
||||
"displayName": "Ancient Adena",
|
||||
"name": "Ancient Adena",
|
||||
"quantity": 1793000,
|
||||
"priceAdena": 3,
|
||||
"priceMode": "price"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "buy-rakot-clean",
|
||||
"file": "Discord_wQc5FDayAK.jpg",
|
||||
"groupId": "buy-rakot",
|
||||
"order": 0,
|
||||
"scene": "clean",
|
||||
"mode": "sequence",
|
||||
"expected": {
|
||||
"side": "buy",
|
||||
"merchant": "RAKOT",
|
||||
"occupiedSlots": [0, 1, 2],
|
||||
"pendingSlots": [0, 1, 2],
|
||||
"foundSlots": []
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "buy-rakot-scroll",
|
||||
"file": "Discord_TxbA20ZOQB.jpg",
|
||||
"groupId": "buy-rakot",
|
||||
"order": 1,
|
||||
"scene": "tooltip",
|
||||
"mode": "sequence",
|
||||
"catalogAssetId": "scroll-enchant-weapon-d",
|
||||
"expected": {
|
||||
"side": "buy",
|
||||
"merchant": "RAKOT",
|
||||
"occupiedSlots": [0, 1, 2],
|
||||
"hoveredSlot": 0,
|
||||
"pendingSlots": [1, 2],
|
||||
"foundSlots": [0],
|
||||
"item": {
|
||||
"displayName": "Scroll: Enchant Weapon (D)",
|
||||
"name": "Scroll Enchant Weapon D",
|
||||
"quantity": 0,
|
||||
"priceAdena": 200000,
|
||||
"priceMode": "each"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "buy-rakot-coarse-bone-powder",
|
||||
"file": "Discord_ikzA9VhROZ.jpg",
|
||||
"groupId": "buy-rakot",
|
||||
"order": 2,
|
||||
"scene": "tooltip",
|
||||
"mode": "sequence",
|
||||
"catalogAssetId": "coarse-bone-powder",
|
||||
"expected": {
|
||||
"side": "buy",
|
||||
"merchant": "RAKOT",
|
||||
"occupiedSlots": [0, 1, 2],
|
||||
"hoveredSlot": 2,
|
||||
"pendingSlots": [1],
|
||||
"foundSlots": [0, 2],
|
||||
"item": {
|
||||
"displayName": "Coarse Bone Powder",
|
||||
"name": "Coarse Bone Powder",
|
||||
"quantity": 0,
|
||||
"priceAdena": 1,
|
||||
"priceMode": "each"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "buy-kapayji-ancient-adena",
|
||||
"file": "Discord_2KrJ9GGjCP.jpg",
|
||||
"groupId": "buy-kapayji",
|
||||
"order": 0,
|
||||
"scene": "tooltip",
|
||||
"mode": "contextual",
|
||||
"catalogAssetId": "ancient-adena",
|
||||
"initialState": {
|
||||
"side": "buy",
|
||||
"merchant": "KapayJI",
|
||||
"activeShopKey": "buy:kapayji",
|
||||
"pendingSlots": [0],
|
||||
"foundSlots": []
|
||||
},
|
||||
"expected": {
|
||||
"side": "buy",
|
||||
"merchant": "KapayJI",
|
||||
"occupiedSlots": [0],
|
||||
"hoveredSlot": 0,
|
||||
"pendingSlots": [],
|
||||
"foundSlots": [0],
|
||||
"item": {
|
||||
"displayName": "Ancient Adena",
|
||||
"name": "Ancient Adena",
|
||||
"quantity": 0,
|
||||
"priceAdena": 2,
|
||||
"priceMode": "each"
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,39 @@
|
||||
# Dynamic Tooltip Crop Evidence
|
||||
|
||||
## Acceptance Evidence
|
||||
|
||||
- Browser geometry: a 1920 x 1080 reference keeps an intrinsic canvas of
|
||||
1920 x 1080 and renders at 3840 x 2160. The stage scroll width grows to
|
||||
3888 px, so the image is not squeezed back into the panel. See
|
||||
[calibration-2x.png](./calibration-2x.png).
|
||||
- Browser pointer smoke: drawing on the 2x canvas creates calibration regions
|
||||
and advances through all six calibration steps. The marker drag from display
|
||||
`(326.6, 604)` to `(426.6, 642)` mapped to source `(288, 259)` through
|
||||
`(338, 278)` and was subsequently found by anchor matching at 99.8%.
|
||||
- Real tooltip fixture: the complete calibration check finished 6/6 stages and
|
||||
parsed `Sword of Revolution`, quantity `1`, and `1,700,000 Adena` at 70% OCR
|
||||
confidence. The third descriptive tooltip row was not passed to item parsing.
|
||||
See [tooltip-diagnostics.png](./tooltip-diagnostics.png).
|
||||
- The wide tooltip region is used only for the `Price :` anchor search. The
|
||||
item OCR preview contains the fitted name and price rows, not the full search
|
||||
band.
|
||||
|
||||
## Verification
|
||||
|
||||
- `node.exe --test`: 18 tests passed, including white/yellow pixel masks,
|
||||
stopping a text run before distant UI noise, and fitting a short row whose
|
||||
legacy saved width crosses the frame edge.
|
||||
- `node.exe node_modules/vite/bin/vite.js build`: production build passed.
|
||||
Vite reports the existing large OpenCV chunk warning.
|
||||
- `git diff --check`: passed; Git only reported LF-to-CRLF conversion warnings.
|
||||
|
||||
## Review Notes
|
||||
|
||||
- PRE plan review: aligned after clarifying the single owner of the dynamic
|
||||
right edge, the no-wide-OCR fallback, and legacy calibration behavior.
|
||||
- POST plan review: aligned; saved screenshots and the unbounded-run regression
|
||||
close the evidence and no-wide-fallback gates.
|
||||
- Correctness review: no remaining findings after isolating fitted rows in a
|
||||
masked composite and applying yellow pixels only to the price row.
|
||||
- Maintainability review: no remaining findings after reading pixels directly
|
||||
from the source canvas and simplifying the text-run control flow.
|
||||
@@ -0,0 +1,10 @@
|
||||
# Goal: Dynamic Tooltip Crop
|
||||
|
||||
Use Krypton Execution to execute `docs/goals/dynamic-tooltip-crop/PLAN.md`.
|
||||
|
||||
Core rules:
|
||||
- Treat PLAN.md as the source plan.
|
||||
- Preserve intent, ownership, contract, cutover, evidence, and kill criteria.
|
||||
- Do not add a second wide-area OCR path.
|
||||
- Keep existing calibration data readable; fixed widths provide seed geometry only and must not be used as a wide OCR fallback. Empty masks produce diagnostics `not-found`.
|
||||
- Capture acceptance evidence in `EVIDENCE.md`.
|
||||
@@ -0,0 +1,69 @@
|
||||
# Dynamic Tooltip Crop Implementation Plan
|
||||
|
||||
**Intent:** Сделать ручную калибровку пиксель-точной при масштабе 2× и читать строки tooltip переменной ширины без OCR всего кадра.
|
||||
**Current Behavior:** Калибратор показывает кадр 1:1, а runtime после поиска `Price :` использует фиксированную ширину калиброванных строк названия и цены.
|
||||
**Expected Outcome:** Кадр размечается в масштабе 2×; найденные строки названия и итоговой цены tooltip локально подгоняются вправо по фактическим белым и жёлтым пикселям до первого устойчивого пустого промежутка, включая текст длиннее калиброванного образца.
|
||||
**Target-Perspective Output:** Пользователь выделяет мелкие области на увеличенном кадре и получает название и цену как для короткого, так и для длинного tooltip без постороннего текста за его правой границей.
|
||||
**Truth Owner:** `src/vision.js` владеет определением фактической правой границы OCR-строки; калибровка сохраняет стабильный левый край, положение и высоту строки относительно `Price :`.
|
||||
**Contract Boundary:** Чистый `findTextRunEnd(columns, gap)` находит конец первого текстового пробега. Canvas-wrapper `fitOcrTextRect(canvas, rect, options)` сканирует от калиброванного левого края до правой границы кадра, использует gap 12 px и padding 6 px, применяет white mask для названия и white+yellow для цены и возвращает frame-clamped rect либо `null`, если пробег не найден.
|
||||
**Cutover:** `analyzeFrame` подгоняет `nameRect` и `priceRect` перед `combineRects`; прямое использование фиксированной ширины прекращается.
|
||||
**Displaced Path:** Фиксированные ширины `itemNameRegion` и `itemPriceRegion` больше не задают окончательную границу и не используются как широкий OCR fallback; при пустой маске item OCR пропускается с диагностикой.
|
||||
**Value Density:** Один локальный пиксельный проход по двум тонким строкам вместо OCR широкой области или нескольких OCR-попыток.
|
||||
**Evidence Gate:** На реальном тестовом кадре 1920×1080 canvas отображается 3840×2160; выделение в 2× визуально совпадает с целевыми source-пикселями; реальный tooltip читается при искусственно короткой калиброванной ширине, а удалённый соседний текст не попадает в diagnostics crop/OCR.
|
||||
**Acceptance Evidence:** Browser geometry и screenshot выделения; границы двух реальных строк разной длины из предоставленного tooltip; diagnostics crop + parsed name/price этого tooltip при намеренно узкой калиброванной ширине; runnable-тест чистого определения конца текстового пробега; production build.
|
||||
**Evidence Lane:** Локальный браузер и `node --test`.
|
||||
**Kill Criteria:** В runtime нет второго OCR-пути по широкой области или широкого фиксированного fallback; `combineRects` не вызывается до успешной подгонки обеих строк.
|
||||
**Architecture Slice:** `src/main.js`, `src/vision.js`, `src/style.css`, `test/parser.test.js`, `README.md`; документация калибровки уточняет различие зоны поиска и OCR-строк.
|
||||
**Plan Review Gate:** Requires PRE review before execution.
|
||||
|
||||
## Architecture Slice
|
||||
|
||||
- Files to create: только этот goal-пакет.
|
||||
- Files to modify: `src/main.js`, `src/vision.js`, `src/style.css`, `test/parser.test.js`, `README.md`.
|
||||
- Files to avoid: сохранённый формат калибровки, парсер текста tooltip, market outbox и backend.
|
||||
- Source of truth: найденный `Price :` задаёт позицию; пиксели текущего кадра задают фактическую ширину.
|
||||
- Read path: кадр → поиск магазина → поиск `Price :` → калиброванные строки → подгонка по пикселям → OCR → parser.
|
||||
- Write path: без новых сохраняемых полей; существующая калибровка остаётся совместимой.
|
||||
- Contract boundary: чистый `findTextRunEnd` и canvas-wrapper `fitOcrTextRect` в vision-слое.
|
||||
- Integration points: два прямоугольника в `analyzeFrame`, CSS-размер canvas.
|
||||
- Migration/cutover: немедленный, без миграции данных.
|
||||
- Displaced path: фиксированная ширина перестаёт быть OCR fallback; пустая маска становится наблюдаемым `not-found` в diagnostics.
|
||||
- Acceptance evidence gate: 2× browser geometry + screenshot точного выделения + diagnostics реального tooltip + unit test длинной строки + full build.
|
||||
|
||||
## Runtime Invariants
|
||||
|
||||
- Левый край строк названия и цены стабилен относительно найденного `Price :`; переменной является правая граница.
|
||||
- Продукт сохраняет только название, количество и итоговую цену; остальные описательные строки tooltip не являются данными продукта.
|
||||
- `tooltipSearchRegion` может быть широкой горизонтальной полосой за пределами окна трейда, но исключает нижний `Price` основного окна; эта полоса используется только для template matching.
|
||||
- Сканирование OCR-строки начинается с её калиброванного левого края, идёт не дальше правой границы кадра, допускает внутренние промежутки меньше 12 px и добавляет не более 6 px после последнего текстового столбца.
|
||||
- Название использует белую маску; цена использует белую и жёлтую маски.
|
||||
- Если текстовый пробег не найден, OCR предмета не запускается, а diagnostics явно сообщает причину.
|
||||
|
||||
## Tasks
|
||||
|
||||
1. Последовательно изменить `src/main.js` и `src/style.css`: отображать calibration canvas ровно в 2× intrinsic size; координаты остаются корректными через существующий `canvasPoint`.
|
||||
- Allowed scope: только display size и crisp integer scaling.
|
||||
- Verification: браузер показывает intrinsic 1920×1080 и rendered 3840×2160; screenshot известного выделения совпадает с source-пикселями.
|
||||
2. Затем изменить `src/vision.js` и `test/parser.test.js`: добавить чистый `findTextRunEnd` и локальную подгонку правого края с раздельными масками, gap 12, padding 6 и `null` для пустой маски.
|
||||
- Allowed scope: preprocessing границы; без нового OCR-прохода и без зависимости canvas в Node.
|
||||
- Verification: `node --test` покрывает длинный текст, внутренние пробелы, удалённый шум и пустую маску.
|
||||
3. После задачи 2 интегрировать подгонку в `src/main.js` до `combineRects`; при `null` остановить item OCR и записать причину в diagnostics.
|
||||
- Allowed scope: существующий item OCR branch.
|
||||
- Verification: diagnostics показывают фактический локальный crop, а широкий search region и удалённый соседний текст не попадают в OCR.
|
||||
4. Обязательно обновить `README.md`: широкая полоса ищет только `Price :`; ширины строк являются начальными левыми/вертикальными ориентирами и подгоняются runtime.
|
||||
- Allowed scope: только инструкции выбора `tooltipSearch`, `itemName` и `itemPrice`.
|
||||
- Verification: README больше не требует калибровать ширину по самому длинному предмету.
|
||||
5. Прогнать прямые эквиваленты `npm test`, `npm run build`, `git diff --check` и визуальную проверку большого кадра; записать реальные результаты в `EVIDENCE.md`.
|
||||
- Acceptance evidence: geometry 2×, screenshot точного выделения, границы двух реальных строк, diagnostics OCR/parse и вывод проверочных команд.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Не определять всю рамку tooltip и не сохранять его описательные строки, потому что продукту нужны название, количество и итоговая цена.
|
||||
- Не искать `Price :` по всему кадру без ограничивающей полосы.
|
||||
- Не добавлять настройку масштаба или новый UI-контрол в этом срезе.
|
||||
- Не менять формат сохранённой калибровки.
|
||||
|
||||
## Risks
|
||||
|
||||
- Слишком короткий допустимый пробел обрежет текст; слишком длинный захватит соседний UI.
|
||||
- Горизонтальная рамка внутри строки может выглядеть как непрерывный текст; точные вертикальные калиброванные строки остаются обязательными, а пустой/неограниченный пробег должен завершаться `not-found`, не широким OCR.
|
||||
|
After Width: | Height: | Size: 57 KiB |
|
After Width: | Height: | Size: 62 KiB |
@@ -0,0 +1,37 @@
|
||||
# Store Icon Progress Overlay Evidence
|
||||
|
||||
## Implemented
|
||||
|
||||
- `data/fixtures.json` describes every one of the 17 committed JPG files exactly once, including clean/tooltip sequence state and reviewed merchant/item/slot expectations.
|
||||
- Live capture and the browser harness call the same `src/frame-analyzer.js#analyzeFrame`; no second OCR pipeline was added.
|
||||
- Calibration supports optional independent Sell and Buy 6 × 3 grid regions. The calibration canvas is displayed at 2× while stored coordinates remain source pixels.
|
||||
- A clean recognized shop frame records occupied cells as `pending`; a unique icon match above the absolute threshold and second-best margin changes only that cell to `found`.
|
||||
- Active-shop changes and three consecutive missing sale anchors clear grid/slot progress without deleting parsed historical items.
|
||||
- The selected source preview expands to at least 640 CSS px and draws pending slots yellow, found slots green, followed by transient diagnostics and a parsed/total counter.
|
||||
- Slot thumbnails and the grid-captured marker are runtime-only and are excluded from clipboard JSON and the market outbox payload.
|
||||
|
||||
## Verification run (2026-08-11)
|
||||
|
||||
- `node --test`: **30 passed, 0 failed**.
|
||||
- `vite build`: **passed**. Existing OpenCV browser-externalization and large-chunk warnings remain.
|
||||
- `git diff --check`: **passed**; only the repository's expected LF-to-CRLF conversion warnings were printed.
|
||||
- Playwright screenshot lane: **17/17 cases executed** through production OpenCV/Tesseract/parser, but the semantic assertion currently reports **42 mismatches**. The lane is intentionally red and must not be represented as acceptance-complete.
|
||||
|
||||
Useful browser evidence despite the red aggregate:
|
||||
|
||||
- Clean `sell-gnumli` detected exactly pending slots `[0,1]`.
|
||||
- Clean `buy-rakot` detected exactly pending slots `[0,1,2]`.
|
||||
- In the Gnumli sequence, the Blessed Spiritshot catalog crop uniquely matched slot `1`, producing `pending=[0]`, `found=[1]`.
|
||||
- A targeted rerun wrote `sell-gnumli-clean-overlay.jpg` with both occupied cells yellow and `sell-gnumli-blessed-spiritshot-overlay.jpg` with slot 0 yellow / slot 1 green under the ignored Playwright result directory.
|
||||
- The contextual `buy-kapayji` tooltip-only case produced no reported mismatch, including its pending-to-found slot transition.
|
||||
- The remaining failures are dominated by low-resolution Lineage font OCR (for example `Domestos` → `Domastos`, `Gnumli` → `Enumli`) and lost tooltip quantities/names. Raising the occupancy threshold from `0.15` to `0.18` removed the observed false-positive `shotD` slot 16 on the final full-corpus rerun.
|
||||
- The fixture failure output now contains `fixture-report.json`, one annotated overlay and per-stage crops for every failing case under ignored `test-results/`.
|
||||
- Desktop and 600 px narrow source-card states were rendered from the production CSS contract and visually inspected during implementation. The preview measured at least 640 px on desktop and collapsed to one column without horizontal overflow at 600 px, but those synthetic-layout screenshots were not retained as durable goal artifacts and do not prove the live renderer/canvas call path.
|
||||
|
||||
## Unproven / blocked evidence
|
||||
|
||||
- Fixture catalog PNGs are reviewed screenshot crops, not bytes returned by the real `/api/l2/items` `iconUrl`. They prove the matching path and ambiguity guard, but not live catalog artwork compatibility.
|
||||
- The local host returns no usable real catalog/calibration endpoints, so hosted calibration PUT→GET preservation and a real catalog icon score remain unverified.
|
||||
- Manual OS display capture was not exercised; the CSS layout screenshots do not prove the real `getDisplayMedia` source or the actual `drawSourceOverlay` canvas call path.
|
||||
|
||||
Acceptance remains partial until the 17-case semantic lane is green and one real API icon is paired with an occupied live slot.
|
||||
@@ -0,0 +1,21 @@
|
||||
# Goal: Store Icon Progress Overlay
|
||||
|
||||
Use Krypton Execution to execute `docs/goals/store-icon-progress/PLAN.md` after the required Buy-store fixture and catalog icon URL evidence are available.
|
||||
|
||||
Core rules:
|
||||
- Treat PLAN.md as the source plan.
|
||||
- Create `data/fixtures.json` from the reviewed 17-file corpus before changing recognition behavior.
|
||||
- Keep fixture expected data human-owned; tests may report actual values but must never bless or rewrite expectations automatically.
|
||||
- Keep fixture calibration, templates, catalog JSON/icon bytes and sequence pending/found states human-owned and versioned.
|
||||
- Run fixtures through the single `src/frame-analyzer.js#analyzeFrame` path used by live capture; remove the inline implementation from `main.js` and do not create a test-only parser.
|
||||
- Use real production OpenCV/Tesseract/parser in the Playwright lane; substitute only catalog transport with committed fixture assets.
|
||||
- Preserve the active result as slot-state truth owner.
|
||||
- Keep old six-field calibration readable and operational.
|
||||
- Do not infer the parsed slot from tooltip position or hover order.
|
||||
- Mark a slot green only for a unique best visual match above both the absolute threshold and second-best margin; ties remain yellow.
|
||||
- Do not send runtime images or slot state to the backend.
|
||||
- Make the active source preview readable at the real source-card scale before accepting overlay evidence.
|
||||
- Verify optional grid calibration survives the hosted PUT-to-GET round trip.
|
||||
- Capture yellow-to-green browser evidence and record it in EVIDENCE.md.
|
||||
- Finish with one `npm run check` gate covering unit tests, all fixture screenshots, the production build and `git diff --check`.
|
||||
- Say `implemented but unproven` if Buy/Sell or catalog-icon matching cannot be demonstrated.
|
||||
@@ -0,0 +1,167 @@
|
||||
# Store Icon Progress Overlay Implementation Plan
|
||||
|
||||
**Intent:** Показывать прогресс обхода открытого магазина прямо поверх захваченного игрового кадра: занятые иконки ожидают tooltip в жёлтой рамке и становятся зелёными после успешного распознавания и привязки к слоту.
|
||||
**Current Behavior:** Overlay показывает только служебные области текущего кадра и общий статус одного tooltip; приложение не знает список занятых слотов и не хранит прогресс по каждой иконке.
|
||||
**Expected Outcome:** После распознавания магазина приложение находит занятые слоты его Buy/Sell-сетки, показывает их жёлтыми, сохраняет чистый снимок сетки без tooltip и переводит точный слот в зелёный после OCR предмета и совпадения его каталоговой иконки.
|
||||
**Target-Perspective Output:** В развёрнутом превью выбранного окна (не уже 640 CSS px на desktop, когда позволяет viewport) виден реальный магазин с компактным чек-листом поверх его иконок: жёлтый означает «наведи курсор», зелёный — «этот слот уже прочитан»; рядом показан счётчик `2/5 иконок`.
|
||||
**Truth Owner:** Активный `result` источника владеет текущим магазином и его `iconSlots`; `src/vision.js` владеет определением занятых ячеек и сравнением визуальной иконки, а `drawSourceOverlay` только рисует состояние.
|
||||
**Contract Boundary:** Калибровка хранит необязательные `sellItemGridRegion` и `buyItemGridRegion` относительно найденного маркера магазина. `detectOccupiedSlots(frame, absoluteGridRect)` возвращает `{slotId,row,column,frameRect,score}[]`. После `resolveCatalogItem` `matchCatalogIconToSlot(slots, icon)` возвращает ровно один `slotId` только при прохождении абсолютного порога и отрыва от второго результата; иначе возвращает `null`.
|
||||
**Cutover:** Существующие diagnostics boxes продолжают показывать pipeline, но прогресс иконок рисуется из отдельного устойчивого `result.iconSlots`; общий статус «Предмет ожидает» заменяется счётчиком слотов, когда сетка доступна.
|
||||
**Displaced Path:** Не использовать ближайший к tooltip слот, порядок наведения или «первый жёлтый» как источник истины — эти эвристики могут покрасить не ту иконку.
|
||||
**Value Density:** Две необязательные области калибровки, один лёгкий пиксельный проход по 18 ячейкам при открытии магазина и одно сравнение с pending slots после нового предмета; без OCR всех иконок и без нового backend-контракта.
|
||||
**Evidence Gate:** Все 17 кадров из `data/` описаны в human-reviewed manifest и прогоняются тем же analyzer path, что live capture; на полном Sell-кадре занятые слоты выделены, пустые не выделены; на полном Buy-кадре работает отдельная область; после связанного tooltip ровно соответствующий слот меняет жёлтый на зелёный, а остальные остаются жёлтыми.
|
||||
**Acceptance Evidence:** `npm run test:fixtures` сравнивает merchant/side/occupied slots/tooltip item с manifest для каждого файла и проверяет связанные store sequences; browser screenshots до/после tooltip для Sell и Buy при фактическом размере source card; unit tests occupied-slot classifier, unique-best/tie и active-shop reset; production build; calibration PUT→GET round-trip; отсутствие slot-данных в clipboard export/outbox.
|
||||
**Evidence Lane:** Human-reviewed `data/fixtures.json`, локальный browser fixture runner, `node --test`, production build.
|
||||
**Kill Criteria:** Нет второго overlay canvas, нет позиционного угадывания hovered slot, нет обязательной миграции старой калибровки, нет отправки thumbnail/data URL в backend; inline `analyzeFrame` удалён из `main.js`, и live capture/fixture runner импортируют один `analyzeFrame` из `src/frame-analyzer.js`.
|
||||
**Architecture Slice:** `data/*.jpg`, `data/fixtures.json`, `index.html`, `src/main.js`, выделяемый production analyzer seam, `src/vision.js`, `src/parser.js`, fixture tests, `package.json`, `README.md`, `src/style.css`.
|
||||
**Plan Review Gate:** Requires PRE review before execution.
|
||||
|
||||
## Product Direction
|
||||
|
||||
- Domain: private store, Buy/Sell grid, occupied slot, tooltip, catalog icon, captured frame, scan progress.
|
||||
- Color world: тёмный игровой кадр, приглушённый amber ожидания, зелёный подтверждения, красный ошибки, нейтральный синий diagnostics.
|
||||
- Signature: живой чек-лист непосредственно на слотах Lineage II, а не отдельный dashboard со списком.
|
||||
- Rejected defaults: отдельная карточка каждого pending slot; анимированные пульсирующие рамки; окрашивание всех пустых клеток сетки.
|
||||
- Direction: сохранить реальный кадр главным слоем и добавить только семантические рамки и компактный счётчик.
|
||||
|
||||
## Component Checkpoint
|
||||
|
||||
- Intent: владелец инструмента быстро видит, какие предметы в конкретном магазине ещё нужно обойти; интерфейс остаётся спокойным и утилитарным.
|
||||
- Hierarchy: сам кадр и рамки слотов — focal point; textual status вторичен.
|
||||
- Palette: существующие `pending` amber и `found` green; новые декоративные цвета не добавляются.
|
||||
- Depth: borders-only поверх видео, без теней и glow.
|
||||
- Surfaces: существующее preview surface; новый контейнер не создаётся.
|
||||
- Typography: существующий маленький overlay status с tabular counter.
|
||||
- Spacing: существующая плотность 4/8 px; рамка не перекрывает содержимое иконки.
|
||||
- Readability: активное source preview разворачивается минимум до 640 CSS px на desktop; на узком viewport занимает доступную ширину без отдельного уменьшенного дубликата.
|
||||
|
||||
## Architecture Slice
|
||||
|
||||
- Files to create: `data/fixtures.json`, `data/fixture-calibration.json`, reviewed template/catalog assets under `data/fixture-assets/`, `src/frame-analyzer.js`, Playwright fixture regression spec and browser evidence images.
|
||||
- Files to modify: `index.html`, `src/main.js`, `src/vision.js`, `src/parser.js`, fixture/unit tests, `package.json`, `README.md`, `src/style.css`; hosted calibration endpoint/schema добавляется в scope только если round-trip отбрасывает optional regions.
|
||||
- Files to avoid: `src/market-outbox.js`, import proxy, backend payload, catalog API contract.
|
||||
- Source of truth: `data/fixtures.json` владеет reviewed expected result/initial state/slot transitions каждого screenshot sequence; `data/fixture-calibration.json` и `data/fixture-assets/` владеют reviewed test geometry/templates/catalog bytes; production analyzer владеет фактическим результатом кадра; `result.activeShopKey` + `result.iconSlots` владеют live progress.
|
||||
- Read path: fixture JPG + fixture calibration → тот же production analyzer, что live frame → actual result → semantic comparison with manifest; live frame → sale anchor → header side/merchant → side-specific grid → occupied slots → yellow overlay; tooltip → parsed item → catalog icon → saved grid snapshot → matched slot → green overlay.
|
||||
- Write path: fixture manifest/calibration/assets редактируются только при добавлении или ручной перепроверке screenshots; тесты не переписывают expected values автоматически; calibration дополняется двумя optional regions; runtime slot state остаётся только в памяти источника.
|
||||
- Contract boundary: `src/frame-analyzer.js` экспортирует единственный `analyzeFrame({frame,result,calibration,templates,source,services}) -> Promise<result>`; production OCR/OpenCV/parser импортируются внутри этого модуля или передаются только через production adapters, а `services.catalog` является единственной fixture substitution; `detectOccupiedSlots(frame, absoluteGridRect) -> [{slotId,row,column,frameRect,score}]`; `matchCatalogIconToSlot(slots, icon) -> slotId|null`; canvas wrappers live in `vision.js`.
|
||||
- Integration points: fixture manifest loader/schema, production `analyzeFrame`, `emptyCalibration`, calibration UI/addRegion/rendering, `resolveCatalogItem` result, `drawSourceOverlay`, package verification scripts.
|
||||
- Migration/cutover: existing v4 calibration is read unchanged; missing grid region disables only icon progress for that side.
|
||||
- Displaced path: общий item status остаётся fallback только при отсутствии grid calibration.
|
||||
- Acceptance evidence gate: full-window Buy and Sell fixtures plus tooltip/item catalog icon.
|
||||
|
||||
## Fixture Corpus and Ground Truth
|
||||
|
||||
Все кадры имеют размер 1560 x 1360 и сняты персонажем `Deela`. `merchant` ниже — персонаж открытого магазина. Номер slot считается с нуля слева направо по верхней строке. Значения `unknown` не угадываются по иконке: такой fixture проверяет заголовок и занятость слота, но не название предмета.
|
||||
|
||||
| File | Scene | Expected store | Visible/hovered contents |
|
||||
| --- | --- | --- | --- |
|
||||
| `Discord_2KrJ9GGjCP.jpg` | Buy + tooltip | `buy:KapayJI`, occupied `[0]` | slot 0: `Ancient Adena`, quantity `0`, each `2` Adena |
|
||||
| `Discord_7xmx8OSuCH.jpg` | Sell, clean | `sell:Domestos`, occupied `[0]` | slot 0: `unknown` |
|
||||
| `Discord_ARWzEIaayl.jpg` | Sell + tooltip | `sell:Gnumli`, occupied `[0,1]` | slot 0: `Soulshot: C-grade`, quantity `36,370`, price `20` Adena |
|
||||
| `Discord_dxpCzaJEWD.jpg` | Sell + tooltip | `sell:Boroda4`, occupied `[0,1]` | slot 1: `Brigandine Helmet`, quantity `1`, price `800,000` Adena |
|
||||
| `Discord_ezvlzpNHRg.jpg` | Sell + tooltip | `sell:shotD`, occupied `[0,1]` | slot 1: `Dimensional Fragment`, quantity `687`, price `3,500` Adena |
|
||||
| `Discord_fb9f61z8Ek.jpg` | Sell + tooltip | `sell:shotD`, occupied `[0,1]` | slot 0: `Tutorial Guide`, quantity `1`, price `5,000,000` Adena |
|
||||
| `Discord_fFbzX9vDhW.jpg` | Sell + tooltip | `sell:1SHOP`, occupied `[0]` | slot 0: `Ancient Adena`, quantity `1,793,000`, price `3` Adena |
|
||||
| `Discord_ikzA9VhROZ.jpg` | Buy + tooltip | `buy:RAKOT`, occupied `[0,1,2]` | slot 2: `Coarse Bone Powder`, quantity `0`, each `1` Adena |
|
||||
| `Discord_K9RkTrU6B2.jpg` | Sell, clean | `sell:Crom`, occupied `[0,1]` | slots 0–1: `unknown` |
|
||||
| `Discord_KL0fnDcvr7.jpg` | Sell + tooltip | `sell:Boroda4`, occupied `[0,1]` | slot 0: `Spellbook: Prominence`, quantity `2`, price `50,000` Adena |
|
||||
| `Discord_qy3EYIEcMV.jpg` | Sell + tooltip | `sell:Gnumli`, occupied `[0,1]` | slot 1: `Blessed Spiritshot: D-Grade`, quantity `7,245`, price `58` Adena |
|
||||
| `Discord_sSJAGZLHvm.jpg` | Sell, clean | `sell:Boroda4`, occupied `[0,1]` | paired contents: `Spellbook: Prominence`, `Brigandine Helmet` |
|
||||
| `Discord_tJFxvFi7DM.jpg` | Sell, clean | `sell:1SHOP`, occupied `[0]` | paired content: `Ancient Adena` |
|
||||
| `Discord_TxbA20ZOQB.jpg` | Buy + tooltip | `buy:RAKOT`, occupied `[0,1,2]` | slot 0: `Scroll: Enchant Weapon (D)`, quantity `0`, each `200,000` Adena |
|
||||
| `Discord_uqGcQhxIWc.jpg` | Sell, clean | `sell:shotD`, occupied `[0,1]` | paired contents: `Tutorial Guide`, `Dimensional Fragment` |
|
||||
| `Discord_w3Smu4bOQK.jpg` | Sell, clean | `sell:Gnumli`, occupied `[0,1]` | paired contents: `Soulshot: C-grade`, `Blessed Spiritshot: D-Grade` |
|
||||
| `Discord_wQc5FDayAK.jpg` | Buy, clean | `buy:RAKOT`, occupied `[0,1,2]` | slot 0: `Scroll: Enchant Weapon (D)`; slot 1: `unknown`; slot 2: `Coarse Bone Powder` |
|
||||
|
||||
Manifest records both human-facing `displayName` and parser-facing normalized `name`, for example `Spellbook: Prominence` → `Spellbook Prominence`. It also records `viewerCharacter`, `groupId`, `scene`, standalone vs contextual assertion mode, `side`, `merchant`, `occupiedSlots`, optional `hoveredSlot`, optional reviewed `initialState`, `expectedPendingSlots`, `expectedFoundSlots`, `catalogAssetId`, and optional `{displayName,name,quantity,priceAdena,priceMode}`. Every `data/*.jpg` must appear exactly once; missing files and stale manifest entries fail before OCR starts.
|
||||
|
||||
`data/fixture-calibration.json` is human-reviewed versioned truth for this 1560 x 1360 corpus: sale anchor, store header, tooltip search/anchor, name/price regions and Buy/Sell grid regions. Template images are committed as explicit files under `data/fixture-assets/`; runner never derives calibration from actual results. Catalog mappings point to committed API JSON plus original icon bytes by `catalogAssetId/path`, not to manifest-derived generated icons.
|
||||
|
||||
Sequence groups:
|
||||
|
||||
- `sell-gnumli`: clean → Soulshot tooltip → Blessed Spiritshot tooltip.
|
||||
- `sell-boroda4`: clean → Spellbook tooltip → Brigandine Helmet tooltip.
|
||||
- `sell-shotd`: clean → Tutorial Guide tooltip → Dimensional Fragment tooltip.
|
||||
- `sell-1shop`: clean → Ancient Adena tooltip.
|
||||
- `buy-rakot`: clean → Scroll tooltip → Coarse Bone Powder tooltip; middle slot intentionally remains unknown/pending.
|
||||
- `sell-domestos`, `sell-crom`: standalone clean frames guarding the confirmed header and occupancy fields.
|
||||
- `buy-kapayji`: tooltip-only partial group with human-reviewed `initialState` for `buy:KapayJI` and slot 0 pending; it validates tooltip parsing/transition only and does not claim standalone header/grid recognition.
|
||||
|
||||
## Fixture Test Contract
|
||||
|
||||
- Unit/schema lane (`node --test`): manifest validity, exact JPG coverage, unique ids, slot ranges, paired group consistency, normalization and pure state transitions.
|
||||
- Screenshot lane (`npm run test:fixtures`): `@playwright/test` starts the Vite fixture page in bundled Chromium, loads real JPGs, applies committed fixture calibration, calls `src/frame-analyzer.js#analyzeFrame`, and compares only declared expected fields. It uses real production OpenCV/Tesseract/parser; only `/api/l2/items` transport is fulfilled from committed catalog JSON/icon bytes.
|
||||
- Sequence lane: feed group frames in manifest order and assert remembered `side:merchant`, item aggregation and explicit `expectedPendingSlots`/`expectedFoundSlots` after every frame. Contextual-only fixtures start from their reviewed `initialState`; they are never reported as standalone full-frame passes.
|
||||
- Diagnostics on failure: write an actual-vs-expected JSON report plus crop/overlay artifacts under an ignored test-output directory; never overwrite the manifest or source JPGs.
|
||||
- Stable assertions: exact semantic merchant/side/name/quantity/price and slot ids; no pixel-perfect full-screen snapshots and no assertion on OCR confidence unless a threshold regression is specifically under test.
|
||||
- Three-miss reset stays in the pure state-transition unit lane because the current corpus has no three-frame no-store sequence.
|
||||
- Verification command: add `npm run check` for unit tests, all 17 fixture cases, production build and `git diff --check`; hosted calibration PUT→GET remains a separately recorded environment evidence gate because it requires the deployed backend.
|
||||
|
||||
## Runtime Invariants
|
||||
|
||||
- Buy и Sell используют разные calibrated grid regions; side выбирается только из распознанного заголовка.
|
||||
- Grid region делится на фиксированную Interlude-сетку 6 x 3; пользователь выделяет внешний прямоугольник клеток без заголовка и scroll buttons.
|
||||
- Пустые ячейки не получают рамку; занятые начинаются как `pending`.
|
||||
- Occupancy считается только внутри фиксированного inset каждой ячейки, исключающего border; метрика и порог выбираются по измеренному разрыву между occupied/empty примерами обоих типов магазина и фиксируются константой с fixture-тестом.
|
||||
- Grid snapshot обновляется только на кадре с распознанным заголовком и без найденного tooltip, чтобы tooltip не закрыл иконки.
|
||||
- Смена `side:merchant` полностью сбрасывает runtime slots, но не исторические parsed items.
|
||||
- Зелёный статус ставится только если лучший visual match превышает измеренный threshold и опережает второй результат не меньше чем на измеренный margin; tie/identical icons возвращают `null`, остаются жёлтыми и получают статус «неоднозначно».
|
||||
- Повторный OCR того же предмета не перекрашивает другой одинаковый slot без нового однозначного совпадения.
|
||||
- Slot thumbnails, grid snapshots и match scores не попадают в outbox или exported JSON.
|
||||
- После трёх последовательных кадров без `saleAnchor`/нижнего маркера магазина (существующий `result.misses`) очищаются `activeShopKey`, grid snapshot и slots; отсутствие header OCR при видимом маркере, в том числе из-за tooltip, прогресс не сбрасывает; повторное открытие того же merchant строит состояние заново.
|
||||
|
||||
## Tasks
|
||||
|
||||
1. Зафиксировать corpus в `data/fixtures.json` и добавить schema/coverage tests.
|
||||
- Allowed scope: существующие 17 JPG остаются неизменными; expected values вводятся вручную по таблице выше; unknown fields остаются явно unknown.
|
||||
- Expected output: каждый JPG описан ровно один раз, связанные кадры объединены `groupId`, human display и parser-normalized item names разделены.
|
||||
- Verification: `node --test` validates schema, coverage and group consistency.
|
||||
- Parallel: no; establishes regression truth.
|
||||
2. Выделить `src/frame-analyzer.js#analyzeFrame` и добавить Playwright browser fixture runner без дублирования pipeline.
|
||||
- Allowed scope: перенести только ownership анализа кадра из DOM-heavy `main.js`; удалить inline implementation; live capture и fixtures импортируют один symbol; runner использует production OCR/OpenCV/parser, подменяя только внешний catalog transport committed fixture-ответом.
|
||||
- Expected output: `npm run test:fixtures` обрабатывает JPGs, сравнивает только declared fields и сохраняет понятные failure artifacts.
|
||||
- Verification: source search находит одно определение `analyzeFrame`; намеренно испорченный expected field даёт targeted diff; после возврата manifest corpus проходит.
|
||||
- Parallel: no; establishes executable evidence lane.
|
||||
3. Добавить optional Sell/Buy grid calibration в `index.html`, `src/main.js`, `src/parser.js`.
|
||||
- Allowed scope: два поля относительно sale anchor; существующие шесть остаются обязательным минимумом.
|
||||
- Expected output: старые сохранения работают; grid fields можно выбрать, удалить, сохранить и увидеть на calibration canvas.
|
||||
- Verification: parser/calibration tests, browser selection smoke и hosted PUT→GET round-trip; если поля теряются, минимально расширить серверную схему.
|
||||
- Parallel: no; extends the contract.
|
||||
4. Добавить в `src/vision.js` минимальные slot helpers: 6 x 3 geometry, occupied-cell classifier и unique-best matching catalog icon к pending slots.
|
||||
- Allowed scope: локальная grid image analysis; использовать существующий OpenCV loader/matcher.
|
||||
- Expected output: provided Sell fixture returns exactly its occupied cells; empty grid returns none.
|
||||
- Verification: Node tests для geometry/classifier seam, absolute threshold, second-best margin, identical-icon tie и browser fixture for canvas wrapper.
|
||||
- Parallel: no; consumes tasks 1-3 contracts and uses both Sell/Buy fixture corpus.
|
||||
5. Интегрировать runtime state в `src/main.js`.
|
||||
- Allowed scope: active-shop reset, clean grid snapshot, slot match after `resolveCatalogItem`, no backend writes.
|
||||
- Expected output: stable pending/found slots persist across frames and reset on shop change/close.
|
||||
- Verification: focused state-transition tests, three-miss close/reset, reopening same merchant and two-frame browser flow; assert no slot/snapshot fields in clipboard export and outbox.
|
||||
- Parallel: no; consumes tasks 3-4.
|
||||
6. Развернуть активное live-превью и расширить существующий `drawSourceOverlay`/status copy.
|
||||
- Allowed scope: selected source preview минимум 640 CSS px на desktop; draw slot rects before transient diagnostics; reuse `boxColor(pending/found)`.
|
||||
- Expected output: игровые слоты читаемы в карточке источника, yellow/green outlines и `parsed/total` counter; no new canvas or motion.
|
||||
- Verification: desktop screenshot именно при фактическом размере source card, narrow viewport screenshot, squint/token/state checks.
|
||||
- Parallel: can start after runtime contract is fixed.
|
||||
7. Обновить `README.md`, добавить единый `npm run check`, прогнать tests/build/diff-check и записать browser evidence в `EVIDENCE.md`.
|
||||
- Acceptance evidence: manifest coverage report `17/17`, Sell and Buy yellow states, exact green transition, shop reset.
|
||||
- Parallel: no.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Автоматически кликать или наводить курсор в игровом окне.
|
||||
- OCR названий прямо с 32 px icon.
|
||||
- Сохранять slot progress между перезапусками приложения.
|
||||
- Менять каталог, import payload или исторический список результатов.
|
||||
- Угадывать слот по положению tooltip или порядку обхода.
|
||||
|
||||
## Required Input Before Execution
|
||||
|
||||
- Реальный ответ `/api/l2/items` для одного известного предмета из Sell fixture, включая `iconUrl`, и доступные по этому URL bytes (либо контролируемый локальный/hosted fixture). Same-origin сам по себе не доказывает, что catalog artwork, alpha и размер совпадают с игровой иконкой.
|
||||
- Buy fixture теперь есть: `Discord_wQc5FDayAK.jpg`; по нему и clean Sell fixtures во время исполнения измерить occupied/empty metric gap и только затем зафиксировать cell inset/threshold.
|
||||
- По реальной паре slot crop ↔ catalog icon измерить absolute match threshold и best-vs-second margin; до этого зелёная привязка считается непроверенной.
|
||||
|
||||
## Risks
|
||||
|
||||
- Кастомный клиент может изменить размер/число slot cells; фиксированная 6 x 3 геометрия тогда потребует отдельной настройки.
|
||||
- Catalog icon может отличаться рамкой, альфой или масштабом от клиентского asset; evidence должен зафиксировать реальный match score до выбора порога.
|
||||
- Одинаковые иконки в нескольких слотах требуют дополнительного различителя; до него неоднозначные совпадения должны оставаться жёлтыми.
|
||||
@@ -0,0 +1,29 @@
|
||||
import { test, expect } from "@playwright/test";
|
||||
import { writeFile } from "node:fs/promises";
|
||||
|
||||
test("production analyzer matches the reviewed screenshot corpus", async ({ page }, testInfo) => {
|
||||
const fixture = process.env.FIXTURE_ID;
|
||||
await page.goto(fixture ? `/fixture.html?fixture=${encodeURIComponent(fixture)}` : "/fixture.html");
|
||||
const result = await page.evaluate(() => window.__fixtureRun);
|
||||
await writeFile(testInfo.outputPath("fixture-report.json"), JSON.stringify(result, null, 2));
|
||||
const artifacts = await page.evaluate(() => window.__fixtureArtifacts);
|
||||
for (const [fixtureId, artifact] of Object.entries(artifacts)) {
|
||||
const images = [
|
||||
["overlay", artifact.preview],
|
||||
...artifact.stages.map(({ key, image }) => [key, image]),
|
||||
];
|
||||
for (const [label, dataUrl] of images) {
|
||||
if (!dataUrl) continue;
|
||||
const [, encoded] = dataUrl.split(",", 2);
|
||||
const extension = dataUrl.startsWith("data:image/png") ? "png" : "jpg";
|
||||
await writeFile(
|
||||
testInfo.outputPath(`${fixtureId}-${label}.${extension}`),
|
||||
Buffer.from(encoded, "base64"),
|
||||
);
|
||||
}
|
||||
}
|
||||
expect(result.total).toBeGreaterThan(0);
|
||||
expect(result.completed).toBe(result.total);
|
||||
if (!fixture) expect(result.completed).toBe(17);
|
||||
expect(result.failures, result.failures.join("\n")).toEqual([]);
|
||||
});
|
||||
@@ -0,0 +1,19 @@
|
||||
<!doctype html>
|
||||
<html lang="ru">
|
||||
<head>
|
||||
<meta charset="UTF-8" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||||
<title>L2 fixture regression</title>
|
||||
<style>
|
||||
body { margin: 24px; background: #171714; color: #eee; font: 14px/1.45 system-ui; }
|
||||
#progress { color: #e7bd54; }
|
||||
pre { white-space: pre-wrap; }
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<h1>Fixture regression</h1>
|
||||
<p id="progress">Подготовка…</p>
|
||||
<pre id="report"></pre>
|
||||
<script type="module" src="/src/fixture-runner.js"></script>
|
||||
</body>
|
||||
</html>
|
||||
@@ -110,6 +110,8 @@
|
||||
<select id="region-type">
|
||||
<option value="saleAnchor">Маркер магазина</option>
|
||||
<option value="storeHeader">Заголовок магазина</option>
|
||||
<option value="sellItemGrid">Сетка предметов Sell</option>
|
||||
<option value="buyItemGrid">Сетка предметов Buy</option>
|
||||
<option value="tooltipSearch">Зона поиска tooltip</option>
|
||||
<option value="tooltipAnchor">Якорь Price :</option>
|
||||
<option value="itemName">Название предмета</option>
|
||||
|
||||
@@ -12,6 +12,7 @@
|
||||
"tesseract.js": "7.0.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@playwright/test": "^1.62.1",
|
||||
"vite": "^7.3.6"
|
||||
}
|
||||
},
|
||||
@@ -474,6 +475,22 @@
|
||||
"node": "^22.20 || ^24.12 || >=25"
|
||||
}
|
||||
},
|
||||
"node_modules/@playwright/test": {
|
||||
"version": "1.62.1",
|
||||
"resolved": "https://registry.npmjs.org/@playwright/test/-/test-1.62.1.tgz",
|
||||
"integrity": "sha512-DTcUc8qii+cpHvtOwggMtBRMjKZHXYWdw8syRYu2vtzuq4Wxphqq4NfCs5Zt44L6mA8rfDfj+PHnxFc/FeK6mQ==",
|
||||
"dev": true,
|
||||
"license": "Apache-2.0",
|
||||
"dependencies": {
|
||||
"playwright": "1.62.1"
|
||||
},
|
||||
"bin": {
|
||||
"playwright": "cli.js"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=20"
|
||||
}
|
||||
},
|
||||
"node_modules/@rollup/rollup-android-arm-eabi": {
|
||||
"version": "4.62.4",
|
||||
"resolved": "https://registry.npmjs.org/@rollup/rollup-android-arm-eabi/-/rollup-android-arm-eabi-4.62.4.tgz",
|
||||
@@ -991,6 +1008,7 @@
|
||||
"integrity": "sha512-RvwwcruNjI1ncT5xRakeyS9Lf8lcItv34KD+aif+VH9kduAyfYBipGh12274xtenIPZ119/R9BdTBa8gAwSh0A==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"engines": {
|
||||
"node": ">=12"
|
||||
},
|
||||
@@ -998,6 +1016,53 @@
|
||||
"url": "https://github.com/sponsors/jonschlinkert"
|
||||
}
|
||||
},
|
||||
"node_modules/playwright": {
|
||||
"version": "1.62.1",
|
||||
"resolved": "https://registry.npmjs.org/playwright/-/playwright-1.62.1.tgz",
|
||||
"integrity": "sha512-0M+L3LAD8/nm554LOla9Ayx0j0tmFZ0FBcoQ7F1VuVHpM/XpiC8RcDzBQB8W5+hA8L22THxELzeF+2WcUzvcLg==",
|
||||
"dev": true,
|
||||
"license": "Apache-2.0",
|
||||
"dependencies": {
|
||||
"playwright-core": "1.62.1"
|
||||
},
|
||||
"bin": {
|
||||
"playwright": "cli.js"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=20"
|
||||
},
|
||||
"optionalDependencies": {
|
||||
"fsevents": "2.3.2"
|
||||
}
|
||||
},
|
||||
"node_modules/playwright-core": {
|
||||
"version": "1.62.1",
|
||||
"resolved": "https://registry.npmjs.org/playwright-core/-/playwright-core-1.62.1.tgz",
|
||||
"integrity": "sha512-wPYSwEBJY9GHraISXqyqtx0na0LpO3XEX7jNDhntbex7tzUS7kLnZsOlFruFJB4Hi/rhDMjXGqHewDZ68nYZVw==",
|
||||
"dev": true,
|
||||
"license": "Apache-2.0",
|
||||
"bin": {
|
||||
"playwright-core": "cli.js"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=20"
|
||||
}
|
||||
},
|
||||
"node_modules/playwright/node_modules/fsevents": {
|
||||
"version": "2.3.2",
|
||||
"resolved": "https://registry.npmjs.org/fsevents/-/fsevents-2.3.2.tgz",
|
||||
"integrity": "sha512-xiqMQR4xAeHTuB9uWm+fFRcIOgKBMiOBP+eXiyT7jsgVCq1bkVygt00oASowB7EdtpOHaaPgKt812P9ab+DDKA==",
|
||||
"dev": true,
|
||||
"hasInstallScript": true,
|
||||
"license": "MIT",
|
||||
"optional": true,
|
||||
"os": [
|
||||
"darwin"
|
||||
],
|
||||
"engines": {
|
||||
"node": "^8.16.0 || ^10.6.0 || >=11.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/postcss": {
|
||||
"version": "8.5.26",
|
||||
"resolved": "https://registry.npmjs.org/postcss/-/postcss-8.5.26.tgz",
|
||||
|
||||
@@ -6,13 +6,16 @@
|
||||
"scripts": {
|
||||
"dev": "vite --host 127.0.0.1",
|
||||
"build": "vite build",
|
||||
"test": "node --test"
|
||||
"test": "node --test",
|
||||
"test:fixtures": "playwright test",
|
||||
"check": "npm test && npm run test:fixtures && npm run build && git diff --check"
|
||||
},
|
||||
"dependencies": {
|
||||
"@techstark/opencv-js": "5.0.0-release.1",
|
||||
"tesseract.js": "7.0.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@playwright/test": "^1.62.1",
|
||||
"vite": "^7.3.6"
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,18 @@
|
||||
import { defineConfig } from "@playwright/test";
|
||||
|
||||
export default defineConfig({
|
||||
testDir: "./e2e",
|
||||
timeout: 15 * 60 * 1000,
|
||||
use: {
|
||||
baseURL: "http://127.0.0.1:4173",
|
||||
headless: true,
|
||||
screenshot: "only-on-failure",
|
||||
trace: "retain-on-failure",
|
||||
},
|
||||
webServer: {
|
||||
command: "node node_modules/vite/bin/vite.js --mode test --host 127.0.0.1 --port 4173",
|
||||
url: "http://127.0.0.1:4173/fixture.html",
|
||||
reuseExistingServer: false,
|
||||
timeout: 120000,
|
||||
},
|
||||
});
|
||||
@@ -0,0 +1,26 @@
|
||||
export function createCatalogResolver(fetchMatch, { minimumScore = 0.35 } = {}) {
|
||||
const cache = new Map();
|
||||
return async (item, alternatives = [item]) => {
|
||||
const unique = [...new Map(alternatives.map((candidate) => [candidate.name, candidate])).values()];
|
||||
const matches = await Promise.all(
|
||||
unique.map(async (candidate) => {
|
||||
if (!cache.has(candidate.name)) {
|
||||
cache.set(candidate.name, Promise.resolve(fetchMatch(candidate.name)).catch(() => null));
|
||||
}
|
||||
return { candidate, match: await cache.get(candidate.name) };
|
||||
}),
|
||||
);
|
||||
const best = matches
|
||||
.filter(({ match }) => match?.matchScore >= minimumScore)
|
||||
.sort((left, right) => right.match.matchScore - left.match.matchScore)[0];
|
||||
return best
|
||||
? {
|
||||
...best.candidate,
|
||||
itemId: best.match.id,
|
||||
name: best.match.name,
|
||||
iconUrl: best.match.iconUrl,
|
||||
matchScore: best.match.matchScore,
|
||||
}
|
||||
: item;
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,154 @@
|
||||
import { analyzeFrame, createAnalysisResult, loadImageCanvas } from "./frame-analyzer.js";
|
||||
import { createCatalogResolver } from "./catalog-resolver.js";
|
||||
import { ocrEditDistance } from "./parser.js";
|
||||
|
||||
const progress = document.querySelector("#progress");
|
||||
const report = document.querySelector("#report");
|
||||
window.__fixtureArtifacts = {};
|
||||
|
||||
function createResult(groupId, initial = {}) {
|
||||
return createAnalysisResult(
|
||||
{ id: `fixture:${groupId}`, name: groupId },
|
||||
initial,
|
||||
);
|
||||
}
|
||||
|
||||
function sameValues(actual, expected) {
|
||||
return actual.length === expected.length && actual.every((value, index) => value === expected[index]);
|
||||
}
|
||||
|
||||
function compareFixture(fixture, result) {
|
||||
const failures = [];
|
||||
const expected = fixture.expected;
|
||||
if (result.side !== expected.side) failures.push(`side: ${result.side || "null"} != ${expected.side}`);
|
||||
if (result.merchant !== expected.merchant) {
|
||||
failures.push(`merchant: ${result.merchant || "null"} != ${expected.merchant}`);
|
||||
}
|
||||
if (expected.item) {
|
||||
const item = result.items.find(
|
||||
(entry) => entry.quantity === expected.item.quantity && entry.priceAdena === expected.item.priceAdena,
|
||||
);
|
||||
if (!item) {
|
||||
failures.push(
|
||||
`item: missing ${expected.item.name}, quantity=${expected.item.quantity}, price=${expected.item.priceAdena}`,
|
||||
);
|
||||
} else if (item.name !== expected.item.name) {
|
||||
failures.push(`item.name: ${item.name} != ${expected.item.name}`);
|
||||
} else if (item.priceMode !== expected.item.priceMode) {
|
||||
failures.push(`item.priceMode: ${item.priceMode} != ${expected.item.priceMode}`);
|
||||
}
|
||||
}
|
||||
if (result.iconSlots) {
|
||||
const pending = result.iconSlots.filter((slot) => slot.status === "pending").map((slot) => slot.slotId);
|
||||
const found = result.iconSlots.filter((slot) => slot.status === "found").map((slot) => slot.slotId);
|
||||
if (!sameValues(pending, expected.pendingSlots)) {
|
||||
failures.push(`pendingSlots: [${pending}] != [${expected.pendingSlots}]`);
|
||||
}
|
||||
if (!sameValues(found, expected.foundSlots)) {
|
||||
failures.push(`foundSlots: [${found}] != [${expected.foundSlots}]`);
|
||||
}
|
||||
}
|
||||
return failures;
|
||||
}
|
||||
|
||||
async function run() {
|
||||
const [manifest, fixtureCalibration, catalog] = await Promise.all([
|
||||
fetch("/data/fixtures.json").then((response) => response.json()),
|
||||
fetch("/data/fixture-calibration.json").then((response) => response.json()),
|
||||
fetch("/data/fixture-assets/catalog.json").then((response) => response.json()),
|
||||
]);
|
||||
const calibration = { ...fixtureCalibration };
|
||||
const [saleTemplate, tooltipTemplate] = await Promise.all([
|
||||
loadImageCanvas(`/data/${calibration.saleAnchor.image}`),
|
||||
loadImageCanvas(`/data/${calibration.tooltipAnchor.image}`),
|
||||
]);
|
||||
const catalogItems = Object.values(catalog.items);
|
||||
const resolveCatalogItem = createCatalogResolver(async (name) => {
|
||||
const query = name.toLocaleLowerCase("en-US");
|
||||
const best = catalogItems
|
||||
.map((entry) => {
|
||||
const distance = ocrEditDistance(query, entry.name.toLocaleLowerCase("en-US"));
|
||||
return { entry, similarity: 1 - distance / Math.max(query.length, entry.name.length) };
|
||||
})
|
||||
.sort((left, right) => right.similarity - left.similarity)[0];
|
||||
return best ? { ...best.entry, matchScore: best.similarity } : null;
|
||||
});
|
||||
const groups = new Map();
|
||||
manifest.cases.forEach((fixture) => {
|
||||
const group = groups.get(fixture.groupId) || [];
|
||||
group.push(fixture);
|
||||
groups.set(fixture.groupId, group);
|
||||
});
|
||||
const fixtureFilter = new URLSearchParams(location.search).get("fixture");
|
||||
const expectedTotal = fixtureFilter
|
||||
? manifest.cases.filter(
|
||||
(fixture) => fixture.id === fixtureFilter || fixture.groupId === fixtureFilter,
|
||||
).length
|
||||
: manifest.cases.length;
|
||||
const results = [];
|
||||
const failures = [];
|
||||
let completed = 0;
|
||||
|
||||
for (const [groupId, fixtures] of groups) {
|
||||
let result = null;
|
||||
for (const fixture of fixtures.sort((left, right) => left.order - right.order)) {
|
||||
if (fixtureFilter && fixture.id !== fixtureFilter && fixture.groupId !== fixtureFilter) continue;
|
||||
if (!result || fixture.mode === "standalone") {
|
||||
result = createResult(groupId, fixture.initialState || {});
|
||||
}
|
||||
if (fixture.mode === "contextual" && !result.activeShopKey) {
|
||||
result = createResult(groupId, fixture.initialState);
|
||||
}
|
||||
if (fixture.mode === "contextual" && !result.iconSlots.length) {
|
||||
const icon = await loadImageCanvas(catalog.items[fixture.catalogAssetId].iconUrl);
|
||||
result.iconSlots = fixture.initialState.pendingSlots.map((slotId) => ({
|
||||
slotId,
|
||||
status: "pending",
|
||||
itemKey: null,
|
||||
thumbnail: icon,
|
||||
frameRect: { x: 0, y: 0, width: 32, height: 32 },
|
||||
}));
|
||||
}
|
||||
progress.textContent = `${completed + 1}/${manifest.cases.length}: ${fixture.id}`;
|
||||
const frame = await loadImageCanvas(`/data/${fixture.file}`);
|
||||
await analyzeFrame(frame, result, saleTemplate, tooltipTemplate, {
|
||||
calibration,
|
||||
resolveCatalogItem,
|
||||
catalogAlternatives: true,
|
||||
});
|
||||
const fixtureFailures = compareFixture(fixture, result);
|
||||
if (fixtureFailures.length) {
|
||||
window.__fixtureArtifacts[fixture.id] = {
|
||||
preview: result.diagnostics?.preview || null,
|
||||
stages: result.diagnostics?.stages
|
||||
.filter(({ image }) => image)
|
||||
.map(({ key, image }) => ({ key, image })) || [],
|
||||
};
|
||||
}
|
||||
results.push({
|
||||
id: fixture.id,
|
||||
saleFound: result.saleFound,
|
||||
score: result.score,
|
||||
side: result.side || null,
|
||||
merchant: result.merchant || null,
|
||||
stages: result.diagnostics?.stages.map(({ key, status, detail }) => ({ key, status, detail })),
|
||||
items: result.items.map(({ name, quantity, priceAdena }) => ({ name, quantity, priceAdena })),
|
||||
failures: fixtureFailures,
|
||||
});
|
||||
failures.push(...fixtureFailures.map((message) => `${fixture.id}: ${message}`));
|
||||
completed += 1;
|
||||
}
|
||||
}
|
||||
|
||||
const output = { completed, total: expectedTotal, failures, results };
|
||||
progress.textContent = failures.length ? `Ошибок: ${failures.length}` : `Пройдено: ${completed}/${expectedTotal}`;
|
||||
report.textContent = JSON.stringify(output, null, 2);
|
||||
return output;
|
||||
}
|
||||
|
||||
window.__fixtureRun = run().catch((error) => {
|
||||
const output = { completed: 0, total: 0, failures: [error.stack || error.message], results: [] };
|
||||
progress.textContent = "Ошибка запуска fixture";
|
||||
report.textContent = JSON.stringify(output, null, 2);
|
||||
return output;
|
||||
});
|
||||
@@ -0,0 +1,545 @@
|
||||
import {
|
||||
combineRects,
|
||||
isRectInside,
|
||||
parseStoreHeaderText,
|
||||
parseTooltipText,
|
||||
resolveFieldRect,
|
||||
} from "./parser.js";
|
||||
import {
|
||||
detectOccupiedSlots,
|
||||
findAnchor,
|
||||
fitOcrTextRect,
|
||||
matchCatalogIconToSlot,
|
||||
recognizeText,
|
||||
recognizeTextRows,
|
||||
} from "./vision.js";
|
||||
|
||||
export function cropCanvas(source, rect) {
|
||||
const canvas = document.createElement("canvas");
|
||||
canvas.width = rect.width;
|
||||
canvas.height = rect.height;
|
||||
canvas
|
||||
.getContext("2d", { willReadFrequently: true })
|
||||
.drawImage(source, rect.x, rect.y, rect.width, rect.height, 0, 0, rect.width, rect.height);
|
||||
return canvas;
|
||||
}
|
||||
|
||||
export function canvasPreview(canvas, maxWidth = 760) {
|
||||
const scale = Math.min(1, maxWidth / canvas.width);
|
||||
const preview = document.createElement("canvas");
|
||||
preview.width = Math.max(1, Math.round(canvas.width * scale));
|
||||
preview.height = Math.max(1, Math.round(canvas.height * scale));
|
||||
preview.getContext("2d").drawImage(canvas, 0, 0, preview.width, preview.height);
|
||||
return preview.toDataURL("image/jpeg", 0.78);
|
||||
}
|
||||
|
||||
function cropPreview(frame, rect) {
|
||||
return isRectInside(rect, frame.width, frame.height)
|
||||
? cropCanvas(frame, rect).toDataURL("image/png")
|
||||
: null;
|
||||
}
|
||||
|
||||
export function boxColor(status) {
|
||||
if (status === "found") return "oklch(0.72 0.16 155)";
|
||||
if (status === "pending") return "oklch(0.8 0.16 85)";
|
||||
if (status === "not-found") return "oklch(0.68 0.18 25)";
|
||||
return "oklch(0.72 0.13 250)";
|
||||
}
|
||||
|
||||
function annotatedPreview(frame, boxes, slots = []) {
|
||||
const scale = Math.min(1, 760 / frame.width);
|
||||
const preview = document.createElement("canvas");
|
||||
preview.width = Math.max(1, Math.round(frame.width * scale));
|
||||
preview.height = Math.max(1, Math.round(frame.height * scale));
|
||||
const context = preview.getContext("2d");
|
||||
context.drawImage(frame, 0, 0, preview.width, preview.height);
|
||||
context.font = "12px system-ui";
|
||||
|
||||
slots.forEach(({ frameRect, status }) => {
|
||||
const color = boxColor(status);
|
||||
context.strokeStyle = color;
|
||||
context.fillStyle = status === "found" ? "rgb(30 190 105 / 0.16)" : "rgb(245 190 40 / 0.14)";
|
||||
context.lineWidth = 2;
|
||||
context.fillRect(
|
||||
frameRect.x * scale,
|
||||
frameRect.y * scale,
|
||||
frameRect.width * scale,
|
||||
frameRect.height * scale,
|
||||
);
|
||||
context.strokeRect(
|
||||
frameRect.x * scale,
|
||||
frameRect.y * scale,
|
||||
frameRect.width * scale,
|
||||
frameRect.height * scale,
|
||||
);
|
||||
});
|
||||
|
||||
boxes.forEach(({ rect, label, status }) => {
|
||||
const color = boxColor(status);
|
||||
const x = rect.x * scale;
|
||||
const y = rect.y * scale;
|
||||
const width = rect.width * scale;
|
||||
const height = rect.height * scale;
|
||||
context.strokeStyle = color;
|
||||
context.fillStyle = color;
|
||||
context.lineWidth = 2;
|
||||
context.strokeRect(x, y, width, height);
|
||||
const labelWidth = context.measureText(label).width + 10;
|
||||
const labelY = Math.max(0, y - 20);
|
||||
context.fillRect(x, labelY, labelWidth, 20);
|
||||
context.fillStyle = "oklch(0.98 0.004 250)";
|
||||
context.fillText(label, x + 5, labelY + 14);
|
||||
});
|
||||
|
||||
return preview.toDataURL("image/jpeg", 0.82);
|
||||
}
|
||||
|
||||
function addDiagnosticStage(diagnostics, key, label, status, detail, image = null) {
|
||||
diagnostics.stages.push({ key, label, status, detail, image });
|
||||
}
|
||||
|
||||
function addSkippedStages(diagnostics, stages, reason) {
|
||||
stages.forEach(([key, label]) => {
|
||||
addDiagnosticStage(diagnostics, key, label, "skipped", `Не запускалось: ${reason}`);
|
||||
});
|
||||
}
|
||||
|
||||
function finishDiagnostics(result, frame, diagnostics) {
|
||||
diagnostics.preview = annotatedPreview(frame, diagnostics.boxes, result.iconSlots);
|
||||
result.diagnostics = diagnostics;
|
||||
}
|
||||
|
||||
function clearRememberedShop(result) {
|
||||
result.side = "";
|
||||
result.merchant = "";
|
||||
result.storeRawText = "";
|
||||
result.storeConfidence = 0;
|
||||
resetShopProgress(result);
|
||||
}
|
||||
|
||||
export function resetShopProgress(result) {
|
||||
result.activeShopKey = "";
|
||||
result.iconSlots = [];
|
||||
result.gridCaptured = false;
|
||||
result.slotMatchStatus = "";
|
||||
}
|
||||
|
||||
export function createAnalysisResult(source, initial = {}) {
|
||||
return {
|
||||
sourceId: source.id,
|
||||
source: source.name,
|
||||
saleFound: false,
|
||||
score: 0,
|
||||
side: "",
|
||||
merchant: "",
|
||||
storeRawText: "",
|
||||
storeConfidence: 0,
|
||||
items: [],
|
||||
misses: 0,
|
||||
activeShopKey: "",
|
||||
iconSlots: [],
|
||||
gridCaptured: false,
|
||||
slotMatchStatus: "",
|
||||
error: null,
|
||||
...initial,
|
||||
};
|
||||
}
|
||||
|
||||
function shopKey(side, merchant) {
|
||||
return `${side}:${merchant.toLocaleLowerCase("en-US")}`;
|
||||
}
|
||||
|
||||
export function activateShop(result, side, merchant) {
|
||||
const key = shopKey(side, merchant);
|
||||
if (result.activeShopKey && result.activeShopKey !== key) resetShopProgress(result);
|
||||
result.activeShopKey = key;
|
||||
}
|
||||
|
||||
function reconcileOccupiedSlots(result, detected) {
|
||||
const previous = new Map((result.iconSlots || []).map((slot) => [slot.slotId, slot]));
|
||||
result.iconSlots = detected.map((slot) => ({
|
||||
...slot,
|
||||
status: previous.get(slot.slotId)?.status === "found" ? "found" : "pending",
|
||||
itemKey: previous.get(slot.slotId)?.itemKey || null,
|
||||
}));
|
||||
}
|
||||
|
||||
export function loadImageCanvas(url) {
|
||||
return new Promise((resolve, reject) => {
|
||||
const image = new Image();
|
||||
image.crossOrigin = "anonymous";
|
||||
image.onload = () => {
|
||||
const canvas = document.createElement("canvas");
|
||||
canvas.width = image.naturalWidth;
|
||||
canvas.height = image.naturalHeight;
|
||||
canvas.getContext("2d", { willReadFrequently: true }).drawImage(image, 0, 0);
|
||||
resolve(canvas);
|
||||
};
|
||||
image.onerror = () => reject(new Error(`Не удалось загрузить иконку ${url}`));
|
||||
image.src = url;
|
||||
});
|
||||
}
|
||||
|
||||
function captureCleanGrid(frame, result, saleMatch, calibration) {
|
||||
if (result.gridCaptured) return;
|
||||
const region =
|
||||
result.side === "sell" ? calibration.sellItemGridRegion : calibration.buyItemGridRegion;
|
||||
if (!region) return;
|
||||
const gridRect = resolveFieldRect(saleMatch, region);
|
||||
if (!isRectInside(gridRect, frame.width, frame.height)) return;
|
||||
reconcileOccupiedSlots(result, detectOccupiedSlots(frame, gridRect));
|
||||
result.gridCaptured = true;
|
||||
}
|
||||
|
||||
export function recordSaleMiss(result) {
|
||||
result.misses += 1;
|
||||
if (result.misses >= 3) clearRememberedShop(result);
|
||||
}
|
||||
|
||||
async function markMatchedSlot(result, item, key) {
|
||||
if (!item.iconUrl || !result.iconSlots?.some((slot) => slot.status === "pending")) return null;
|
||||
try {
|
||||
const icon = await loadImageCanvas(item.iconUrl);
|
||||
const slotId = matchCatalogIconToSlot(result.iconSlots, icon);
|
||||
if (slotId === null) {
|
||||
result.slotMatchStatus = "ambiguous";
|
||||
return null;
|
||||
}
|
||||
const slot = result.iconSlots.find((entry) => entry.slotId === slotId);
|
||||
slot.status = "found";
|
||||
slot.itemKey = key;
|
||||
result.slotMatchStatus = "matched";
|
||||
return slotId;
|
||||
} catch {
|
||||
result.slotMatchStatus = "unavailable";
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
export async function analyzeFrame(
|
||||
frame,
|
||||
result,
|
||||
saleTemplate,
|
||||
tooltipTemplate,
|
||||
{
|
||||
calibration,
|
||||
source = null,
|
||||
resolveCatalogItem = async (item) => item,
|
||||
catalogAlternatives = false,
|
||||
enqueueObservation = null,
|
||||
},
|
||||
) {
|
||||
const diagnostics = { boxes: [], stages: [], preview: null };
|
||||
addDiagnosticStage(
|
||||
diagnostics,
|
||||
"capture",
|
||||
"Захват кадра",
|
||||
"found",
|
||||
`${frame.width} × ${frame.height}`,
|
||||
canvasPreview(frame, 240),
|
||||
);
|
||||
result.error = null;
|
||||
|
||||
const saleMatch = await findAnchor(frame, saleTemplate);
|
||||
const saleRect = {
|
||||
x: saleMatch.x,
|
||||
y: saleMatch.y,
|
||||
width: saleTemplate.width,
|
||||
height: saleTemplate.height,
|
||||
};
|
||||
const saleFound = saleMatch.score >= calibration.threshold;
|
||||
result.score = saleMatch.score;
|
||||
result.saleFound = saleFound;
|
||||
diagnostics.boxes.push({
|
||||
rect: saleRect,
|
||||
label: "Маркер магазина",
|
||||
status: saleFound ? "found" : "not-found",
|
||||
});
|
||||
addDiagnosticStage(
|
||||
diagnostics,
|
||||
"sale",
|
||||
"Маркер магазина",
|
||||
saleFound ? "found" : "not-found",
|
||||
`${(saleMatch.score * 100).toFixed(1)}%, нужно ${(calibration.threshold * 100).toFixed(0)}%`,
|
||||
cropPreview(frame, saleRect),
|
||||
);
|
||||
|
||||
if (!saleFound) {
|
||||
recordSaleMiss(result);
|
||||
addSkippedStages(
|
||||
diagnostics,
|
||||
[
|
||||
["storeHeader", "Заголовок магазина"],
|
||||
["tooltip", "Якорь Price :"],
|
||||
["itemName", "Название предмета"],
|
||||
["itemPrice", "Цена предмета"],
|
||||
],
|
||||
"маркер магазина не прошёл порог",
|
||||
);
|
||||
finishDiagnostics(result, frame, diagnostics);
|
||||
return null;
|
||||
}
|
||||
|
||||
result.misses = 0;
|
||||
const storeHeaderRect = resolveFieldRect(saleMatch, calibration.storeHeaderRegion);
|
||||
const tooltipSearchRect = resolveFieldRect(saleMatch, calibration.tooltipSearchRegion);
|
||||
const storeHeaderBox = { rect: storeHeaderRect, label: "Заголовок", status: "info" };
|
||||
let recognizedHeader = null;
|
||||
diagnostics.boxes.push(storeHeaderBox);
|
||||
diagnostics.boxes.push({ rect: tooltipSearchRect, label: "Зона tooltip", status: "info" });
|
||||
|
||||
if (isRectInside(storeHeaderRect, frame.width, frame.height)) {
|
||||
const headerOcr = await recognizeText(frame, storeHeaderRect, {
|
||||
singleLine: true,
|
||||
preprocess: "luminance",
|
||||
scale: 4,
|
||||
whitelist: "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789()- ",
|
||||
});
|
||||
const header = parseStoreHeaderText(headerOcr.text);
|
||||
const rememberedStore = !header && result.side && result.merchant;
|
||||
storeHeaderBox.status = header ? "found" : rememberedStore ? "info" : "not-found";
|
||||
if (header) {
|
||||
recognizedHeader = header;
|
||||
result.side = header.side;
|
||||
result.merchant = header.merchant;
|
||||
result.storeRawText = header.rawText;
|
||||
result.storeConfidence = headerOcr.confidence;
|
||||
activateShop(result, header.side, header.merchant);
|
||||
storeHeaderBox.label = `${header.side === "buy" ? "Buy" : "Sell"} · ${header.merchant}`;
|
||||
} else if (rememberedStore) {
|
||||
storeHeaderBox.label = `Последний: ${result.side === "buy" ? "Buy" : "Sell"} · ${result.merchant}`;
|
||||
}
|
||||
addDiagnosticStage(
|
||||
diagnostics,
|
||||
"storeHeader",
|
||||
"Заголовок магазина",
|
||||
header ? "found" : rememberedStore ? "skipped" : "not-found",
|
||||
header
|
||||
? `OCR ${headerOcr.confidence}%: ${header.side === "buy" ? "Покупка" : "Продажа"}, ${header.merchant}`
|
||||
: rememberedStore
|
||||
? `Заголовок закрыт или не читается. Используется последний магазин: ${result.side === "buy" ? "Покупка" : "Продажа"}, ${result.merchant}`
|
||||
: `OCR ${headerOcr.confidence}%: «${headerOcr.text || "пусто"}» — сначала нужно распознать тип и имя`,
|
||||
cropPreview(frame, storeHeaderRect),
|
||||
);
|
||||
} else {
|
||||
const rememberedStore = result.side && result.merchant;
|
||||
storeHeaderBox.status = rememberedStore ? "info" : "not-found";
|
||||
addDiagnosticStage(
|
||||
diagnostics,
|
||||
"storeHeader",
|
||||
"Заголовок магазина",
|
||||
rememberedStore ? "skipped" : "error",
|
||||
rememberedStore
|
||||
? `Область вне кадра. Используется последний магазин: ${result.side === "buy" ? "Покупка" : "Продажа"}, ${result.merchant}`
|
||||
: "Область вышла за границы кадра, а ранее распознанного магазина нет",
|
||||
);
|
||||
}
|
||||
|
||||
if (!isRectInside(tooltipSearchRect, frame.width, frame.height)) {
|
||||
addDiagnosticStage(
|
||||
diagnostics,
|
||||
"tooltip",
|
||||
"Якорь Price :",
|
||||
"error",
|
||||
"Зона поиска tooltip вышла за границы кадра",
|
||||
);
|
||||
addSkippedStages(
|
||||
diagnostics,
|
||||
[
|
||||
["itemName", "Название предмета"],
|
||||
["itemPrice", "Цена предмета"],
|
||||
],
|
||||
"зона поиска tooltip недоступна",
|
||||
);
|
||||
finishDiagnostics(result, frame, diagnostics);
|
||||
return null;
|
||||
}
|
||||
|
||||
const tooltipSearch = cropCanvas(frame, tooltipSearchRect);
|
||||
const tooltipMatch = await findAnchor(tooltipSearch, tooltipTemplate);
|
||||
const tooltipRect = {
|
||||
x: tooltipSearchRect.x + tooltipMatch.x,
|
||||
y: tooltipSearchRect.y + tooltipMatch.y,
|
||||
width: tooltipTemplate.width,
|
||||
height: tooltipTemplate.height,
|
||||
};
|
||||
const tooltipFound = tooltipMatch.score >= calibration.threshold;
|
||||
diagnostics.boxes.push({
|
||||
rect: tooltipRect,
|
||||
label: "Price :",
|
||||
status: tooltipFound ? "found" : "not-found",
|
||||
});
|
||||
addDiagnosticStage(
|
||||
diagnostics,
|
||||
"tooltip",
|
||||
"Якорь Price :",
|
||||
tooltipFound ? "found" : "not-found",
|
||||
`${(tooltipMatch.score * 100).toFixed(1)}%, нужно ${(calibration.threshold * 100).toFixed(0)}%`,
|
||||
cropPreview(frame, tooltipRect),
|
||||
);
|
||||
|
||||
if (!tooltipFound) {
|
||||
if (recognizedHeader && result.activeShopKey) {
|
||||
captureCleanGrid(frame, result, saleMatch, calibration);
|
||||
}
|
||||
addSkippedStages(
|
||||
diagnostics,
|
||||
[
|
||||
["itemName", "Название предмета"],
|
||||
["itemPrice", "Цена предмета"],
|
||||
],
|
||||
"якорь Price : не найден",
|
||||
);
|
||||
finishDiagnostics(result, frame, diagnostics);
|
||||
return null;
|
||||
}
|
||||
|
||||
if (!result.side || !result.merchant) {
|
||||
addSkippedStages(
|
||||
diagnostics,
|
||||
[
|
||||
["itemName", "Название предмета"],
|
||||
["itemPrice", "Цена предмета"],
|
||||
],
|
||||
"сначала нужно увидеть заголовок магазина без tooltip",
|
||||
);
|
||||
finishDiagnostics(result, frame, diagnostics);
|
||||
return null;
|
||||
}
|
||||
|
||||
const calibratedNameRect = resolveFieldRect(tooltipRect, calibration.itemNameRegion);
|
||||
const calibratedPriceRect = resolveFieldRect(tooltipRect, calibration.itemPriceRegion);
|
||||
const itemBox = {
|
||||
rect: combineRects(calibratedNameRect, calibratedPriceRect),
|
||||
label: "Название + цена",
|
||||
status: "pending",
|
||||
};
|
||||
diagnostics.boxes.push(itemBox);
|
||||
|
||||
const nameRect = fitOcrTextRect(frame, calibratedNameRect);
|
||||
const priceRect = fitOcrTextRect(frame, calibratedPriceRect, { includeYellow: true });
|
||||
if (!nameRect || !priceRect) {
|
||||
itemBox.status = "not-found";
|
||||
addDiagnosticStage(
|
||||
diagnostics,
|
||||
"itemName",
|
||||
"Название предмета",
|
||||
nameRect ? "found" : "not-found",
|
||||
nameRect ? "Граница текста найдена" : "Светлый текст не найден или строка вне кадра",
|
||||
nameRect ? cropPreview(frame, nameRect) : null,
|
||||
);
|
||||
addDiagnosticStage(
|
||||
diagnostics,
|
||||
"itemPrice",
|
||||
"Цена предмета",
|
||||
priceRect ? "found" : "not-found",
|
||||
priceRect ? "Граница текста найдена" : "Светлый текст не найден или строка вне кадра",
|
||||
priceRect ? cropPreview(frame, priceRect) : null,
|
||||
);
|
||||
finishDiagnostics(result, frame, diagnostics);
|
||||
return null;
|
||||
}
|
||||
|
||||
const itemRect = combineRects(nameRect, priceRect);
|
||||
itemBox.rect = itemRect;
|
||||
const itemOcr = await recognizeTextRows(frame, [
|
||||
{ rect: nameRect },
|
||||
{ rect: priceRect, includeYellow: true },
|
||||
]);
|
||||
let item = parseTooltipText(itemOcr.text);
|
||||
if (item) {
|
||||
item.priceMode = /for\s*each/i.test(item.rawText) ? "each" : "price";
|
||||
if (result.side === "buy" && item.quantity === 1) item.quantity = 0;
|
||||
}
|
||||
itemBox.status = item ? "found" : "pending";
|
||||
addDiagnosticStage(
|
||||
diagnostics,
|
||||
"itemName",
|
||||
"Название предмета",
|
||||
item ? "found" : "not-found",
|
||||
item ? `OCR ${itemOcr.confidence}%: ${item.name}` : `OCR не разобран: ${itemOcr.text || "пусто"}`,
|
||||
cropPreview(frame, nameRect),
|
||||
);
|
||||
addDiagnosticStage(
|
||||
diagnostics,
|
||||
"itemPrice",
|
||||
"Цена предмета",
|
||||
item ? "found" : "not-found",
|
||||
item
|
||||
? `OCR ${itemOcr.confidence}%: ${item.priceAdena.toLocaleString("en-US")} Adena`
|
||||
: `OCR не разобран: ${itemOcr.text || "пусто"}`,
|
||||
cropPreview(frame, priceRect),
|
||||
);
|
||||
finishDiagnostics(result, frame, diagnostics);
|
||||
if (!item) return null;
|
||||
|
||||
const alternateNameOcr = catalogAlternatives
|
||||
? await Promise.all([
|
||||
recognizeText(frame, nameRect, { singleLine: true, preprocess: false, scale: 3 }),
|
||||
recognizeText(frame, nameRect, { singleLine: true, preprocess: "luminance", scale: 3 }),
|
||||
])
|
||||
: [];
|
||||
const alternatives = [
|
||||
item,
|
||||
...alternateNameOcr
|
||||
.map(({ text }) => parseTooltipText(`${text}\n${item.priceAdena}`))
|
||||
.filter(Boolean)
|
||||
.map((candidate) =>
|
||||
result.side === "buy" && candidate.quantity === 1
|
||||
? { ...candidate, quantity: 0, priceMode: item.priceMode }
|
||||
: { ...candidate, priceMode: item.priceMode },
|
||||
),
|
||||
];
|
||||
item = await resolveCatalogItem(item, alternatives);
|
||||
const confidence = itemOcr.confidence;
|
||||
const key = `${result.side}:${result.merchant.toLocaleLowerCase("en-US")}:${item.name.toLocaleLowerCase("en-US")}:${item.quantity ?? ""}:${item.priceAdena}`;
|
||||
const existing = result.items.find((entry) => entry.key === key);
|
||||
const seenAt = new Date().toISOString();
|
||||
let matchedSlotId = null;
|
||||
|
||||
if (source && enqueueObservation) {
|
||||
await enqueueObservation({
|
||||
observationId: crypto.randomUUID(),
|
||||
seenAt,
|
||||
sourceId: source.id,
|
||||
source: source.name.slice(0, 256),
|
||||
side: result.side || null,
|
||||
merchant: result.merchant.slice(0, 128),
|
||||
storeRawText: result.storeRawText.slice(0, 4096),
|
||||
storeConfidence: Number.isFinite(result.storeConfidence)
|
||||
? Math.max(0, Math.min(100, Math.round(result.storeConfidence)))
|
||||
: 0,
|
||||
storeScore: Number.isFinite(saleMatch.score)
|
||||
? Math.max(0, Math.min(1, saleMatch.score))
|
||||
: 0,
|
||||
itemId: item.itemId ?? null,
|
||||
name: item.name.slice(0, 256),
|
||||
quantity: item.quantity,
|
||||
priceAdena: item.priceAdena,
|
||||
rawText: item.rawText.slice(0, 4096),
|
||||
key: key.slice(0, 512),
|
||||
confidence: Number.isFinite(confidence) ? Math.max(0, Math.min(100, confidence)) : 0,
|
||||
});
|
||||
}
|
||||
|
||||
const alreadyMatched = result.iconSlots?.some((slot) => slot.itemKey === key);
|
||||
if (!alreadyMatched) matchedSlotId = await markMatchedSlot(result, item, key);
|
||||
if (matchedSlotId !== null) finishDiagnostics(result, frame, diagnostics);
|
||||
|
||||
if (existing) {
|
||||
existing.seenAt = seenAt;
|
||||
existing.confidence = confidence;
|
||||
return { item: existing, added: false, matchedSlotId };
|
||||
}
|
||||
|
||||
const storedItem = {
|
||||
...item,
|
||||
side: result.side,
|
||||
merchant: result.merchant,
|
||||
key,
|
||||
confidence,
|
||||
seenAt,
|
||||
};
|
||||
result.items.push(storedItem);
|
||||
return { item: storedItem, added: true, matchedSlotId };
|
||||
}
|
||||
@@ -1,23 +1,26 @@
|
||||
import "./style.css";
|
||||
import {
|
||||
combineRects,
|
||||
displayCaptureIssue,
|
||||
isCalibrationReady,
|
||||
isRectInside,
|
||||
parseStoreHeaderText,
|
||||
parseTooltipText,
|
||||
rectFromPoints,
|
||||
resolveFieldRect,
|
||||
toExportableResults,
|
||||
} from "./parser.js";
|
||||
import { findAnchor, onOcrProgress, terminateVision } from "./vision.js";
|
||||
import {
|
||||
findAnchor,
|
||||
onOcrProgress,
|
||||
recognizeText,
|
||||
terminateVision,
|
||||
} from "./vision.js";
|
||||
analyzeFrame,
|
||||
boxColor,
|
||||
canvasPreview,
|
||||
createAnalysisResult,
|
||||
cropCanvas,
|
||||
loadImageCanvas,
|
||||
} from "./frame-analyzer.js";
|
||||
import { createMarketOutbox } from "./market-outbox.js";
|
||||
import { createCatalogResolver } from "./catalog-resolver.js";
|
||||
|
||||
const STORAGE_KEY = "l2-market-parser.calibration.v4";
|
||||
const CALIBRATION_DISPLAY_SCALE = 2;
|
||||
const appHosted = location.pathname.startsWith("/l2/market-audit/");
|
||||
const embedded = new URLSearchParams(location.search).has("embedded");
|
||||
document.documentElement.dataset.theme =
|
||||
@@ -26,6 +29,8 @@ if (embedded) document.documentElement.classList.add("is-embedded");
|
||||
const regionNames = {
|
||||
saleAnchor: "Маркер магазина",
|
||||
storeHeader: "Заголовок магазина",
|
||||
sellItemGrid: "Сетка предметов Sell",
|
||||
buyItemGrid: "Сетка предметов Buy",
|
||||
tooltipSearch: "Зона поиска tooltip",
|
||||
tooltipAnchor: "Якорь Price :",
|
||||
itemName: "Название предмета",
|
||||
@@ -36,6 +41,8 @@ function emptyCalibration() {
|
||||
return {
|
||||
saleAnchor: null,
|
||||
storeHeaderRegion: null,
|
||||
sellItemGridRegion: null,
|
||||
buyItemGridRegion: null,
|
||||
tooltipSearchRegion: null,
|
||||
tooltipAnchor: null,
|
||||
itemNameRegion: null,
|
||||
@@ -68,7 +75,6 @@ const state = {
|
||||
collecting: false,
|
||||
};
|
||||
const marketOutbox = createMarketOutbox(appHosted ? "/api/l2/market/import" : undefined);
|
||||
const catalogMatches = new Map();
|
||||
|
||||
const elements = {
|
||||
status: document.querySelector("#global-status"),
|
||||
@@ -251,15 +257,31 @@ function drawSourceOverlay(source, result) {
|
||||
const context = canvas.getContext("2d");
|
||||
context.clearRect(0, 0, canvas.width, canvas.height);
|
||||
context.lineWidth = Math.max(3, canvas.width / 500);
|
||||
result?.iconSlots?.forEach(({ frameRect, status: slotStatus }) => {
|
||||
context.strokeStyle = boxColor(slotStatus);
|
||||
context.fillStyle = slotStatus === "found" ? "rgb(30 190 105 / 0.16)" : "rgb(245 190 40 / 0.14)";
|
||||
context.fillRect(frameRect.x, frameRect.y, frameRect.width, frameRect.height);
|
||||
context.strokeRect(frameRect.x, frameRect.y, frameRect.width, frameRect.height);
|
||||
});
|
||||
result?.diagnostics?.boxes.forEach(({ rect, status: boxStatus }) => {
|
||||
context.strokeStyle = boxColor(boxStatus);
|
||||
context.strokeRect(rect.x, rect.y, rect.width, rect.height);
|
||||
});
|
||||
|
||||
const itemBox = result?.diagnostics?.boxes.find(({ label }) => label === "Название + цена");
|
||||
const totalSlots = result?.iconSlots?.length || 0;
|
||||
const foundSlots = result?.iconSlots?.filter(({ status: slotStatus }) => slotStatus === "found").length || 0;
|
||||
const slotText = result?.slotMatchStatus === "ambiguous"
|
||||
? "Иконка неоднозначна"
|
||||
: totalSlots
|
||||
? `${foundSlots}/${totalSlots} иконок распознано`
|
||||
: "Жду чистый кадр сетки";
|
||||
const progressChip = totalSlots
|
||||
? `<span data-tone="${foundSlots === totalSlots ? "found" : "pending"}">${slotText}</span>`
|
||||
: `<span data-tone="${itemBox?.status === "found" ? "found" : "pending"}">${itemBox?.status === "found" ? "Предмет распознан" : "Предмет ожидает"}</span>`;
|
||||
status.innerHTML = `
|
||||
<span data-tone="${result?.saleFound ? "found" : "pending"}">${result?.saleFound ? "Магазин найден" : "Ищу магазин"}</span>
|
||||
<span data-tone="${itemBox?.status === "found" ? "found" : "pending"}">${itemBox?.status === "found" ? "Предмет распознан" : "Предмет ожидает"}</span>
|
||||
${progressChip}
|
||||
`;
|
||||
}
|
||||
|
||||
@@ -376,7 +398,7 @@ async function alignReference(reference) {
|
||||
reference.tooltipScore = null;
|
||||
if (!state.calibration.saleAnchor?.image) return;
|
||||
|
||||
const saleTemplate = await imageToCanvas(state.calibration.saleAnchor.image);
|
||||
const saleTemplate = await loadImageCanvas(state.calibration.saleAnchor.image);
|
||||
const saleMatch = await findAnchor(reference.canvas, saleTemplate);
|
||||
reference.saleScore = saleMatch.score;
|
||||
if (saleMatch.score < state.calibration.threshold) return;
|
||||
@@ -391,7 +413,7 @@ async function alignReference(reference) {
|
||||
|
||||
const searchRect = resolveFieldRect(reference.saleAnchor, state.calibration.tooltipSearchRegion);
|
||||
if (!isRectInside(searchRect, reference.canvas.width, reference.canvas.height)) return;
|
||||
const tooltipTemplate = await imageToCanvas(state.calibration.tooltipAnchor.image);
|
||||
const tooltipTemplate = await loadImageCanvas(state.calibration.tooltipAnchor.image);
|
||||
const tooltipMatch = await findAnchor(cropCanvas(reference.canvas, searchRect), tooltipTemplate);
|
||||
reference.tooltipScore = tooltipMatch.score;
|
||||
if (tooltipMatch.score < state.calibration.threshold) return;
|
||||
@@ -412,6 +434,8 @@ async function selectReference(referenceId) {
|
||||
const canvas = reference.canvas;
|
||||
elements.canvas.width = canvas.width;
|
||||
elements.canvas.height = canvas.height;
|
||||
elements.canvas.style.width = `${canvas.width * CALIBRATION_DISPLAY_SCALE}px`;
|
||||
elements.canvas.style.height = `${canvas.height * CALIBRATION_DISPLAY_SCALE}px`;
|
||||
elements.canvas.hidden = false;
|
||||
elements.canvasEmpty.hidden = true;
|
||||
elements.testCalibration.disabled = false;
|
||||
@@ -562,82 +586,6 @@ function canvasPoint(event) {
|
||||
};
|
||||
}
|
||||
|
||||
function cropCanvas(source, rect) {
|
||||
const canvas = document.createElement("canvas");
|
||||
canvas.width = rect.width;
|
||||
canvas.height = rect.height;
|
||||
canvas
|
||||
.getContext("2d", { willReadFrequently: true })
|
||||
.drawImage(source, rect.x, rect.y, rect.width, rect.height, 0, 0, rect.width, rect.height);
|
||||
return canvas;
|
||||
}
|
||||
|
||||
function canvasPreview(canvas, maxWidth = 760) {
|
||||
const scale = Math.min(1, maxWidth / canvas.width);
|
||||
const preview = document.createElement("canvas");
|
||||
preview.width = Math.max(1, Math.round(canvas.width * scale));
|
||||
preview.height = Math.max(1, Math.round(canvas.height * scale));
|
||||
preview.getContext("2d").drawImage(canvas, 0, 0, preview.width, preview.height);
|
||||
return preview.toDataURL("image/jpeg", 0.78);
|
||||
}
|
||||
|
||||
function cropPreview(frame, rect) {
|
||||
return isRectInside(rect, frame.width, frame.height)
|
||||
? cropCanvas(frame, rect).toDataURL("image/png")
|
||||
: null;
|
||||
}
|
||||
|
||||
function boxColor(status) {
|
||||
if (status === "found") return "oklch(0.72 0.16 155)";
|
||||
if (status === "pending") return "oklch(0.8 0.16 85)";
|
||||
if (status === "not-found") return "oklch(0.68 0.18 25)";
|
||||
return "oklch(0.72 0.13 250)";
|
||||
}
|
||||
|
||||
function annotatedPreview(frame, boxes) {
|
||||
const scale = Math.min(1, 760 / frame.width);
|
||||
const preview = document.createElement("canvas");
|
||||
preview.width = Math.max(1, Math.round(frame.width * scale));
|
||||
preview.height = Math.max(1, Math.round(frame.height * scale));
|
||||
const context = preview.getContext("2d");
|
||||
context.drawImage(frame, 0, 0, preview.width, preview.height);
|
||||
context.font = "12px system-ui";
|
||||
|
||||
boxes.forEach(({ rect, label, status }) => {
|
||||
const color = boxColor(status);
|
||||
const x = rect.x * scale;
|
||||
const y = rect.y * scale;
|
||||
const width = rect.width * scale;
|
||||
const height = rect.height * scale;
|
||||
context.strokeStyle = color;
|
||||
context.fillStyle = color;
|
||||
context.lineWidth = 2;
|
||||
context.strokeRect(x, y, width, height);
|
||||
const labelWidth = context.measureText(label).width + 10;
|
||||
const labelY = Math.max(0, y - 20);
|
||||
context.fillRect(x, labelY, labelWidth, 20);
|
||||
context.fillStyle = "oklch(0.98 0.004 250)";
|
||||
context.fillText(label, x + 5, labelY + 14);
|
||||
});
|
||||
|
||||
return preview.toDataURL("image/jpeg", 0.82);
|
||||
}
|
||||
|
||||
function addDiagnosticStage(diagnostics, key, label, status, detail, image = null) {
|
||||
diagnostics.stages.push({ key, label, status, detail, image });
|
||||
}
|
||||
|
||||
function addSkippedStages(diagnostics, stages, reason) {
|
||||
stages.forEach(([key, label]) => {
|
||||
addDiagnosticStage(diagnostics, key, label, "skipped", `Не запускалось: ${reason}`);
|
||||
});
|
||||
}
|
||||
|
||||
function finishDiagnostics(result, frame, diagnostics) {
|
||||
diagnostics.preview = annotatedPreview(frame, diagnostics.boxes);
|
||||
result.diagnostics = diagnostics;
|
||||
}
|
||||
|
||||
function addRegion(rect) {
|
||||
const type = elements.regionType.value;
|
||||
const reference = selectedReference();
|
||||
@@ -656,6 +604,8 @@ function addRegion(rect) {
|
||||
reference.saleAnchor = rect;
|
||||
reference.saleScore = 1;
|
||||
state.calibration.storeHeaderRegion = null;
|
||||
state.calibration.sellItemGridRegion = null;
|
||||
state.calibration.buyItemGridRegion = null;
|
||||
state.calibration.tooltipSearchRegion = null;
|
||||
elements.regionType.value = "storeHeader";
|
||||
setStatus("Маркер выбран. Теперь выдели строку Private Store(...) - имя", "success");
|
||||
@@ -673,6 +623,18 @@ function addRegion(rect) {
|
||||
};
|
||||
elements.regionType.value = "tooltipSearch";
|
||||
setStatus("Заголовок выбран. Теперь выдели зону появления tooltip", "success");
|
||||
} else if (type === "sellItemGrid" || type === "buyItemGrid") {
|
||||
if (!reference?.saleAnchor) {
|
||||
setStatus("Маркер магазина не найден на активном снимке", "error");
|
||||
return;
|
||||
}
|
||||
state.calibration[type === "sellItemGrid" ? "sellItemGridRegion" : "buyItemGridRegion"] = {
|
||||
offsetX: rect.x - reference.saleAnchor.x,
|
||||
offsetY: rect.y - reference.saleAnchor.y,
|
||||
width: rect.width,
|
||||
height: rect.height,
|
||||
};
|
||||
setStatus(`Сетка ${type === "sellItemGrid" ? "Sell" : "Buy"} выбрана`, "success");
|
||||
} else if (type === "tooltipSearch") {
|
||||
if (!reference?.saleAnchor) {
|
||||
setStatus("Маркер магазина не найден на активном снимке", "error");
|
||||
@@ -784,6 +746,22 @@ function drawCalibration() {
|
||||
if (referenceStoreHeaderRect()) {
|
||||
drawBox(context, referenceStoreHeaderRect(), "Тип + торговец", "oklch(0.65 0.17 250)");
|
||||
}
|
||||
if (referenceGridRect(state.calibration.sellItemGridRegion)) {
|
||||
drawBox(
|
||||
context,
|
||||
referenceGridRect(state.calibration.sellItemGridRegion),
|
||||
"Сетка Sell",
|
||||
"oklch(0.8 0.16 85)",
|
||||
);
|
||||
}
|
||||
if (referenceGridRect(state.calibration.buyItemGridRegion)) {
|
||||
drawBox(
|
||||
context,
|
||||
referenceGridRect(state.calibration.buyItemGridRegion),
|
||||
"Сетка Buy",
|
||||
"oklch(0.72 0.16 155)",
|
||||
);
|
||||
}
|
||||
if (referenceTooltipSearchRect()) {
|
||||
drawBox(context, referenceTooltipSearchRect(), "Поиск tooltip", "oklch(0.64 0.14 155)");
|
||||
}
|
||||
@@ -820,12 +798,17 @@ function updateDrawHint() {
|
||||
"Выдели постоянный нижний фрагмент торгового окна, например подпись Adena и кнопку Confirm. Числа лучше не включать.",
|
||||
storeHeader:
|
||||
"Выдели одну строку целиком: Private Store(Buy/Sell) - имя. Не включай рамку и кнопку закрытия.",
|
||||
sellItemGrid:
|
||||
"На чистом Sell-магазине обведи только прямоугольник 6 × 3 клеток Items on Sale, без заголовка и стрелок прокрутки.",
|
||||
buyItemGrid:
|
||||
"На чистом Buy-магазине обведи только прямоугольник 6 × 3 клеток For Sale, без заголовка и стрелок прокрутки.",
|
||||
tooltipSearch:
|
||||
"Выдели широкую область всех возможных положений tooltip. Нижний блок Price торгового окна не включай.",
|
||||
"Выдели широкую горизонтальную полосу всех возможных положений tooltip, даже за пределами окна трейда. Она ищет только Price :; нижний Price основного окна не включай.",
|
||||
tooltipAnchor: "Обведи только постоянную надпись Price : внутри tooltip, без изменяемого числа.",
|
||||
itemName:
|
||||
"Выдели точную верхнюю строку названия. Возьми скриншот с длинным названием и оставь запас справа.",
|
||||
itemPrice: "Выдели точную строку Price вместе с числом и словом Adena.",
|
||||
"Выдели левый край и точную высоту верхней строки названия. Ширина будет автоматически продлена до конца текста.",
|
||||
itemPrice:
|
||||
"Выдели левый край и точную высоту строки Price. Число и Adena будут захвачены автоматически.",
|
||||
};
|
||||
elements.drawHint.textContent = hints[elements.regionType.value];
|
||||
}
|
||||
@@ -836,6 +819,8 @@ function renderCalibration() {
|
||||
const regions = [
|
||||
["saleAnchor", "saleAnchor", "Шаблон поиска торгового окна"],
|
||||
["storeHeader", "storeHeaderRegion", "Тип сделки и имя торговца"],
|
||||
["sellItemGrid", "sellItemGridRegion", "Необязательная сетка 6 × 3 для прогресса Sell"],
|
||||
["buyItemGrid", "buyItemGridRegion", "Необязательная сетка 6 × 3 для прогресса Buy"],
|
||||
["tooltipSearch", "tooltipSearchRegion", "Ограничивает поиск всплывающего окна"],
|
||||
["tooltipAnchor", "tooltipAnchor", "Постоянная надпись Price :"],
|
||||
["itemName", "itemNameRegion", "Точная строка OCR"],
|
||||
@@ -865,6 +850,8 @@ function renderCalibration() {
|
||||
state.calibration[key] = null;
|
||||
if (key === "saleAnchor") {
|
||||
state.calibration.storeHeaderRegion = null;
|
||||
state.calibration.sellItemGridRegion = null;
|
||||
state.calibration.buyItemGridRegion = null;
|
||||
state.calibration.tooltipSearchRegion = null;
|
||||
state.references.forEach((item) => {
|
||||
item.saleAnchor = null;
|
||||
@@ -927,7 +914,7 @@ async function loadSharedCalibration() {
|
||||
setStatus("Общая калибровка ещё не настроена", "neutral");
|
||||
return;
|
||||
}
|
||||
state.calibration = settings.calibration;
|
||||
state.calibration = { ...emptyCalibration(), ...settings.calibration };
|
||||
localStorage.setItem(STORAGE_KEY, JSON.stringify(state.calibration));
|
||||
renderCalibration();
|
||||
renderReferences();
|
||||
@@ -953,21 +940,6 @@ function clearCalibration() {
|
||||
setStatus("Калибровка сброшена", "neutral");
|
||||
}
|
||||
|
||||
function imageToCanvas(dataUrl) {
|
||||
return new Promise((resolve, reject) => {
|
||||
const image = new Image();
|
||||
image.onload = () => {
|
||||
const canvas = document.createElement("canvas");
|
||||
canvas.width = image.naturalWidth;
|
||||
canvas.height = image.naturalHeight;
|
||||
canvas.getContext("2d").drawImage(image, 0, 0);
|
||||
resolve(canvas);
|
||||
};
|
||||
image.onerror = reject;
|
||||
image.src = dataUrl;
|
||||
});
|
||||
}
|
||||
|
||||
function appendDiagnostics(section, result) {
|
||||
if (!result.diagnostics) return;
|
||||
|
||||
@@ -1169,19 +1141,7 @@ function renderResults() {
|
||||
function resultFor(source) {
|
||||
let result = state.results.find((item) => item.sourceId === source.id);
|
||||
if (!result) {
|
||||
result = {
|
||||
sourceId: source.id,
|
||||
source: source.name,
|
||||
saleFound: false,
|
||||
score: 0,
|
||||
side: "",
|
||||
merchant: "",
|
||||
storeRawText: "",
|
||||
storeConfidence: 0,
|
||||
items: [],
|
||||
misses: 0,
|
||||
error: null,
|
||||
};
|
||||
result = createAnalysisResult(source);
|
||||
state.results.push(result);
|
||||
}
|
||||
return result;
|
||||
@@ -1191,284 +1151,40 @@ function delay(milliseconds) {
|
||||
return new Promise((resolve) => window.setTimeout(resolve, milliseconds));
|
||||
}
|
||||
|
||||
async function analyzeFrame(frame, result, saleTemplate, tooltipTemplate, source = null) {
|
||||
const diagnostics = { boxes: [], stages: [], preview: null };
|
||||
addDiagnosticStage(
|
||||
diagnostics,
|
||||
"capture",
|
||||
"Захват кадра",
|
||||
"found",
|
||||
`${frame.width} × ${frame.height}`,
|
||||
canvasPreview(frame, 240),
|
||||
);
|
||||
result.error = null;
|
||||
const hostedCatalogResolver = createCatalogResolver((name) =>
|
||||
fetch(`/api/l2/items?${new URLSearchParams({ q: name.slice(0, 128), limit: "1", fuzzy: "true" })}`, { cache: "no-store" })
|
||||
.then((response) => (response.ok ? response.json() : []))
|
||||
.then((items) => items[0] ?? null),
|
||||
);
|
||||
|
||||
const saleMatch = await findAnchor(frame, saleTemplate);
|
||||
const saleRect = {
|
||||
x: saleMatch.x,
|
||||
y: saleMatch.y,
|
||||
width: saleTemplate.width,
|
||||
height: saleTemplate.height,
|
||||
};
|
||||
const saleFound = saleMatch.score >= state.calibration.threshold;
|
||||
result.score = saleMatch.score;
|
||||
result.saleFound = saleFound;
|
||||
diagnostics.boxes.push({ rect: saleRect, label: "Маркер магазина", status: saleFound ? "found" : "not-found" });
|
||||
addDiagnosticStage(
|
||||
diagnostics,
|
||||
"sale",
|
||||
"Маркер магазина",
|
||||
saleFound ? "found" : "not-found",
|
||||
`${(saleMatch.score * 100).toFixed(1)}%, нужно ${(state.calibration.threshold * 100).toFixed(0)}%`,
|
||||
cropPreview(frame, saleRect),
|
||||
);
|
||||
async function resolveCatalogItem(item, alternatives = [item]) {
|
||||
return appHosted ? hostedCatalogResolver(item, alternatives) : item;
|
||||
}
|
||||
|
||||
if (!saleFound) {
|
||||
result.misses += 1;
|
||||
if (result.misses >= 3) {
|
||||
result.side = "";
|
||||
result.merchant = "";
|
||||
result.storeRawText = "";
|
||||
result.storeConfidence = 0;
|
||||
}
|
||||
addSkippedStages(
|
||||
diagnostics,
|
||||
[
|
||||
["storeHeader", "Заголовок магазина"],
|
||||
["tooltip", "Якорь Price :"],
|
||||
["itemName", "Название предмета"],
|
||||
["itemPrice", "Цена предмета"],
|
||||
],
|
||||
"маркер магазина не прошёл порог",
|
||||
);
|
||||
finishDiagnostics(result, frame, diagnostics);
|
||||
return;
|
||||
}
|
||||
function referenceGridRect(region) {
|
||||
const anchor = selectedReference()?.saleAnchor;
|
||||
return anchor && region ? resolveFieldRect(anchor, region) : null;
|
||||
}
|
||||
|
||||
result.misses = 0;
|
||||
const storeHeaderRect = resolveFieldRect(saleMatch, state.calibration.storeHeaderRegion);
|
||||
const tooltipSearchRect = resolveFieldRect(
|
||||
saleMatch,
|
||||
state.calibration.tooltipSearchRegion,
|
||||
);
|
||||
const storeHeaderBox = { rect: storeHeaderRect, label: "Заголовок", status: "info" };
|
||||
diagnostics.boxes.push(storeHeaderBox);
|
||||
diagnostics.boxes.push({ rect: tooltipSearchRect, label: "Зона tooltip", status: "info" });
|
||||
|
||||
if (isRectInside(storeHeaderRect, frame.width, frame.height)) {
|
||||
const headerOcr = await recognizeText(frame, storeHeaderRect, { singleLine: true });
|
||||
const header = parseStoreHeaderText(headerOcr.text);
|
||||
const rememberedStore = !header && result.side && result.merchant;
|
||||
storeHeaderBox.status = header ? "found" : rememberedStore ? "info" : "not-found";
|
||||
if (header) {
|
||||
result.side = header.side;
|
||||
result.merchant = header.merchant;
|
||||
result.storeRawText = header.rawText;
|
||||
result.storeConfidence = headerOcr.confidence;
|
||||
storeHeaderBox.label = `${header.side === "buy" ? "Buy" : "Sell"} · ${header.merchant}`;
|
||||
} else if (rememberedStore) {
|
||||
storeHeaderBox.label = `Последний: ${result.side === "buy" ? "Buy" : "Sell"} · ${result.merchant}`;
|
||||
}
|
||||
addDiagnosticStage(
|
||||
diagnostics,
|
||||
"storeHeader",
|
||||
"Заголовок магазина",
|
||||
header ? "found" : rememberedStore ? "skipped" : "not-found",
|
||||
header
|
||||
? `OCR ${headerOcr.confidence}%: ${header.side === "buy" ? "Покупка" : "Продажа"}, ${header.merchant}`
|
||||
: rememberedStore
|
||||
? `Заголовок закрыт или не читается. Используется последний магазин: ${result.side === "buy" ? "Покупка" : "Продажа"}, ${result.merchant}`
|
||||
: `OCR ${headerOcr.confidence}%: «${headerOcr.text || "пусто"}» — сначала нужно распознать тип и имя`,
|
||||
cropPreview(frame, storeHeaderRect),
|
||||
);
|
||||
} else {
|
||||
const rememberedStore = result.side && result.merchant;
|
||||
storeHeaderBox.status = rememberedStore ? "info" : "not-found";
|
||||
addDiagnosticStage(
|
||||
diagnostics,
|
||||
"storeHeader",
|
||||
"Заголовок магазина",
|
||||
rememberedStore ? "skipped" : "error",
|
||||
rememberedStore
|
||||
? `Область вне кадра. Используется последний магазин: ${result.side === "buy" ? "Покупка" : "Продажа"}, ${result.merchant}`
|
||||
: "Область вышла за границы кадра, а ранее распознанного магазина нет",
|
||||
);
|
||||
}
|
||||
|
||||
if (!isRectInside(tooltipSearchRect, frame.width, frame.height)) {
|
||||
addDiagnosticStage(
|
||||
diagnostics,
|
||||
"tooltip",
|
||||
"Якорь Price :",
|
||||
"error",
|
||||
"Зона поиска tooltip вышла за границы кадра",
|
||||
);
|
||||
addSkippedStages(
|
||||
diagnostics,
|
||||
[
|
||||
["itemName", "Название предмета"],
|
||||
["itemPrice", "Цена предмета"],
|
||||
],
|
||||
"зона поиска tooltip недоступна",
|
||||
);
|
||||
finishDiagnostics(result, frame, diagnostics);
|
||||
return;
|
||||
}
|
||||
|
||||
const tooltipSearch = cropCanvas(frame, tooltipSearchRect);
|
||||
const tooltipMatch = await findAnchor(tooltipSearch, tooltipTemplate);
|
||||
const tooltipRect = {
|
||||
x: tooltipSearchRect.x + tooltipMatch.x,
|
||||
y: tooltipSearchRect.y + tooltipMatch.y,
|
||||
width: tooltipTemplate.width,
|
||||
height: tooltipTemplate.height,
|
||||
};
|
||||
const tooltipFound = tooltipMatch.score >= state.calibration.threshold;
|
||||
diagnostics.boxes.push({ rect: tooltipRect, label: "Price :", status: tooltipFound ? "found" : "not-found" });
|
||||
addDiagnosticStage(
|
||||
diagnostics,
|
||||
"tooltip",
|
||||
"Якорь Price :",
|
||||
tooltipFound ? "found" : "not-found",
|
||||
`${(tooltipMatch.score * 100).toFixed(1)}%, нужно ${(state.calibration.threshold * 100).toFixed(0)}%`,
|
||||
cropPreview(frame, tooltipRect),
|
||||
);
|
||||
|
||||
if (!tooltipFound) {
|
||||
addSkippedStages(
|
||||
diagnostics,
|
||||
[
|
||||
["itemName", "Название предмета"],
|
||||
["itemPrice", "Цена предмета"],
|
||||
],
|
||||
"якорь Price : не найден",
|
||||
);
|
||||
finishDiagnostics(result, frame, diagnostics);
|
||||
return;
|
||||
}
|
||||
|
||||
if (!result.side || !result.merchant) {
|
||||
addSkippedStages(
|
||||
diagnostics,
|
||||
[
|
||||
["itemName", "Название предмета"],
|
||||
["itemPrice", "Цена предмета"],
|
||||
],
|
||||
"сначала нужно увидеть заголовок магазина без tooltip",
|
||||
);
|
||||
finishDiagnostics(result, frame, diagnostics);
|
||||
return;
|
||||
}
|
||||
|
||||
const nameRect = resolveFieldRect(tooltipRect, state.calibration.itemNameRegion);
|
||||
const priceRect = resolveFieldRect(tooltipRect, state.calibration.itemPriceRegion);
|
||||
const itemRect = combineRects(nameRect, priceRect);
|
||||
const itemBox = { rect: itemRect, label: "Название + цена", status: "pending" };
|
||||
diagnostics.boxes.push(itemBox);
|
||||
|
||||
if (
|
||||
!isRectInside(nameRect, frame.width, frame.height) ||
|
||||
!isRectInside(priceRect, frame.width, frame.height)
|
||||
) {
|
||||
itemBox.status = "not-found";
|
||||
addSkippedStages(
|
||||
diagnostics,
|
||||
[
|
||||
["itemName", "Название предмета"],
|
||||
["itemPrice", "Цена предмета"],
|
||||
],
|
||||
"точные строки вышли за границы кадра",
|
||||
);
|
||||
finishDiagnostics(result, frame, diagnostics);
|
||||
return;
|
||||
}
|
||||
|
||||
const itemOcr = await recognizeText(frame, itemRect);
|
||||
let item = parseTooltipText(itemOcr.text);
|
||||
itemBox.status = item ? "found" : "pending";
|
||||
addDiagnosticStage(
|
||||
diagnostics,
|
||||
"itemName",
|
||||
"Название предмета",
|
||||
item ? "found" : "not-found",
|
||||
itemOcr.text ? `OCR ${itemOcr.confidence}%: ${itemOcr.text}` : "OCR вернул пустую строку",
|
||||
cropPreview(frame, itemRect),
|
||||
);
|
||||
addDiagnosticStage(
|
||||
diagnostics,
|
||||
"itemPrice",
|
||||
"Цена предмета",
|
||||
item ? "found" : "not-found",
|
||||
itemOcr.text ? `OCR ${itemOcr.confidence}%: ${itemOcr.text}` : "OCR вернул пустую строку",
|
||||
cropPreview(frame, itemRect),
|
||||
);
|
||||
finishDiagnostics(result, frame, diagnostics);
|
||||
if (!item) return;
|
||||
|
||||
item = await resolveCatalogItem(item);
|
||||
|
||||
const confidence = itemOcr.confidence;
|
||||
const key = `${result.side}:${result.merchant.toLocaleLowerCase("en-US")}:${item.name.toLocaleLowerCase("en-US")}:${item.quantity ?? ""}:${item.priceAdena}`;
|
||||
const existing = result.items.find((entry) => entry.key === key);
|
||||
const seenAt = new Date().toISOString();
|
||||
|
||||
if (source) {
|
||||
async function analyzeCurrentFrame(frame, result, saleTemplate, tooltipTemplate, source = null) {
|
||||
const outcome = await analyzeFrame(frame, result, saleTemplate, tooltipTemplate, {
|
||||
calibration: state.calibration,
|
||||
source,
|
||||
resolveCatalogItem,
|
||||
catalogAlternatives: appHosted,
|
||||
enqueueObservation: source
|
||||
? async (observation) => {
|
||||
try {
|
||||
await marketOutbox.enqueue({
|
||||
observationId: crypto.randomUUID(),
|
||||
seenAt,
|
||||
sourceId: source.id,
|
||||
source: source.name.slice(0, 256),
|
||||
side: result.side || null,
|
||||
merchant: result.merchant.slice(0, 128),
|
||||
storeRawText: result.storeRawText.slice(0, 4096),
|
||||
storeConfidence: Number.isFinite(result.storeConfidence) ? Math.max(0, Math.min(100, Math.round(result.storeConfidence))) : 0,
|
||||
storeScore: Number.isFinite(saleMatch.score) ? Math.max(0, Math.min(1, saleMatch.score)) : 0,
|
||||
itemId: item.itemId ?? null,
|
||||
name: item.name.slice(0, 256),
|
||||
quantity: item.quantity,
|
||||
priceAdena: item.priceAdena,
|
||||
rawText: item.rawText.slice(0, 4096),
|
||||
key: key.slice(0, 512),
|
||||
confidence: Number.isFinite(confidence) ? Math.max(0, Math.min(100, confidence)) : 0,
|
||||
});
|
||||
await marketOutbox.enqueue(observation);
|
||||
} catch {
|
||||
state.collecting = false;
|
||||
throw new Error("Не удалось сохранить наблюдение рынка локально");
|
||||
}
|
||||
}
|
||||
|
||||
if (existing) {
|
||||
existing.seenAt = seenAt;
|
||||
existing.confidence = confidence;
|
||||
} else {
|
||||
result.items.push({
|
||||
...item,
|
||||
side: result.side,
|
||||
merchant: result.merchant,
|
||||
key,
|
||||
confidence,
|
||||
seenAt,
|
||||
: null,
|
||||
});
|
||||
setStatus(`Добавлен предмет: ${item.name}`, "success");
|
||||
}
|
||||
}
|
||||
|
||||
async function resolveCatalogItem(item) {
|
||||
if (!appHosted) return item;
|
||||
if (!catalogMatches.has(item.name)) {
|
||||
catalogMatches.set(
|
||||
item.name,
|
||||
fetch(`/api/l2/items?${new URLSearchParams({ q: item.name.slice(0, 128), limit: "1", fuzzy: "true" })}`, { cache: "no-store" })
|
||||
.then((response) => (response.ok ? response.json() : []))
|
||||
.then((items) => items[0] ?? null)
|
||||
.catch(() => null),
|
||||
);
|
||||
}
|
||||
const match = await catalogMatches.get(item.name);
|
||||
return match?.matchScore >= 0.35
|
||||
? { ...item, itemId: match.id, name: match.name, iconUrl: match.iconUrl, matchScore: match.matchScore }
|
||||
: item;
|
||||
if (outcome?.added) setStatus(`Добавлен предмет: ${outcome.item.name}`, "success");
|
||||
return outcome;
|
||||
}
|
||||
|
||||
async function scanSource(source, saleTemplate, tooltipTemplate) {
|
||||
@@ -1476,7 +1192,7 @@ async function scanSource(source, saleTemplate, tooltipTemplate) {
|
||||
let frame;
|
||||
try {
|
||||
frame = await captureSource(source);
|
||||
await analyzeFrame(frame, result, saleTemplate, tooltipTemplate, source);
|
||||
await analyzeCurrentFrame(frame, result, saleTemplate, tooltipTemplate, source);
|
||||
} catch (error) {
|
||||
result.saleFound = false;
|
||||
result.error = error.message || "Ошибка распознавания";
|
||||
@@ -1499,8 +1215,8 @@ async function scanSource(source, saleTemplate, tooltipTemplate) {
|
||||
|
||||
function loadCalibrationTemplates() {
|
||||
return Promise.all([
|
||||
imageToCanvas(state.calibration.saleAnchor.image),
|
||||
imageToCanvas(state.calibration.tooltipAnchor.image),
|
||||
loadImageCanvas(state.calibration.saleAnchor.image),
|
||||
loadImageCanvas(state.calibration.tooltipAnchor.image),
|
||||
]);
|
||||
}
|
||||
|
||||
@@ -1528,7 +1244,7 @@ async function testCalibration() {
|
||||
for (const [index, reference] of state.references.entries()) {
|
||||
frame = reference.canvas;
|
||||
setStatus(`Проверяю снимки: ${index + 1}/${state.references.length}`, "working");
|
||||
await analyzeFrame(frame, result, saleTemplate, tooltipTemplate);
|
||||
await analyzeCurrentFrame(frame, result, saleTemplate, tooltipTemplate);
|
||||
}
|
||||
renderResults();
|
||||
showView("results");
|
||||
@@ -1629,13 +1345,7 @@ function clearResults() {
|
||||
|
||||
async function copyResults() {
|
||||
try {
|
||||
const exportable = state.results.map((result) => ({
|
||||
source: result.source,
|
||||
side: result.saleFound ? result.side || null : null,
|
||||
merchant: result.saleFound ? result.merchant || null : null,
|
||||
saleFound: result.saleFound,
|
||||
items: result.items.map(({ key, ...item }) => item),
|
||||
}));
|
||||
const exportable = toExportableResults(state.results);
|
||||
await navigator.clipboard.writeText(JSON.stringify(exportable, null, 2));
|
||||
setStatus("JSON скопирован", "success");
|
||||
} catch {
|
||||
|
||||
@@ -68,17 +68,48 @@ export function parseFieldValue(type, text) {
|
||||
return digits ? Number(digits) : null;
|
||||
}
|
||||
|
||||
export function ocrEditDistance(first, second) {
|
||||
const row = Array.from({ length: second.length + 1 }, (_, index) => index);
|
||||
for (let firstIndex = 1; firstIndex <= first.length; firstIndex += 1) {
|
||||
let diagonal = row[0];
|
||||
row[0] = firstIndex;
|
||||
for (let secondIndex = 1; secondIndex <= second.length; secondIndex += 1) {
|
||||
const above = row[secondIndex];
|
||||
row[secondIndex] = Math.min(
|
||||
row[secondIndex] + 1,
|
||||
row[secondIndex - 1] + 1,
|
||||
diagonal + (first[firstIndex - 1] === second[secondIndex - 1] ? 0 : 1),
|
||||
);
|
||||
diagonal = above;
|
||||
}
|
||||
}
|
||||
return row[second.length];
|
||||
}
|
||||
|
||||
function recognizedSide(token) {
|
||||
const clean = token.toLocaleLowerCase("en-US").replace(/[^a-z]/g, "");
|
||||
if (!clean) return null;
|
||||
if (ocrEditDistance(clean, "sell") <= 2) return "sell";
|
||||
if (ocrEditDistance(clean, "buy") <= 1) return "buy";
|
||||
return null;
|
||||
}
|
||||
|
||||
export function parseStoreHeaderText(text) {
|
||||
const normalized = normalizeOcrText(text);
|
||||
const match = normalized.match(
|
||||
/private\s*store.*?\b(buy|sell)\b.*?[-–—]\s*(.+)$/i,
|
||||
if (normalized.includes("_")) return null;
|
||||
const strictMatch = normalized.match(
|
||||
/^[^\p{L}\p{N}]*(?:private\s*store\s*)?[({\[]*\s*(buy|sell)\s*[)}\]]*\s*(?:[-–—|]\s*)?([\p{L}\p{N}][\p{L}\p{N}\s]*)[^\p{L}\p{N}]*$/iu,
|
||||
);
|
||||
|
||||
if (!match?.[2]?.trim()) return null;
|
||||
const tokens = normalized.match(/[\p{L}\p{N}]+/gu) || [];
|
||||
const sideIndex = tokens.findIndex((token) => recognizedSide(token));
|
||||
const side = strictMatch?.[1]?.toLowerCase() || (sideIndex >= 0 ? recognizedSide(tokens[sideIndex]) : null);
|
||||
const merchant = (strictMatch?.[2] || (sideIndex >= 0 ? tokens.at(-1) : ""))
|
||||
.replace(/[^\p{L}\p{N}]/gu, "");
|
||||
if (!side || !merchant || merchant === tokens[sideIndex]) return null;
|
||||
|
||||
return {
|
||||
side: match[1].toLowerCase(),
|
||||
merchant: match[2].replace(/\s+/g, ""),
|
||||
side,
|
||||
merchant,
|
||||
rawText: normalized,
|
||||
};
|
||||
}
|
||||
@@ -155,3 +186,13 @@ export function isRectInside(rect, width, height) {
|
||||
rect.y + rect.height <= height
|
||||
);
|
||||
}
|
||||
|
||||
export function toExportableResults(results) {
|
||||
return results.map((result) => ({
|
||||
source: result.source,
|
||||
side: result.saleFound ? result.side || null : null,
|
||||
merchant: result.saleFound ? result.merchant || null : null,
|
||||
saleFound: result.saleFound,
|
||||
items: result.items.map(({ key, ...item }) => item),
|
||||
}));
|
||||
}
|
||||
|
||||
@@ -149,6 +149,7 @@ h3 {
|
||||
color: var(--text-muted);
|
||||
font-size: 0.84rem;
|
||||
text-align: center;
|
||||
overflow-wrap: anywhere;
|
||||
transition: background-color 180ms cubic-bezier(0.22, 1, 0.36, 1),
|
||||
border-color 180ms cubic-bezier(0.22, 1, 0.36, 1);
|
||||
}
|
||||
@@ -530,10 +531,17 @@ input[type="text"] {
|
||||
}
|
||||
|
||||
.source-row.is-selected {
|
||||
grid-template-columns: minmax(0, 1fr) auto;
|
||||
border-color: color-mix(in oklch, var(--accent), transparent 35%);
|
||||
box-shadow: 0 0 0 3px color-mix(in oklch, var(--accent), transparent 88%);
|
||||
}
|
||||
|
||||
.source-row.is-selected .source-row__preview {
|
||||
grid-column: 1 / -1;
|
||||
width: 100%;
|
||||
min-width: min(640px, 100%);
|
||||
}
|
||||
|
||||
.source-row__preview {
|
||||
position: relative;
|
||||
aspect-ratio: 16 / 9;
|
||||
@@ -633,7 +641,7 @@ input[type="text"] {
|
||||
.calibration-layout {
|
||||
display: grid;
|
||||
grid-template-columns: minmax(0, 1fr) 350px;
|
||||
min-height: calc(100vh - 300px);
|
||||
min-height: max(640px, calc(100vh - 220px));
|
||||
overflow: hidden;
|
||||
border: 1px solid var(--border);
|
||||
border-radius: 8px;
|
||||
@@ -644,7 +652,7 @@ input[type="text"] {
|
||||
.canvas-stage {
|
||||
display: grid;
|
||||
min-width: 0;
|
||||
min-height: calc(100vh - 300px);
|
||||
min-height: max(640px, calc(100vh - 220px));
|
||||
max-height: none;
|
||||
place-items: center;
|
||||
overflow: auto;
|
||||
@@ -660,14 +668,19 @@ input[type="text"] {
|
||||
|
||||
#calibration-canvas {
|
||||
display: block;
|
||||
max-width: 100%;
|
||||
max-height: 100%;
|
||||
place-self: start;
|
||||
image-rendering: pixelated;
|
||||
border-radius: 4px;
|
||||
box-shadow: 0 8px 30px oklch(0.2 0.02 255 / 0.2);
|
||||
cursor: crosshair;
|
||||
touch-action: none;
|
||||
}
|
||||
|
||||
#calibration-canvas[hidden],
|
||||
#canvas-empty[hidden] {
|
||||
display: none;
|
||||
}
|
||||
|
||||
.inspector {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
@@ -1253,6 +1266,14 @@ td:nth-child(5) {
|
||||
grid-template-columns: 1fr;
|
||||
}
|
||||
|
||||
.source-row.is-selected {
|
||||
grid-template-columns: 1fr;
|
||||
}
|
||||
|
||||
.source-row.is-selected .source-row__preview {
|
||||
min-width: 0;
|
||||
}
|
||||
|
||||
.source-row__actions {
|
||||
grid-column: auto;
|
||||
flex-wrap: wrap;
|
||||
|
||||
@@ -4,8 +4,16 @@ import { normalizeOcrText } from "./parser.js";
|
||||
let cvPromise;
|
||||
let workerPromise;
|
||||
let progressListener = () => {};
|
||||
const WHITE_TEXT_MIN_CHANNEL = 150;
|
||||
const WHITE_TEXT_MIN_CHANNEL = 120;
|
||||
const WHITE_TEXT_MAX_SPREAD = 55;
|
||||
const TEXT_RUN_GAP = 12;
|
||||
const TEXT_RUN_PADDING = 6;
|
||||
export const OCCUPIED_SLOT_THRESHOLD = 0.18;
|
||||
export const ICON_MATCH_THRESHOLD = 0.72;
|
||||
export const ICON_MATCH_MARGIN = 0.06;
|
||||
const GRID_COLUMNS = 6;
|
||||
const GRID_ROWS = 3;
|
||||
const SLOT_INSET = 3;
|
||||
|
||||
export function isWhiteTextPixel(red, green, blue, alpha = 255) {
|
||||
return (
|
||||
@@ -15,6 +23,196 @@ export function isWhiteTextPixel(red, green, blue, alpha = 255) {
|
||||
);
|
||||
}
|
||||
|
||||
export function isYellowTextPixel(red, green, blue, alpha = 255) {
|
||||
return (
|
||||
alpha >= 128 &&
|
||||
Math.min(red, green) >= WHITE_TEXT_MIN_CHANNEL &&
|
||||
Math.abs(red - green) <= 20 &&
|
||||
Math.min(red, green) - blue >= 80
|
||||
);
|
||||
}
|
||||
|
||||
export function findTextRunEnd(columns, gap = TEXT_RUN_GAP) {
|
||||
let lastText = -1;
|
||||
let emptyColumns = 0;
|
||||
|
||||
for (let index = 0; index < columns.length; index += 1) {
|
||||
if (columns[index]) {
|
||||
lastText = index;
|
||||
emptyColumns = 0;
|
||||
} else if (lastText >= 0) {
|
||||
emptyColumns += 1;
|
||||
if (emptyColumns >= gap) {
|
||||
return lastText;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return -1;
|
||||
}
|
||||
|
||||
export function gridSlotRects(gridRect) {
|
||||
const slots = [];
|
||||
for (let row = 0; row < GRID_ROWS; row += 1) {
|
||||
for (let column = 0; column < GRID_COLUMNS; column += 1) {
|
||||
const left = Math.round(gridRect.x + (column * gridRect.width) / GRID_COLUMNS);
|
||||
const right = Math.round(gridRect.x + ((column + 1) * gridRect.width) / GRID_COLUMNS);
|
||||
const top = Math.round(gridRect.y + (row * gridRect.height) / GRID_ROWS);
|
||||
const bottom = Math.round(gridRect.y + ((row + 1) * gridRect.height) / GRID_ROWS);
|
||||
slots.push({
|
||||
slotId: row * GRID_COLUMNS + column,
|
||||
row,
|
||||
column,
|
||||
frameRect: { x: left, y: top, width: right - left, height: bottom - top },
|
||||
});
|
||||
}
|
||||
}
|
||||
return slots;
|
||||
}
|
||||
|
||||
export function occupiedSlotScore(imageData, rect, inset = SLOT_INSET) {
|
||||
const left = Math.max(0, Math.round(rect.x + inset));
|
||||
const top = Math.max(0, Math.round(rect.y + inset));
|
||||
const right = Math.min(imageData.width, Math.round(rect.x + rect.width - inset));
|
||||
const bottom = Math.min(imageData.height, Math.round(rect.y + rect.height - inset));
|
||||
let luminance = 0;
|
||||
let count = 0;
|
||||
|
||||
for (let y = top; y < bottom; y += 1) {
|
||||
for (let x = left; x < right; x += 1) {
|
||||
const index = (y * imageData.width + x) * 4;
|
||||
luminance +=
|
||||
(imageData.data[index] * 0.2126 +
|
||||
imageData.data[index + 1] * 0.7152 +
|
||||
imageData.data[index + 2] * 0.0722) /
|
||||
255;
|
||||
count += 1;
|
||||
}
|
||||
}
|
||||
return count ? luminance / count : 0;
|
||||
}
|
||||
|
||||
function slotThumbnail(frame, rect) {
|
||||
const canvas = document.createElement("canvas");
|
||||
canvas.width = 32;
|
||||
canvas.height = 32;
|
||||
canvas
|
||||
.getContext("2d", { willReadFrequently: true })
|
||||
.drawImage(
|
||||
frame,
|
||||
rect.x + SLOT_INSET,
|
||||
rect.y + SLOT_INSET,
|
||||
rect.width - SLOT_INSET * 2,
|
||||
rect.height - SLOT_INSET * 2,
|
||||
0,
|
||||
0,
|
||||
32,
|
||||
32,
|
||||
);
|
||||
return canvas;
|
||||
}
|
||||
|
||||
export function detectOccupiedSlots(
|
||||
frame,
|
||||
absoluteGridRect,
|
||||
{ threshold = OCCUPIED_SLOT_THRESHOLD } = {},
|
||||
) {
|
||||
const context = frame.getContext("2d", { willReadFrequently: true });
|
||||
const imageData = context.getImageData(0, 0, frame.width, frame.height);
|
||||
return gridSlotRects(absoluteGridRect)
|
||||
.map((slot) => ({
|
||||
...slot,
|
||||
score: occupiedSlotScore(imageData, slot.frameRect),
|
||||
thumbnail: slotThumbnail(frame, slot.frameRect),
|
||||
}))
|
||||
.filter((slot) => slot.score >= threshold);
|
||||
}
|
||||
|
||||
export function iconSimilarity(first, second) {
|
||||
const size = 32;
|
||||
const canvas = document.createElement("canvas");
|
||||
canvas.width = size * 2;
|
||||
canvas.height = size;
|
||||
const context = canvas.getContext("2d", { willReadFrequently: true });
|
||||
context.drawImage(first, 0, 0, size, size);
|
||||
context.drawImage(second, size, 0, size, size);
|
||||
const left = context.getImageData(0, 0, size, size).data;
|
||||
const right = context.getImageData(size, 0, size, size).data;
|
||||
let difference = 0;
|
||||
for (let index = 0; index < left.length; index += 4) {
|
||||
difference += Math.abs(left[index] - right[index]);
|
||||
difference += Math.abs(left[index + 1] - right[index + 1]);
|
||||
difference += Math.abs(left[index + 2] - right[index + 2]);
|
||||
}
|
||||
return 1 - difference / (size * size * 3 * 255);
|
||||
}
|
||||
|
||||
export function matchCatalogIconToSlot(
|
||||
slots,
|
||||
icon,
|
||||
{ threshold = ICON_MATCH_THRESHOLD, margin = ICON_MATCH_MARGIN } = {},
|
||||
) {
|
||||
const candidates = slots
|
||||
.filter((slot) => slot.status === "pending" && slot.thumbnail)
|
||||
.map((slot) => ({ slotId: slot.slotId, score: iconSimilarity(slot.thumbnail, icon) }))
|
||||
return chooseUniqueBest(candidates, { threshold, margin });
|
||||
}
|
||||
|
||||
export function chooseUniqueBest(candidates, { threshold, margin }) {
|
||||
const ranked = [...candidates].sort((left, right) => right.score - left.score);
|
||||
const best = ranked[0];
|
||||
const second = ranked[1];
|
||||
if (!best || best.score < threshold) return null;
|
||||
if (second && best.score - second.score < margin) return null;
|
||||
return best.slotId;
|
||||
}
|
||||
|
||||
export function fitOcrTextRect(
|
||||
sourceCanvas,
|
||||
rect,
|
||||
{ includeYellow = false } = {},
|
||||
) {
|
||||
const x = Math.round(rect.x);
|
||||
const y = Math.round(rect.y);
|
||||
const width = sourceCanvas.width - x;
|
||||
const height = Math.round(rect.height);
|
||||
if (x < 0 || y < 0 || width <= 0 || height <= 0 || y + height > sourceCanvas.height) {
|
||||
return null;
|
||||
}
|
||||
|
||||
const context = sourceCanvas.getContext("2d", { willReadFrequently: true });
|
||||
const image = context.getImageData(x, y, width, height);
|
||||
const columns = new Uint8Array(width);
|
||||
const firstRow = height > 2 ? 1 : 0;
|
||||
const lastRow = height > 2 ? height - 1 : height;
|
||||
|
||||
for (let row = firstRow; row < lastRow; row += 1) {
|
||||
for (let column = 0; column < width; column += 1) {
|
||||
const index = (row * width + column) * 4;
|
||||
const red = image.data[index];
|
||||
const green = image.data[index + 1];
|
||||
const blue = image.data[index + 2];
|
||||
const alpha = image.data[index + 3];
|
||||
if (
|
||||
isWhiteTextPixel(red, green, blue, alpha) ||
|
||||
(includeYellow && isYellowTextPixel(red, green, blue, alpha))
|
||||
) {
|
||||
columns[column] = 1;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
const end = findTextRunEnd(columns, TEXT_RUN_GAP);
|
||||
if (end < 0) return null;
|
||||
|
||||
return {
|
||||
x,
|
||||
y,
|
||||
width: Math.min(width, end + 1 + TEXT_RUN_PADDING),
|
||||
height,
|
||||
};
|
||||
}
|
||||
|
||||
export function onOcrProgress(listener) {
|
||||
progressListener = listener;
|
||||
}
|
||||
@@ -38,9 +236,20 @@ async function getOpenCv() {
|
||||
|
||||
async function getOcrWorker() {
|
||||
if (!workerPromise) {
|
||||
workerPromise = createWorker("eng", 1, {
|
||||
const fixtureMode = typeof __FIXTURE_OCR__ !== "undefined" && __FIXTURE_OCR__;
|
||||
workerPromise = createWorker(
|
||||
"eng",
|
||||
1,
|
||||
fixtureMode
|
||||
? {
|
||||
workerPath: "/__fixtures/tesseract/worker.min.js",
|
||||
corePath: "/__fixtures/tesseract/tesseract-core-simd-lstm.wasm.js",
|
||||
langPath: "/__fixtures/tessdata",
|
||||
gzip: false,
|
||||
logger: (message) => progressListener(message),
|
||||
});
|
||||
}
|
||||
: { logger: (message) => progressListener(message) },
|
||||
);
|
||||
}
|
||||
|
||||
return workerPromise;
|
||||
@@ -81,8 +290,11 @@ export async function findAnchor(sourceCanvas, templateCanvas) {
|
||||
}
|
||||
}
|
||||
|
||||
function prepareOcrCrop(sourceCanvas, rect) {
|
||||
const scale = 3;
|
||||
function prepareOcrCrop(
|
||||
sourceCanvas,
|
||||
rect,
|
||||
{ includeYellow = false, preprocess = true, scale = 3 } = {},
|
||||
) {
|
||||
const canvas = document.createElement("canvas");
|
||||
canvas.width = rect.width * scale;
|
||||
canvas.height = rect.height * scale;
|
||||
@@ -101,14 +313,18 @@ function prepareOcrCrop(sourceCanvas, rect) {
|
||||
canvas.height,
|
||||
);
|
||||
|
||||
if (!preprocess) return canvas;
|
||||
|
||||
const image = context.getImageData(0, 0, canvas.width, canvas.height);
|
||||
for (let index = 0; index < image.data.length; index += 4) {
|
||||
const value = isWhiteTextPixel(
|
||||
image.data[index],
|
||||
image.data[index + 1],
|
||||
image.data[index + 2],
|
||||
image.data[index + 3],
|
||||
)
|
||||
const red = image.data[index];
|
||||
const green = image.data[index + 1];
|
||||
const blue = image.data[index + 2];
|
||||
const alpha = image.data[index + 3];
|
||||
const isLightPixel =
|
||||
preprocess === "luminance" && (red * 0.2126 + green * 0.7152 + blue * 0.0722) >= 105;
|
||||
const value = isLightPixel || isWhiteTextPixel(red, green, blue, alpha) ||
|
||||
(includeYellow && isYellowTextPixel(red, green, blue, alpha))
|
||||
? 0
|
||||
: 255;
|
||||
image.data[index] = value;
|
||||
@@ -121,16 +337,27 @@ function prepareOcrCrop(sourceCanvas, rect) {
|
||||
return canvas;
|
||||
}
|
||||
|
||||
export async function recognizeText(sourceCanvas, rect, { singleLine = false } = {}) {
|
||||
export async function recognizeText(
|
||||
sourceCanvas,
|
||||
rect,
|
||||
{
|
||||
singleLine = false,
|
||||
singleWord = false,
|
||||
includeYellow = false,
|
||||
preprocess = true,
|
||||
scale = 3,
|
||||
whitelist = "",
|
||||
} = {},
|
||||
) {
|
||||
const worker = await getOcrWorker();
|
||||
|
||||
await worker.setParameters({
|
||||
tessedit_pageseg_mode: singleLine ? PSM.SINGLE_LINE : PSM.SINGLE_BLOCK,
|
||||
tessedit_char_whitelist: "",
|
||||
tessedit_pageseg_mode: singleWord ? PSM.SINGLE_WORD : singleLine ? PSM.SINGLE_LINE : PSM.SINGLE_BLOCK,
|
||||
tessedit_char_whitelist: whitelist,
|
||||
preserve_interword_spaces: "1",
|
||||
});
|
||||
|
||||
const image = prepareOcrCrop(sourceCanvas, rect);
|
||||
const image = prepareOcrCrop(sourceCanvas, rect, { includeYellow, preprocess, scale });
|
||||
const { data } = await worker.recognize(image);
|
||||
const text = singleLine ? normalizeOcrText(data.text) : data.text.trim();
|
||||
|
||||
@@ -140,6 +367,59 @@ export async function recognizeText(sourceCanvas, rect, { singleLine = false } =
|
||||
};
|
||||
}
|
||||
|
||||
export async function recognizeTextRows(sourceCanvas, rows) {
|
||||
const rects = rows.map((row) => row.rect);
|
||||
const left = Math.min(...rects.map((rect) => rect.x));
|
||||
const top = Math.min(...rects.map((rect) => rect.y));
|
||||
const right = Math.max(...rects.map((rect) => rect.x + rect.width));
|
||||
const bottom = Math.max(...rects.map((rect) => rect.y + rect.height));
|
||||
const canvas = document.createElement("canvas");
|
||||
canvas.width = right - left;
|
||||
canvas.height = bottom - top;
|
||||
const context = canvas.getContext("2d", { willReadFrequently: true });
|
||||
|
||||
rows.forEach(({ rect, includeYellow = false }) => {
|
||||
const targetX = rect.x - left;
|
||||
const targetY = rect.y - top;
|
||||
context.drawImage(
|
||||
sourceCanvas,
|
||||
rect.x,
|
||||
rect.y,
|
||||
rect.width,
|
||||
rect.height,
|
||||
targetX,
|
||||
targetY,
|
||||
rect.width,
|
||||
rect.height,
|
||||
);
|
||||
|
||||
if (!includeYellow) {
|
||||
const image = context.getImageData(targetX, targetY, rect.width, rect.height);
|
||||
for (let index = 0; index < image.data.length; index += 4) {
|
||||
if (
|
||||
isYellowTextPixel(
|
||||
image.data[index],
|
||||
image.data[index + 1],
|
||||
image.data[index + 2],
|
||||
image.data[index + 3],
|
||||
)
|
||||
) {
|
||||
image.data[index] = 0;
|
||||
image.data[index + 1] = 0;
|
||||
image.data[index + 2] = 0;
|
||||
}
|
||||
}
|
||||
context.putImageData(image, targetX, targetY);
|
||||
}
|
||||
});
|
||||
|
||||
return recognizeText(
|
||||
canvas,
|
||||
{ x: 0, y: 0, width: canvas.width, height: canvas.height },
|
||||
{ includeYellow: rows.some((row) => row.includeYellow) },
|
||||
);
|
||||
}
|
||||
|
||||
export async function terminateVision() {
|
||||
if (!workerPromise) return;
|
||||
const worker = await workerPromise;
|
||||
|
||||
@@ -0,0 +1,25 @@
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { createCatalogResolver } from "../src/catalog-resolver.js";
|
||||
|
||||
test("chooses the strongest catalog response across OCR alternatives", async () => {
|
||||
const resolver = createCatalogResolver(async (name) => ({
|
||||
id: name,
|
||||
name: name === "garbled" ? "Wrong" : "Ancient Adena",
|
||||
iconUrl: `/${name}.png`,
|
||||
matchScore: name === "garbled" ? 0.4 : 0.92,
|
||||
}));
|
||||
const result = await resolver(
|
||||
{ name: "garbled", quantity: 1 },
|
||||
[{ name: "garbled", quantity: 1 }, { name: "Ancient Adena", quantity: 7 }],
|
||||
);
|
||||
assert.equal(result.name, "Ancient Adena");
|
||||
assert.equal(result.quantity, 7);
|
||||
assert.equal(result.matchScore, 0.92);
|
||||
});
|
||||
|
||||
test("keeps OCR data when every catalog response is below threshold", async () => {
|
||||
const resolver = createCatalogResolver(async () => ({ matchScore: 0.2 }));
|
||||
const item = { name: "Unknown", priceAdena: 3 };
|
||||
assert.equal(await resolver(item), item);
|
||||
});
|
||||
@@ -0,0 +1,92 @@
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { access, readdir, readFile } from "node:fs/promises";
|
||||
|
||||
const dataUrl = new URL("../data/", import.meta.url);
|
||||
|
||||
async function loadManifest() {
|
||||
return JSON.parse(await readFile(new URL("fixtures.json", dataUrl), "utf8"));
|
||||
}
|
||||
|
||||
test("fixture manifest covers every screenshot exactly once", async () => {
|
||||
const manifest = await loadManifest();
|
||||
const files = (await readdir(dataUrl)).filter((file) => file.toLowerCase().endsWith(".jpg")).sort();
|
||||
const declared = manifest.cases.map((fixture) => fixture.file).sort();
|
||||
|
||||
assert.equal(manifest.version, 1);
|
||||
assert.deepEqual(manifest.frame, { width: 1560, height: 1360 });
|
||||
assert.equal(new Set(manifest.cases.map((fixture) => fixture.id)).size, manifest.cases.length);
|
||||
assert.deepEqual(declared, files);
|
||||
});
|
||||
|
||||
test("fixture groups have deterministic order and reviewed slot transitions", async () => {
|
||||
const { cases } = await loadManifest();
|
||||
const groups = new Map();
|
||||
cases.forEach((fixture) => {
|
||||
const group = groups.get(fixture.groupId) || [];
|
||||
group.push(fixture);
|
||||
groups.set(fixture.groupId, group);
|
||||
});
|
||||
|
||||
for (const fixtures of groups.values()) {
|
||||
const orders = fixtures.map((fixture) => fixture.order).sort((left, right) => left - right);
|
||||
assert.deepEqual(orders, [...Array(fixtures.length).keys()]);
|
||||
|
||||
for (const fixture of fixtures) {
|
||||
const expected = fixture.expected;
|
||||
const occupied = expected.occupiedSlots;
|
||||
const classified = [...expected.pendingSlots, ...expected.foundSlots].sort((a, b) => a - b);
|
||||
assert.deepEqual(classified, occupied);
|
||||
assert.equal(new Set(classified).size, classified.length);
|
||||
assert.ok(classified.every((slot) => Number.isInteger(slot) && slot >= 0 && slot < 18));
|
||||
if (expected.item) {
|
||||
assert.ok(fixture.catalogAssetId);
|
||||
assert.ok(expected.foundSlots.includes(expected.hoveredSlot));
|
||||
assert.ok(expected.item.name && expected.item.displayName);
|
||||
assert.ok(Number.isSafeInteger(expected.item.quantity) && expected.item.quantity >= 0);
|
||||
assert.ok(Number.isSafeInteger(expected.item.priceAdena) && expected.item.priceAdena > 0);
|
||||
}
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
test("contextual tooltip fixtures own their initial shop state", async () => {
|
||||
const { cases } = await loadManifest();
|
||||
const contextual = cases.filter((fixture) => fixture.mode === "contextual");
|
||||
assert.ok(contextual.length > 0);
|
||||
contextual.forEach((fixture) => {
|
||||
assert.equal(fixture.initialState.side, fixture.expected.side);
|
||||
assert.equal(fixture.initialState.merchant, fixture.expected.merchant);
|
||||
assert.ok(fixture.initialState.activeShopKey);
|
||||
});
|
||||
});
|
||||
|
||||
test("fixture calibration and every declared catalog asset are committed", async () => {
|
||||
const manifest = await loadManifest();
|
||||
const calibration = JSON.parse(await readFile(new URL(manifest.calibration, dataUrl), "utf8"));
|
||||
const catalog = JSON.parse(
|
||||
await readFile(new URL("fixture-assets/catalog.json", dataUrl), "utf8"),
|
||||
);
|
||||
|
||||
assert.deepEqual(calibration.frame, manifest.frame);
|
||||
for (const field of [
|
||||
"saleAnchor",
|
||||
"storeHeaderRegion",
|
||||
"tooltipSearchRegion",
|
||||
"tooltipAnchor",
|
||||
"itemNameRegion",
|
||||
"itemPriceRegion",
|
||||
"sellItemGridRegion",
|
||||
"buyItemGridRegion",
|
||||
]) {
|
||||
assert.ok(calibration[field], `missing fixture calibration field: ${field}`);
|
||||
}
|
||||
await access(new URL(calibration.saleAnchor.image, dataUrl));
|
||||
await access(new URL(calibration.tooltipAnchor.image, dataUrl));
|
||||
|
||||
for (const fixture of manifest.cases.filter((entry) => entry.catalogAssetId)) {
|
||||
const item = catalog.items[fixture.catalogAssetId];
|
||||
assert.ok(item, `missing catalog fixture: ${fixture.catalogAssetId}`);
|
||||
await access(new URL(item.iconUrl.replace(/^\//, "../"), dataUrl));
|
||||
}
|
||||
});
|
||||
@@ -11,13 +11,123 @@ import {
|
||||
parseTooltipText,
|
||||
rectFromPoints,
|
||||
resolveFieldRect,
|
||||
toExportableResults,
|
||||
} from "../src/parser.js";
|
||||
import { isWhiteTextPixel } from "../src/vision.js";
|
||||
import {
|
||||
gridSlotRects,
|
||||
chooseUniqueBest,
|
||||
findTextRunEnd,
|
||||
fitOcrTextRect,
|
||||
occupiedSlotScore,
|
||||
isWhiteTextPixel,
|
||||
isYellowTextPixel,
|
||||
} from "../src/vision.js";
|
||||
import { activateShop, recordSaleMiss, resetShopProgress } from "../src/frame-analyzer.js";
|
||||
|
||||
test("keeps white text pixels and removes colored or dark pixels", () => {
|
||||
assert.equal(isWhiteTextPixel(220, 215, 195), true);
|
||||
assert.equal(isWhiteTextPixel(120, 120, 120), true);
|
||||
assert.equal(isWhiteTextPixel(210, 165, 80), false);
|
||||
assert.equal(isWhiteTextPixel(120, 120, 120), false);
|
||||
assert.equal(isWhiteTextPixel(119, 119, 119), false);
|
||||
});
|
||||
|
||||
test("keeps yellow tooltip prices without accepting brown UI pixels", () => {
|
||||
assert.equal(isYellowTextPixel(255, 255, 0), true);
|
||||
assert.equal(isYellowTextPixel(174, 173, 6), true);
|
||||
assert.equal(isYellowTextPixel(190, 120, 40), false);
|
||||
});
|
||||
|
||||
test("stops a tooltip text run before distant UI noise", () => {
|
||||
const columns = Array(48).fill(false);
|
||||
for (let index = 2; index <= 7; index += 1) columns[index] = true;
|
||||
for (let index = 15; index <= 25; index += 1) columns[index] = true;
|
||||
columns[42] = true;
|
||||
|
||||
assert.equal(findTextRunEnd(columns), 25);
|
||||
assert.equal(findTextRunEnd(Array(20).fill(false)), -1);
|
||||
assert.equal(findTextRunEnd(Array(40).fill(true)), -1);
|
||||
});
|
||||
|
||||
test("splits the calibrated item grid into stable 6 by 3 frame rectangles", () => {
|
||||
const slots = gridSlotRects({ x: 10, y: 20, width: 222, height: 111 });
|
||||
assert.equal(slots.length, 18);
|
||||
assert.deepEqual(slots[0], {
|
||||
slotId: 0,
|
||||
row: 0,
|
||||
column: 0,
|
||||
frameRect: { x: 10, y: 20, width: 37, height: 37 },
|
||||
});
|
||||
assert.deepEqual(slots[17], {
|
||||
slotId: 17,
|
||||
row: 2,
|
||||
column: 5,
|
||||
frameRect: { x: 195, y: 94, width: 37, height: 37 },
|
||||
});
|
||||
});
|
||||
|
||||
test("classifies a bright icon interior above an empty slot", () => {
|
||||
const image = {
|
||||
width: 12,
|
||||
height: 6,
|
||||
data: new Uint8ClampedArray(12 * 6 * 4).fill(255),
|
||||
};
|
||||
for (let y = 0; y < 6; y += 1) {
|
||||
for (let x = 0; x < 6; x += 1) {
|
||||
const index = (y * image.width + x) * 4;
|
||||
image.data[index] = 20;
|
||||
image.data[index + 1] = 20;
|
||||
image.data[index + 2] = 20;
|
||||
}
|
||||
}
|
||||
assert.ok(
|
||||
occupiedSlotScore(image, { x: 6, y: 0, width: 6, height: 6 }, 1) >
|
||||
occupiedSlotScore(image, { x: 0, y: 0, width: 6, height: 6 }, 1),
|
||||
);
|
||||
});
|
||||
|
||||
test("selects only a unique catalog score above threshold and margin", () => {
|
||||
assert.equal(
|
||||
chooseUniqueBest(
|
||||
[{ slotId: 0, score: 0.91 }, { slotId: 1, score: 0.7 }],
|
||||
{ threshold: 0.72, margin: 0.06 },
|
||||
),
|
||||
0,
|
||||
);
|
||||
assert.equal(
|
||||
chooseUniqueBest(
|
||||
[{ slotId: 0, score: 0.91 }, { slotId: 1, score: 0.89 }],
|
||||
{ threshold: 0.72, margin: 0.06 },
|
||||
),
|
||||
null,
|
||||
);
|
||||
});
|
||||
|
||||
test("fits a short row at the frame edge even when the saved width is wider", () => {
|
||||
const data = new Uint8ClampedArray(20 * 3 * 4);
|
||||
for (let column = 0; column < 4; column += 1) {
|
||||
const index = (20 + column) * 4;
|
||||
data[index] = 220;
|
||||
data[index + 1] = 220;
|
||||
data[index + 2] = 220;
|
||||
data[index + 3] = 255;
|
||||
}
|
||||
const canvas = {
|
||||
width: 40,
|
||||
height: 20,
|
||||
getContext() {
|
||||
return {
|
||||
getImageData(x, y, width, height) {
|
||||
assert.deepEqual([x, y, width, height], [20, 5, 20, 3]);
|
||||
return { data };
|
||||
},
|
||||
};
|
||||
},
|
||||
};
|
||||
|
||||
assert.deepEqual(
|
||||
fitOcrTextRect(canvas, { x: 20, y: 5, width: 50, height: 3 }),
|
||||
{ x: 20, y: 5, width: 10, height: 3 },
|
||||
);
|
||||
});
|
||||
|
||||
test("keeps only letters, digits and spaces in OCR item names", () => {
|
||||
@@ -86,6 +196,74 @@ test("detects OCR rectangles outside the captured frame", () => {
|
||||
assert.equal(isRectInside({ x: 90, y: 10, width: 20, height: 20 }, 100, 100), false);
|
||||
});
|
||||
|
||||
test("keeps transient grid images and slot state out of copied JSON", () => {
|
||||
const [exported] = toExportableResults([
|
||||
{
|
||||
source: "Lineage II",
|
||||
saleFound: true,
|
||||
side: "sell",
|
||||
merchant: "Gnumli",
|
||||
iconSlots: [{ slotId: 0, thumbnail: { pixels: true } }],
|
||||
gridCaptured: true,
|
||||
items: [{ key: "internal", name: "Soulshot C grade", priceAdena: 20 }],
|
||||
},
|
||||
]);
|
||||
assert.deepEqual(exported, {
|
||||
source: "Lineage II",
|
||||
side: "sell",
|
||||
merchant: "Gnumli",
|
||||
saleFound: true,
|
||||
items: [{ name: "Soulshot C grade", priceAdena: 20 }],
|
||||
});
|
||||
assert.doesNotMatch(JSON.stringify(exported), /iconSlots|gridCaptured|thumbnail|internal/);
|
||||
});
|
||||
|
||||
test("resets only transient shop progress when a shop closes or changes", () => {
|
||||
const result = {
|
||||
activeShopKey: "sell:gnumli",
|
||||
iconSlots: [{ slotId: 0, status: "found" }],
|
||||
gridCaptured: true,
|
||||
slotMatchStatus: "matched",
|
||||
items: [{ name: "Soulshot C grade" }],
|
||||
};
|
||||
resetShopProgress(result);
|
||||
assert.deepEqual(result, {
|
||||
activeShopKey: "",
|
||||
iconSlots: [],
|
||||
gridCaptured: false,
|
||||
slotMatchStatus: "",
|
||||
items: [{ name: "Soulshot C grade" }],
|
||||
});
|
||||
});
|
||||
|
||||
test("keeps shop progress through two misses and clears it on the third", () => {
|
||||
const result = {
|
||||
side: "sell",
|
||||
merchant: "Gnumli",
|
||||
storeRawText: "Private Store(Sell) - Gnumli",
|
||||
storeConfidence: 80,
|
||||
misses: 0,
|
||||
activeShopKey: "sell:gnumli",
|
||||
iconSlots: [{ slotId: 0, status: "found" }],
|
||||
gridCaptured: true,
|
||||
slotMatchStatus: "matched",
|
||||
items: [{ name: "Soulshot C grade" }],
|
||||
};
|
||||
recordSaleMiss(result);
|
||||
recordSaleMiss(result);
|
||||
assert.equal(result.activeShopKey, "sell:gnumli");
|
||||
recordSaleMiss(result);
|
||||
assert.equal(result.activeShopKey, "");
|
||||
assert.equal(result.side, "");
|
||||
assert.deepEqual(result.items, [{ name: "Soulshot C grade" }]);
|
||||
|
||||
activateShop(result, "sell", "Gnumli");
|
||||
result.gridCaptured = true;
|
||||
activateShop(result, "buy", "RAKOT");
|
||||
assert.equal(result.activeShopKey, "buy:rakot");
|
||||
assert.equal(result.gridCaptured, false);
|
||||
});
|
||||
|
||||
test("parses a hovered item tooltip", () => {
|
||||
assert.deepEqual(
|
||||
parseTooltipText("Tears of Eva\nPrice : 3,000,000 Adena\n(3 Million Adena)"),
|
||||
@@ -147,5 +325,18 @@ test("parses trade side and merchant from the store header", () => {
|
||||
rawText: "Private Store({Sell) - Dwa",
|
||||
});
|
||||
assert.equal(parseStoreHeaderText("Private Store(Buy) - Hik vision")?.merchant, "Hikvision");
|
||||
assert.deepEqual(parseStoreHeaderText("«Private Store(Sell) - BancoFrances»"), {
|
||||
side: "sell",
|
||||
merchant: "BancoFrances",
|
||||
rawText: "«Private Store(Sell) - BancoFrances»",
|
||||
});
|
||||
assert.equal(parseStoreHeaderText("Sell - BancoFrances")?.merchant, "BancoFrances");
|
||||
assert.deepEqual(parseStoreHeaderText("Brivata Stora Sally Gnumli"), {
|
||||
side: "sell",
|
||||
merchant: "Gnumli",
|
||||
rawText: "Brivata Stora Sally Gnumli",
|
||||
});
|
||||
assert.equal(parseStoreHeaderText("Private Store(Sell) - Banco_Frances"), null);
|
||||
assert.equal(parseStoreHeaderText("Trade - BancoFrances"), null);
|
||||
assert.equal(parseStoreHeaderText("Items on Sale"), null);
|
||||
});
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
import { readFile } from "node:fs/promises";
|
||||
import { resolve } from "node:path";
|
||||
import { defineConfig, loadEnv } from "vite";
|
||||
|
||||
const MAX_BATCH_BYTES = 96 * 1024;
|
||||
@@ -54,6 +56,27 @@ function marketImportProxy(options) {
|
||||
};
|
||||
}
|
||||
|
||||
function fixtureOcrAssets() {
|
||||
const assets = new Map([
|
||||
["/__fixtures/tesseract/worker.min.js", ["node_modules/tesseract.js/dist/worker.min.js", "text/javascript"]],
|
||||
["/__fixtures/tesseract/tesseract-core-simd-lstm.wasm.js", ["node_modules/tesseract.js-core/tesseract-core-simd-lstm.wasm.js", "text/javascript"]],
|
||||
["/__fixtures/tesseract/tesseract-core-simd-lstm.wasm", ["node_modules/tesseract.js-core/tesseract-core-simd-lstm.wasm", "application/wasm"]],
|
||||
["/__fixtures/tessdata/eng.traineddata", ["eng.traineddata", "application/octet-stream"]],
|
||||
]);
|
||||
return {
|
||||
name: "fixture-ocr-assets",
|
||||
configureServer(server) {
|
||||
server.middlewares.use(async (request, response, next) => {
|
||||
const asset = assets.get(request.url?.split("?", 1)[0]);
|
||||
if (!asset) return next();
|
||||
response.statusCode = 200;
|
||||
response.setHeader("Content-Type", asset[1]);
|
||||
response.end(await readFile(resolve(asset[0])));
|
||||
});
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
function readBody(request) {
|
||||
return new Promise((resolve, reject) => {
|
||||
const chunks = [];
|
||||
@@ -82,11 +105,13 @@ export default defineConfig(({ mode }) => {
|
||||
const environment = loadEnv(mode, process.cwd(), "");
|
||||
return {
|
||||
base: "./",
|
||||
define: { __FIXTURE_OCR__: JSON.stringify(mode === "test") },
|
||||
plugins: [
|
||||
mode === "test" ? fixtureOcrAssets() : null,
|
||||
marketImportProxy({
|
||||
url: environment.L2_MARKET_IMPORT_URL,
|
||||
token: L2_MARKET_IMPORT_TOKEN,
|
||||
}),
|
||||
],
|
||||
].filter(Boolean),
|
||||
};
|
||||
});
|
||||
|
||||