core+gui: chunked multi-QR will transfer (export/import + review/sign wizard, audio channel)

BALQR-framed chunk scheduler (bal/core/qrtransfer.py) with zlib compression, four chunk presets and a QR_CHUNK_SIZE setting (row 16). Export/import dialogs (WillQrExportDialog/WillQrImportDialog), per-tx review-and-sign wizard, export filters (All/Valid/Valid-NC) and an Auto slideshow with speed + loop for the QR codes; optional audio_modem channel with a local receive mirror. _prepare_and_sign_tx refactor (no behaviour change); WillItem.status defaults to '' instead of None (import crash fix); invalidate_will guard for a missing date_to_check. Docs: PLAN_QR_TRANSFER, QML_PLAN, README, HANDOFF, CHANGELOG entry 56, AUDIO_MODEM_DEBIAN.
2026-08-28 16:28:37 -04:00
parent 3a9ee5adb9
commit 8647eee586
16 changed files with 3106 additions and 64 deletions
--- a/QML_PLAN.md
+++ b/QML_PLAN.md
@@ -0,0 +1,379 @@
+# QML PLAN — BAL on Electrum QML / Android (Option B: minimal viable support)
+
+> Goal: let Android users (Electrum QML GUI) use BAL. **The Android device is
+> the OFFLINE SIGNING DEVICE**: its primary job is to receive unsigned will
+> transactions from an online machine (desktop/another phone), sign them with
+> the wallet keys held on it, and return the signed transactions — a classic
+> air-gapped signer workflow. Online features (willexecutor contact,
+> broadcast, build) are secondary on Android and belong mainly to the online
+> machine.
+> Strategy: a *third frontend* (`bal/gui/qml/`) that reuses `bal/core` logic
+> through the existing GUI-free `BalController`, exactly like `bal/cli/`
+> already does. The PyQt6 desktop GUI remains untouched and primary.
+>
+> Status: DRAFT for owner review (rule R4 — no code until explicit OK).
+> Chat language: Italian; this document is in English per rule R1.
+
+---
+
+## 1. Verified facts (research done on the local checkouts)
+
+All items below were verified by reading source, not assumed.
+
+| # | Fact | Where |
+|---|------|-------|
+| F1 | The fork's Electrum ships a full QML GUI built on **PyQt6.QtQml** (PyQt6 6.11 installed in runtime env imports `QtQml`/`QtQuick` fine). | `electrum/electrum/gui/qml/` |
+| F2 | The QML GUI has a **plugin mechanism**: manifest `"available_for"` must contain `"qml"`; Electrum then loads `<plugin>/qml.py` and calls the `init_qml(app)` hook. | `electrum/plugin.py` (`load_plugin_by_name`, gui_name), `gui/qml/__init__.py:88` |
+| F3 | On load, `main.qml` reads `plugin.so.loader` and auto-creates the component from `<plugins>/<name>/qml/<loader>.qml`. The plugin itself sets `.so` (a `PluginQObject`). Canonical example: `electrum/plugins/labels/qml.py`. | `gui/qml/components/main.qml` (`onPluginLoaded`), `gui/common_qt/plugins.py` |
+| F4 | The plugin must support **both** target versions: the QML GUI + `common_qt/plugins.py` exist in the 4.7.x line too (verified on the local 4.7.0 checkout). Exact 4.7.2 parity is Phase-0 task T1. | `$BAL_HOME/electrum470/electrum/gui/{qml,common_qt}` |
+| F5 | `BalController(plugin, wallet)` is GUI-free, per-wallet, and already implements state init + sign/build/broadcast flows against the bare `wallet` object (no `ElectrumWindow` needed). This is the reuse cornerstone of this plan. | `bal/cli/controller.py:119-175`, `sign_transactions` at :646 |
+| F6 | In the repo, `bal` is already symlinked into the Electrum tree as an **internal** plugin: `electrum/electrum/plugins/bal -> ../../../bal-electrum-plugin/bal`. Internal plugins are plain files on disk → the QML engine can load `.qml` assets directly. | `ls -la electrum/electrum/plugins/` |
+| F7 | The APK build spec lists packaged plugins explicitly and **BAL is not yet in that list**. | `electrum/contrib/android/buildozer_qml.spec:34-48` |
+| F8 | The QML Preferences page has **hardcoded toggles only** for `labels` and `psbt_nostr`; there is no generic plugin manager UI. Plugin enabling works via config regardless (`plugins.bal.enabled = true`). | `gui/qml/components/Preferences.qml:168,186,511-512` |
+| F9 | Extension points inside the QML app are minimal: `run_hook('init_qml', app)`, `run_hook('load_wallet', wallet)` (**one** argument, unlike Qt's two), `get_tx_extra_fee`, `tc_sign_wrapper`, and one named-component injection slot (`pluginsComponentsByName('export_tx_button')`). No tools menu, no status bar. | grep over `gui/qml/*.py`, `main.qml:778` |
+| F10 | External ZIP plugins cannot serve `.qml` files from inside the zip (zipimport exposes Python modules only; `Qt.resolvedUrl` needs real disk paths). Distribution as internal plugin (F6) or runtime extraction avoids this. | consequence of F3 |
+| F11 | Core signing path used by the CLI controller calls `wallet.sign_transaction(tx, password)` and updates signature counts — identical flow works under `QEWallet.wallet`. | `bal/cli/controller.py:646-700` |
+
+---
+
+## 2. Scope
+
+### In scope (MVP)
+
+Two usage **profiles** share one codebase:
+
+- **Offline signer profile** (Android, PRIORITY): works with no network.
+  1. Import a will bundle exported by the online machine (existing JSON will
+     format, see F12) — via file share, paste, or QR.
+  2. Review what is being signed (destinations, amounts, locktimes, fees).
+  3. Sign internally (`wallet.sign_transaction` + password prompt).
+     Partial signatures are combined when the bundle is re-imported
+     (`combine_with_other_psbt`, already supported by `merge_will` logic).
+  4. Export the signed bundle back to the online machine.
+- **Online manager profile** (desktop QML, secondary):
+  - Will status overview (state, expiry/check-alive date, reminder info).
+  - Heirs list (view/add/edit/remove, addresses or URIs).
+  - Will-executor selection (list, enable/disable, fee display, refresh).
+  - Build will (simplified wizard reusing core validation).
+  - Sign in place (password) or hand off to an offline signer via bundles.
+  - Broadcast / push to will-executors; invalidate will; check-alive refresh.
+  - Basic settings mapped onto `will_settings`.
+
+The offline signer pages are built first and must function with the network
+disabled (Electrum runs fine offline; willexecutor refresh simply degrades).
+
+Also in scope:
+
+- Android packaging: BAL bundled as internal plugin in the custom APK,
+  **enabled by default** (owner decision D3).
+- Keep desktop (`qt`) and CLI (`cmdline`) behavior byte-for-byte unchanged.
+
+### Out of scope (explicitly deferred)
+
+- Full parity with the PyQt6 GUI (calendar widget, preview list editor,
+  advanced fee controls, themes).
+- External-ZIP distribution of QML assets (F10 workaround postponed; ZIP
+  builds keep working for desktop exactly as today, without `qml` UI).
+- iOS, upstream-Electrum (spesmilo) compatibility.
+- Lightning-related features (irrelevant to BAL).
+
+---
+
+## 3. Architecture
+
+```
+                 ┌───────────────────────────────────────────┐
+                 │                bal/core                   │
+                 │ heirs, will, willexecutors, checkalive,   │
+                 │ reminders, input_rules, plugin_base       │
+                 └────────────┬──────────────────────────────┘
+                              │ (no Qt anywhere)
+        ┌─────────────────────┼──────────────────────┐
+        ▼                     ▼                      ▼
+  bal/gui/qt/            bal/cli/               bal/gui/qml/   ← NEW
+  BalWindow etc.      BalController            qml_plugin.py  (BalQmlPlugin)
+  (~9.3k lines)       (headless flows)         models.py      (QObject VMs)
+                                               so.py          (PluginQObject)
+                                               *.qml          (views)
+                                                      ▲
+                                              wraps ONE BalController
+                                              per loaded wallet
+```
+
+Design rules:
+
+- **Reuse, do not duplicate.** `bal/gui/qml/models.py` holds thin QObject
+  wrappers around one `BalController` instance per wallet. No business logic
+  in QML or in the wrappers beyond formatting.
+- **Same persistence.** Wallet DB dicts (`heirs`, `will`, `will_settings`)
+  are registered by `bal/core/plugin_base.py` already; QML reads/writes them
+  through the controller, so a wallet moves between desktop/Android unchanged.
+- **Threading.** Network operations (willexecutor fetch/push, broadcast)
+  run in worker threads exactly as the CLI does; results marshalled to the UI
+  thread via Qt signals on the wrapper objects. No blocking calls in slots.
+- **One transfer format.** The airgap round trip reuses the existing JSON
+  will serialization (`WillItem.to_dict()` maps, exactly what the Qt GUI's
+  `export_json_file`/`import` + `merge_will` flow already produces and
+  consumes — see F12). No new format is invented; export/merge logic gets a
+  single shared home usable by both frontends.
+- **Version gating.** Every import of `electrum.gui.qml.*` happens lazily and
+  defensively; if absent (e.g., odd build), the plugin degrades to core-only
+  behavior instead of crashing the daemon.
+
+---
+
+## 4. Work breakdown
+
+### Phase 0 — Verification spikes (no product code)
+
+| Task | Description | Exit criterion |
+|------|-------------|----------------|
+| T1 | Diff `gui/qml` + `gui/common_qt` between the 4.7.0 checkout here and current 4.8.x, focused on: `PluginQObject`, `init_qml` hook call sites, `onPluginLoaded` handler, `load_wallet` hook arity. If 4.7.2 differs, note shims needed. | Written compatibility note appended to COMPATIBILITY.md draft section |
+| T2 | Run desktop QML GUI headless with BAL enabled via config: `QT_QPA_PLATFORM=offscreen run_electrum -g qml` with `plugins.bal.enabled=true`, manifest updated ad-hoc (throwaway branch). Confirms discovery/loading path end-to-end before writing any code. | Log shows `init_qml` called for bal; no crash |
+| T3 | APK feasibility: add `electrum/plugins/bal` to `buildozer_qml.spec` package list locally, confirm p4a includes `.qml` data files and icons (may need `source.include_exts` adjustment). Do NOT ship. | Test APK contains `plugins/bal/qml/*.qml` |
+| T4 | Decide entry-point UX given F9 (no menu hook): candidate = tiny patch in fork's `main.qml` adding a "BAL" item in the wallet drawer/menu that opens our window object from `app.pluginobjects['bal']`. Confirm with owner. | Decision recorded in this file |
+
+Deliverable: short findings report appended to this document; go/no-go.
+
+### Phase 1 — Skeleton integration
+
+Files (all NEW unless noted):
+
+```
+bal/qml.py                     zipimport-style shim mirroring qt.py/cmdline.py
+bal/gui/qml/__init__.py        package docstring
+bal/gui/qml/qml_plugin.py      class BalQmlPlugin(BalPluginBase)
+bal/gui/qml/so.py              class BalSignalObject(PluginQObject)
+bal/manifest.json              MODIFIED: available_for += ["qml"]
+```
+
+Details:
+
+- `qml_plugin.py`:
+  - `@hook init_qml(self, app)`: store app ref; create `so` parented to app;
+    for each already-loaded wallet call `_on_wallet_loaded(wallet)`
+    (mirrors labels' pattern, see F3).
+  - `@hook load_wallet(self, wallet)` — **single argument** (F9); creates the
+    per-wallet view-model bundle (Phase 2) keyed by `wallet`.
+  - `@hook unload_wallet(self, wallet)`: drop controllers, close windows.
+- `so.py`: `BalSignalObject(PluginQObject)` exposing:
+  - `loader` property returning `"BalMain.qml"` (drives F3 auto-create);
+  - signals: `walletChanged`, `willStateChanged`, `heirsChanged`,
+    `willexecutorsChanged`, `busyChanged`;
+  - slots called from QML: open/close window, refresh willexecutors,
+    check-alive now, build/sign/broadcast/invalidate commands.
+- `manifest.json`: append `"qml"` to `available_for`. Desktop untouched
+  (Electrum filters per running GUI, verified F2).
+
+Exit criterion: with `-g qml`, plugin loads, `so.loader` component is created
+(log line from `onPluginLoaded`), no functional UI yet.
+
+### Phase 2 — View-models (QObject layer)
+
+New file `bal/gui/qml/models.py`:
+
+- `BalQmlWallet(QObject)`: owns one `BalController`; exposes read-only
+  properties (`willState`, `dateToCheck`, `expired`, `reminderInfo`,
+  `sigsHave/sigsRequired` per tx) + notification signals; forwards actions to
+  controller methods (`build_will`, `sign_transactions`, `broadcast_will`,
+  `invalidate_will_headless`, `check_alive`... — names per controller).
+  - **Airgap methods (priority):** `export_will_bundle()` and
+    `import_will_bundle(json_text)` returning summary of what changed. These
+    are small ports of the Qt GUI's `export_json_file` (window.py:1605) and
+    `merge_will` (window.py:1620) semantics. Preferred implementation: move
+    the logic into shared helpers (controller level or `bal/core/will.py`
+    static functions) and make the Qt GUI call the same helpers, so the two
+    frontends cannot diverge; regression-covered by existing core tests plus
+    new round-trip tests.
+  - Offline profile detection: expose an `isOffline` property derived from
+    `wallet.network is None` / config, so QML can hide online-only pages.
+- `HeirListModel(QAbstractListModel)`: roles `name`, `address`, `amountPct`,
+  `valid`; edit methods delegate to `Heirs` helpers through controller.
+- `WillTxListModel(QAbstractListModel)`: roles `txid`, `status`, `fee`,
+  `sigsHave`, `sigsRequired`, `isComplete`.
+- `WillExecutorListModel(QAbstractListModel)`: roles `url`, `selected`,
+  `fee`, `valid`; toggle + async refresh.
+- `BalQrTransferModel(QObject)`: thin scheduler over the shared transfer
+  planner `bal.core.qrtransfer` (already battle-tested by the desktop Qt
+  plugin, P1-P4). Exposes `encode(items)` → frames, `frameAt(i)` (data URL /
+  pixmap for QML), `decode(text)` → tx list, `total`, `current`, presets;
+  re-emits a `frameChanged` notifier so the QML page can step 1..N. No QR
+  rendering inside the model (QML paints it).
+- All list mutations happen on the controller state then `beginResetModel/
+  endResetModel` (datasets are small; simplicity over incremental updates).
+
+Exit criterion: pytest-driven model tests pass offscreen (create models over a
+regtest/testnet wallet fixture, assert roles after mutations).
+
+### Phase 3 — QML views
+
+New directory `bal/gui/qml/components/`:
+
+```
+BalMain.qml            top-level Window; stack of pages below
+BalSignPage.qml        PRIORITY (offline signer): import bundle
+                       (paste / file / QR), review summary of each tx,
+                       password-sign, export signed bundle back
+BalQrExportPage.qml    QR export view: drives BalQrTransferModel, one
+                       frame at a time (Prev/Next, progress i/N, chunk
+                       preset selector), mirror of desktop Qt dialog
+BalQrImportPage.qml    QR import view: slot grid (1..N), camera via
+                       Electrum `QRScan` reuse, manual paste fallback,
+                       then jump into BalSignPage review/sign
+BalStatusPage.qml      will state, expiry countdown, check-alive button
+                       (works offline with last-known data)
+BalHeirsPage.qml       ListView + add/edit dialog            [online profile]
+BalExecutorsPage.qml   ListView with switches + refresh      [online profile]
+BalBuildPage.qml       simplified build form (threshold selector, fees,
+                       executor pick) → runs controller.build_will
+                                                              [online profile]
+BalSettingsPage.qml    maps onto will_settings subset
+controls/BalButton.qml, BalField.qml   minimal styled primitives
+qmldir                 module registration
+```
+
+Notes:
+
+- **BalSignPage review screen is mandatory**: before signing, the user must
+  see per-transaction heir address, amount, locktime (date), and fee — this
+  device is the security boundary, so no "blind signing".
+- QR transfer: a full will bundle may exceed one QR's capacity when there are
+  many heirs/transactions. The desktop Qt plugin now resolves this with
+  chunked multi-QR streams (`bal/core/qrtransfer.py`, chunk presets
+  150/400/900/1800 bytes/frame, EC level M); the QML GUI reuses the same
+  scheduler via a thin model. Order of preference stays (1) share/save file +
+  paste text; (2) chunked QR streams through `BalQrTransferModel`.
+- Styling minimal, follow existing QML components' look (reuse
+  `controls/` from Electrum where importable — prefer copying tiny primitives
+  to avoid coupling to upstream churn; decide during implementation).
+- Every string wrapped with `electrum.i18n._`.
+
+Exit criterion: full manual walkthrough on desktop QML GUI (offscreen +
+interactive) performing BOTH: (a) online profile — configure heirs → select
+executors → build → sign → broadcast → invalidate; and (b) offline signer
+profile — export unsigned bundle from an online wallet, import into a second
+offline wallet instance, sign, re-export, import signed bundle back into the
+first wallet and verify signatures combined/status COMPLETE. Walkthrough (b)
+must pass with networking disabled.
+
+### Phase 4 — Android integration
+
+- Add `electrum/plugins/bal` (+ data extensions for `.qml`, `icons/*`) to
+  `contrib/android/buildozer_qml.spec` in the Electrum fork.
+- Patch fork's `Preferences.qml` with a BAL toggle **(owner approved, D1)**
+  and default-enable BAL in the APK build (D3: enabled by default).
+- Entry point per T4 decision (menu/drawer patch in fork's `main.qml`, D1
+  approved). Fallback if T4 picks auto-open: window opens on wallet load.
+- Rebuild APK; smoke-test on device/emulator:
+  install → BAL already enabled → open wallet → full MVP walkthrough,
+  including airplane-mode signing round trip (file/QR transfer between an
+  online desktop and the offline device — for emulator testing, "offline" =
+  network disabled via settings).
+- Watch-outs: filesystem paths (use `os.path.join`, no hardcoded separators —
+  already house style), background network on mobile (willexecutor timeouts),
+  APK size impact (bal is small; icons only), share-intent/file access
+  permissions for bundle import/export.
+
+Exit criterion: signed test APK passes the same walkthrough as Phase 3.
+
+### Phase 5 — Tests & CI hygiene
+
+- New tests following repo conventions (`def test_*` + `__main__` block):
+  - `tests/test_qml_models.py` — models over fake/controller-backed wallet
+    (offscreen, no network).
+  - `tests/test_qml_airgap_roundtrip.py` — PRIORITY: export unsigned bundle
+    from wallet A → import into wallet B (same seed, offline) → sign →
+    export signed → merge back into A; assert COMPLETE status and signature
+    counts. Must pass with no network.
+  - `tests/test_qml_plugin_loading.py` — plugin instantiates under a stubbed
+    QML app object; `so` wiring correct; load/unload wallet lifecycle.
+  - Extend `tests/smoke_test.py` usage: `QT_QPA_PLATFORM=offscreen python3
+    tests/smoke_test.py electrum.plugins.bal` still green (qt path intact).
+- Regression gate: full `tests/test_core_*.py` batch + ruff (no NEW
+  violations) before any delivery ZIP.
+- Manual matrix recorded in CHANGELOG entry: [desktop qt, desktop qml
+  offscreen, Android APK] × [4.7.2, 4.8.0] where applicable.
+
+### Phase 6 — Release plumbing & docs
+
+- `build_zip.py`: ensure new `bal/gui/qml/**` and `components/*.qml` included
+  in deterministic zip (harmless on desktop; enables future extraction-based
+  loading).
+- `COMPATIBILITY.md`, `README.md`, `HANDOFF.md`: document QML/Android status,
+  limitations, and how to enable (`-g qml` / APK toggle).
+- `CHANGELOG.md`: numbered entry at END per house rules.
+- Version bump + release handled by `make-release.sh` as usual (owner-driven).
+
+---
+
+## 5. Risks & mitigations
+
+| Risk | Impact | Mitigation |
+|------|--------|------------|
+| R1: 4.7.2 vs 4.8 QML internals drift | broken load on one version | Phase-0 T1 diff first; lazy imports + capability checks; shim module if needed |
+| R2: no generic plugin UI in QML (F8) | users can't enable BAL from UI | config default-enable in fork APK; small Preferences.qml patch in fork (we control it); document manual config for desktop |
+| R3: no natural entry point in main.qml (F9) | user can't find/open BAL window | T4 decision: fork-side menu/drawer patch; fallback = auto-open window on wallet load behind a setting |
+| R4: dual-GUI maintenance burden | long-term cost | strict reuse of `BalController`; QML layer forbidden from business logic (review rule); parity features stay in qt GUI |
+| R5: threading bugs on mobile networks | ANRs/crashes | all network ops in threads like CLI; signals-only UI updates; timeouts already configurable |
+| R6: airgap transfer friction (bundle size vs QR capacity, share permissions on Android) | users cannot move bundles reliably | desktop Qt: chunked multi-QR streams + audio-modem optional channel shipped (P1-P4) and regression-gated; QML: file share + paste first, QR via `BalQrTransferModel` (Phase 2/3 notes); Android camera on Qt6 is known-flaky, file/paste stays the primary mobile fallback and is tested first in Phase 4 |
+| R7: export/merge semantics divergence between frontends | signed bundles rejected or double-counted | single shared helper used by qt GUI and qml layer (Phase 2); round-trip regression test |
+| R8: hidden coupling of qt code into shared modules | qml import pulls QtWidgets | lint guard idea: import-linter/ruff rule forbidding `PyQt6.QtWidgets` under `bal/gui/qml/` |
+| R9: zip distribution ambiguity (F10) | confusion about what ships where | clear policy: ZIP = desktop qt+cmdline only; QML requires internal-plugin/APK route (Phase 6 documents this) |
+| R10: unknown Android/Electrum baseline (owner to confirm, OQ4) | wrong Qt/PyQt6 assumptions in APK build | Phase-0 T3 builds against the fork's current toolchain; code keeps 4.7.x/4.8.x dual support so the answer can arrive late without rework |
+
+---
+
+## 6. Questions for the owner — ANSWERED (2026-08-25)
+
+1. **Fork patches (T4/R3):** ✅ **D1 — APPROVED.** Patching the fork's
+   `main.qml`/`Preferences.qml` is allowed for the BAL menu entry and toggle.
+2. **Offline signing topology:** ✅ **D2 — Android IS the offline device.**
+   The phone holds the keys and acts as air-gapped signer: import unsigned
+   bundle → review → sign → export signed bundle back to the online machine.
+   The sign/import/export page is therefore the top priority of Phase 3, and
+   the round-trip test is the top priority of Phase 5.
+3. **Enable-by-default:** ✅ **D3 — BAL pre-enabled in the custom APK**
+   (toggle still available to disable).
+4. **Target Android/Electrum baseline:** ⏳ **OPEN (OQ4)** — owner will get
+   back later. Not blocking: see risk R10 mitigation.
+
+---
+
+## 7. Effort estimate
+
+| Phase | Rough size |
+|-------|-----------|
+| 0 spikes | ~half day (mostly reading + one throwaway branch) |
+| 1 skeleton | ~300 lines Python |
+| 2 models (incl. shared export/merge helpers) | ~600–800 lines Python |
+| 3 views (offline signer page first) | ~900–1300 lines QML |
+| 4 android | fork-side patches + build iteration (device-dependent) |
+| 5 tests | ~500 lines |
+| 6 release/docs | small |
+
+Overall: comparable to a medium feature, dominated by Phase 3 UI polish and
+Phase 4 device iteration.
+
+---
+
+## 8. Findings log (append-only)
+
+- **F12** — The airgap round trip already exists in the Qt GUI layer and can
+  be ported almost verbatim: `export_json_file()` (window.py:1605) exports
+  all will items as `{wid: WillItem.to_dict()}` JSON (marking them
+  `EXPORTED`); the import side goes through `merge_will()`
+  (window.py:1620), which carries operational statuses, combines partial
+  signatures via `tx.combine_with_other_psbt()` when txids match, substitutes
+  the tx otherwise, and recomputes validity locally without network.
+  Conclusion: no new transfer format is needed; the plan is to give this
+  logic a shared home (controller/core) so Qt, CLI-adjacent tooling and QML
+  all use one implementation.
+
+### Decisions
+
+- **D1** — Fork-side patches to `main.qml` / `Preferences.qml` approved by
+  the owner.
+- **D2** — Android = offline signing device; sign/import/export flow has
+  top priority.
+- **D3** — BAL enabled by default in the custom APK.
+- **OQ4** — Android/Electrum baseline: open, non-blocking (see R10).