docs: translate networking report to English and update with Check/counters/GUI

Translate REPORT_NETWORKING_PARALLELO.md to English and bring it up to date:
- parallel push/ping/check with fast-fail timeouts + global deadline
- reliable Xs/Ns elapsed-time counter driven by on_tick from the calling thread
  (replacing the unreliable raw heartbeat thread)
- status-bar icon restore, toolbar tooltips and reorder
- updated verification section (182 official tests, ruff 0 new issues)
This commit is contained in:
GenSpark AI Developer
2026-06-15 18:50:45 +00:00
parent 4abd2e508f
commit a8155183d7

View File

@@ -1,133 +1,213 @@
# Report tecnico — Networking parallelo (anti-freeze Will-Executor)
# Technical report — Parallel networking (Will-Executor anti-freeze) + UI feedback
**Destinatario:** programmatore esterno / manutentore del plugin
**Autore:** refactoring AI (lavoro su GitHub `Bitcoin-after-life/test`)
**Data:** 2026-06-15
**Branch:** `feature/networking-parallelo`
**Repository Gitea privato `kaibot/bal-plugin-ai`: NON modificato** (per richiesta esplicita).
**Audience:** external programmer / plugin maintainer
**Author:** AI refactoring work (on GitHub `Bitcoin-after-life/test`)
**Date:** 2026-06-15
**Branch:** `feature/networking-parallelo` (Pull Request #4)
**Private Gitea repo `kaibot/bal-plugin-ai`: NOT modified** (per explicit request; cloned read-only only to run the official tests).
---
## 1. Problema
## 1. Problem
Quando il plugin contatta i server Will-Executor (invio transazioni, ping/aggiornamento
eredità, download lista), lo fa **in sequenza**. Se un server non risponde, il thread
resta bloccato sui timeout di connessione e, peggio, sui **retry**:
When the plugin contacts the Will-Executor servers (pushing transactions,
pinging/refreshing the inheritance, downloading the list, and **checking**
transactions), it used to do it **sequentially**. If a server did not answer,
the thread stayed blocked on the connection timeouts and, worse, on the
**retries**:
- `send_request` riprovava fino a **10 volte** con `time.sleep(3)` ad ogni timeout
→ circa **130 secondi per ogni server irraggiungibile**, sommati uno dopo l'altro.
- `send_request` retried up to **10 times** with `time.sleep(3)` on every
timeout → roughly **~140 seconds per unreachable server**, summed one after
another.
Conseguenze:
- Con pochi server già si avverte; con **20 server** diventa ingestibile.
- L'utente vede "Rimani in attesa — Non risponde" senza capire cosa succede.
- Un singolo server morto blocca l'intera operazione.
Consequences:
- Noticeable already with a few servers; with **20 servers** it became
unusable.
- The user saw "Stay waiting — Not responding" with no idea what was happening.
- A single dead server blocked the whole operation.
---
## 2. Soluzione (in sintesi)
## 2. Solution (overview)
1. **Parallelismo** con `ThreadPoolExecutor`: i server vengono contattati
contemporaneamente. Il tempo totale ≈ server **più lento**, non la **somma**.
2. **Fast-fail** per le operazioni interattive (ping/info/download): niente retry-storm,
un solo timeout breve e il server viene marcato "KO".
3. **Feedback live**: callback `on_each(...)` che aggiorna la finestra di attesa
server-per-server, in modo thread-safe.
4. **Push transazioni**: parallelo, ma mantiene i retry per ogni singolo server
(una transazione reale non deve andare persa per un hiccup transitorio).
1. **Parallelism** with `ThreadPoolExecutor`: servers are contacted
concurrently. Total time ≈ the **slowest** server, not the **sum**.
2. **Fast-fail** for interactive operations (ping/info/download): no retry
storm, a single short timeout and the server is marked "KO".
3. **Aggressive timeouts + global deadline** for push/check: a short per-server
retry budget is kept (a real transaction must survive a transient hiccup),
but a wall-clock **global deadline** caps the whole batch so a dialog never
freezes behind one unresponsive server.
4. **Live feedback + reliable elapsed-time counter**: `on_each(...)` updates the
dialog as results arrive, and `on_tick()` refreshes an elapsed-time counter
(`Xs / DEADLINEs`) so the user always knows progress and the maximum wait.
### Perché è thread-safe
`Network.send_http_on_proxy()` usa `asyncio.run_coroutine_threadsafe(coro, loop)` e poi
`coro.result()`: ogni chiamata schedula la propria coroutine sullo stesso loop asyncio
condiviso di Electrum e blocca **solo il proprio worker thread**. Più chiamate
concorrenti sono quindi sicure → `ThreadPoolExecutor` dà vero parallelismo.
### Why it is thread-safe
`Network.send_http_on_proxy()` uses `asyncio.run_coroutine_threadsafe(coro,
loop)` and then `coro.result()`: every call schedules its own coroutine on
Electrum's shared asyncio loop and blocks **only its own worker thread**.
Multiple concurrent calls are therefore safe → `ThreadPoolExecutor` gives true
parallelism.
Gli aggiornamenti UI passano per `BalWaitingDialog.update()` che emette un
`pyqtSignal` → marshalling automatico sul thread GUI. I callback dai worker thread
possono quindi aggiornare il dialog in sicurezza.
UI updates go through `BalWaitingDialog.update()` / the dialog's `pyqtSignal`,
which marshals to the GUI thread automatically. Callbacks from worker threads
can therefore update the dialog safely.
### Why the counter is driven from the calling thread (important)
An earlier attempt refreshed the elapsed-time counter from a separate raw
`threading.Thread` heartbeat that emitted the `pyqtSignal`. That proved
**unreliable**: a `pyqtSignal` emitted from a raw (non-Qt) Python thread inside
the wizard's `TaskThread` was not reliably marshalled and the dialog never
repainted — the counter was invisible.
The fix: the parallel helpers accept an **`on_tick` callback that is invoked
periodically from the CALLING thread** (the same thread that already drives
`on_each` and successfully repaints). The helpers poll the futures in short
slices (`concurrent.futures.wait(..., timeout=tick_interval)`) and call
`on_tick()` between waits. No heartbeat thread is used anymore.
---
## 3. File modificati (tutto su GitHub `Bitcoin-after-life/test`, branch `feature/networking-parallelo`)
## 3. Modified files (all on GitHub `Bitcoin-after-life/test`, branch `feature/networking-parallelo`)
### 3.1 `bal/core/willexecutors.py`
**`send_request(...)`** — aggiunti due parametri keyword-only:
**Networking constants** (module level, also exposed as `Willexecutors` class
attributes for a single source of truth in the GUI):
```python
DEFAULT_TIMEOUT = 5 # interactive ops (ping/info/list)
PUSH_TIMEOUT = 8 # broadcast (pushtxs)
PUSH_MAX_RETRIES = 2
PUSH_RETRY_SLEEP = 1
PUSH_GLOBAL_DEADLINE = 30 # wall-clock cap for the whole parallel push
CHECK_TIMEOUT = 8 # check (searchtx)
CHECK_MAX_RETRIES = 1
CHECK_RETRY_SLEEP = 1
CHECK_GLOBAL_DEADLINE = 30 # wall-clock cap for the whole parallel check
```
Worst case per server is now ~26s (push) / ~17s (check) instead of ~140s, and
the global deadline guarantees the dialog proceeds within 30s regardless.
**`send_request(...)`** — keyword-only retry controls:
```python
def send_request(method, url, data=None, *, timeout=10, handle_response=None,
count_reply=0, max_retries=10, retry_sleep=3):
```
- `max_retries` / `retry_sleep` controllano i retry sui timeout.
- **Default invariato** (`10` / `3s`) → il push critico mantiene il comportamento storico.
- Le operazioni interattive passano `max_retries=0` → fast-fail.
- Defaults unchanged → callers that need the historical behaviour are
unaffected.
- Interactive callers pass `max_retries=0` → fast-fail.
**`get_info_task(...)`** — fast-fail di default:
```python
def get_info_task(url, willexecutor, *, timeout=DEFAULT_TIMEOUT,
max_retries=0, retry_sleep=0):
```
- Se la risposta non è un `dict` (timeout/vuoto) → `status="KO"`.
**`get_info_task(...)`** — fast-fail by default (`max_retries=0`); a
timeout/empty response yields `status="KO"`.
**NUOVO `ping_servers_parallel(willexecutors, *, on_each=None, max_workers=8, timeout=DEFAULT_TIMEOUT)`**
- `ThreadPoolExecutor` + `as_completed`; un worker per server (`_ping_one`).
- Muta `willexecutors` in place (come il vecchio `ping_servers`).
- Invoca `on_each(url, we, ok)` man mano che arrivano i risultati.
- Un worker che esplode non blocca gli altri (try/except difensivo).
**`check_transaction(...)`** — now accepts `timeout`/`max_retries`/`retry_sleep`
(defaults from the `CHECK_*` constants) and forwards them to `send_request`,
replacing the old ~140s default storm.
**NUOVO `push_transactions_parallel(willexecutors, *, on_each=None, max_workers=8)`**
- Push in parallelo solo verso le voci con chiave `"txs"`.
- Ogni server mantiene i propri retry (`push_transactions_to_willexecutor`).
- `on_each(url, we, ok, exc)`; raccoglie `AlreadyPresentException` separatamente.
- Ritorna `{url: (ok, exc)}`.
**NEW `ping_servers_parallel(willexecutors, *, on_each=None, max_workers=8,
timeout=DEFAULT_TIMEOUT, on_tick=None, tick_interval=1.0)`**
- `ThreadPoolExecutor`; polls futures in slices and calls `on_tick()` from the
calling thread; mutates `willexecutors` in place; invokes
`on_each(url, we, ok)` as results arrive; a worker exception never blocks the
others (defensive try/except).
**`DEFAULT_TIMEOUT = 5`** (costante a livello di modulo).
**NEW `push_transactions_parallel(willexecutors, *, on_each=None, max_workers=8,
deadline=PUSH_GLOBAL_DEADLINE, on_timeout=None, on_tick=None,
tick_interval=1.0)`**
- Parallel push only to entries that have a `"txs"` key; each server keeps its
short retry budget.
- `on_each(url, we, ok, exc)` per server; `on_timeout(url, we)` for servers
still pending when the global deadline elapses; `on_tick()` for the counter.
- Manual pool (no `with`) so `shutdown(wait=False, cancel_futures=True)` does
not block on a hung worker once the deadline is reached.
- Returns `{url: (ok, exc)}` for the servers that answered in time.
**NEW `check_transactions_parallel(items, *, on_each=None, max_workers=8,
deadline=CHECK_GLOBAL_DEADLINE, on_timeout=None, on_tick=None,
tick_interval=1.0)`**
- Same design as the push helper but for the **Check** (searchtx) operation.
- `items` is an iterable of `(wid, url)` pairs; `_check_one` calls
`check_transaction`.
- `on_each(wid, url, result_or_None, exc)`, `on_timeout(wid, url)`, `on_tick()`.
- Returns `{wid: (result_or_None, exc)}`.
### 3.2 `bal/gui/qt/window.py`
- **`ping_willexecutors_task(self, wes)`** riscritto su `ping_servers_parallel(...)`
con feedback live (set `pinged`/`failed`, `get_title()` mostra "Ok"/"Ko"/"waiting...").
- **`push_transactions_to_willexecutors(self, force=False)`** riscritto su
`push_transactions_parallel(...)`. `on_each` fa book-keeping + update UI thread-safe;
i server "already present" sono raccolti in `already_present[]` e il loro
`check_transaction` viene eseguito dopo, nel task thread (logica di check originale intatta).
- **`fetch_will_executors_list(...)`** download fast-fail:
`send_request("get", url, timeout=10, max_retries=1, retry_sleep=1)`.
- **`ping_willexecutors_task(self, wes)`** rewritten on `ping_servers_parallel`
with live feedback and a counter `Ping Will-Executors: 2/3 (3s / 30s)` driven
by `on_tick` from the calling thread.
- **`push_transactions_to_willexecutors(self, force=False)`** rewritten on
`push_transactions_parallel`; `on_each` does thread-safe book-keeping + UI
update; "already present" servers are verified afterwards (original check
logic intact).
- **`check_transactions_task(self, will)`** rewritten on
`check_transactions_parallel`; shows `Checking transactions: 2/5 (4s / 30s)`,
reusing the original `set_check_willexecutor(...)` per-item logic inside
`on_each` (and `set_check_willexecutor(None)` on `on_timeout`).
- **`fetch_will_executors_list(...)`** fast-fail download
(`timeout=10, max_retries=1, retry_sleep=1`); the download dialog shows
`Downloading will-executors list... (Xs / 45s)`.
### 3.3 `bal/core/util.py` — BUGFIX (regressione pre-esistente)
### 3.3 `bal/gui/qt/dialogs.py`
In `get_value_amount` (riga 324) era stato erroneamente usato `Util.in_output(...)`
(ritorna `bool`) al posto di `Util.din_output(...)` (ritorna la tupla
`(same_amount, same_address)`), causando:
- **`BalBuildWillDialog.loop_push`** (the "Building Will" wizard broadcast step)
rewritten on `push_transactions_parallel` with the `on_tick` counter
`Broadcasting 2/3 (5s / 30s)`. The previous raw heartbeat thread was removed.
### 3.4 `bal/gui/qt/plugin.py` — status-bar icon (restored)
`create_status_bar` re-adds the BAL `StatusBarButton` (bottom-right of the
Electrum status bar). It shows that the plugin is installed and opens the plugin
settings on click; it also de-duplicates the button per window. (Comments in
English.)
### 3.5 `bal/gui/qt/lists.py` and `bal/gui/qt/widgets.py` — GUI usability
- **Tooltips** (hover) on the Will toolbar icons, all in English:
Wizard (`Wizard - Build your will`), Delivery time (truck), Check Alive
(siren), Calendar, Check (refresh).
- **Toolbar order** changed to:
`Wizard | Delivery time | Check Alive | Calendar | Check`; layout margins
tightened so everything fits the Will window.
### 3.6 `bal/core/util.py` — BUGFIX (pre-existing regression)
In `get_value_amount` (line 324) `Util.in_output(...)` (returns `bool`) had been
used instead of `Util.din_output(...)` (returns the tuple
`(same_amount, same_address)`), causing:
```
TypeError: cannot unpack non-iterable bool object
```
**Corretto** ripristinando `din_output`. Bug scoperto eseguendo i test ufficiali del
repo Gitea (`tests/test_core_util.py::test_get_value_amount`).
**Fixed** by restoring `din_output`. Found by running the official Gitea tests
(`tests/test_core_util.py::test_get_value_amount`).
---
## 4. Verifica (ruff + test ufficiali)
## 4. Verification (ruff + official tests)
### 4.1 ruff (lint / PEP8)
- `ruff check` sul codice nuovo: **nessun nuovo problema** introdotto
(i `F403/F405/F401` presenti derivano dal pattern `from .common import *`
dell'originale; conteggio identico HEAD vs working tree: 121 = 121).
- Le funzioni parallele nuove rispettano il limite di 88 caratteri (0 `E501`).
- `ruff check tests/parallel_ping_test.py`**All checks passed**.
- `ruff check` on the new code: **no new issues** introduced. The `F403/F405/
F401` warnings come from the original `from .common import *` pattern;
per-file counts are identical between HEAD and the working tree.
- The new parallel functions add **0 `E501`** (line-length) issues; in
`window.py` the count actually decreased after the rewrite.
- `ruff check tests/parallel_ping_test.py` → no new issues.
### 4.2 Test ufficiali del repo Gitea `kaibot/bal-plugin-ai/tests`
Eseguiti contro il codice refactorizzato (con le modifiche networking):
### 4.2 Official tests from the Gitea repo `kaibot/bal-plugin-ai/tests`
Run against the refactored code (with all the networking + UI changes):
| Suite | Esito |
|-------|-------|
| `test_core_*` (pytest) | **117 passed** |
| `test_gui_*` (pytest) | **65 passed** |
| Suite | Result |
|-------|--------|
| `test_core_*` + `test_gui_*` (pytest) | **182 passed** |
| `smoke_test.py` | OK |
| `external_zip_test.py` | OK |
| `windows_overflow_test.py` | OK |
| `gui_fixes_test.py` | OK |
| `parallel_ping_test.py` (nuovo) | OK — `0.50s` per 8 server (sequenziale ~`4.00s`) |
| `parallel_ping_test.py` (new) | OK — parallel ping/push/check ~`0.50s` for 8 servers (sequential would be ~`4.00s`); global deadline enforced; `on_tick` fired from the calling thread; static checks that the dialogs use the parallel helpers + the `Xs / Ns` counter |
Comandi (come da README):
Commands (as per README):
```bash
QT_QPA_PLATFORM=offscreen PYTHONPATH=<electrum-src> \
python3 -m pytest tests/ -q
@@ -135,30 +215,39 @@ QT_QPA_PLATFORM=offscreen PYTHONPATH=<electrum-src> \
python3 tests/smoke_test.py electrum.plugins.bal
QT_QPA_PLATFORM=offscreen PYTHONPATH=<electrum-src> \
python3 tests/external_zip_test.py bal-electrum-plugin.zip
QT_QPA_PLATFORM=offscreen PYTHONPATH=<electrum-src> \
python3 tests/parallel_ping_test.py bal
```
---
## 5. Note di integrazione / rischi
## 5. Integration notes / risks
- **Nessuna modifica al protocollo server**: solo il *come* (parallelo) e il *quando*
(retry) delle chiamate cambia, non i payload.
- **Push transazioni**: i retry per-server sono mantenuti apposta, per non perdere
una transazione reale per un hiccup. Solo il ping/info/download usa fast-fail.
- **`max_workers=8`** è prudente; con molti server (es. 20) si può alzare, ma 8
worker già abbattono il tempo totale al server più lento.
- **Thread/UI**: tutti gli aggiornamenti UI dai worker passano per
`BalWaitingDialog.update()` (pyqtSignal) → safe. Non toccare quel canale.
- **Compatibilità**: firme retro-compatibili (i nuovi parametri sono keyword-only
con default che preservano il vecchio comportamento).
- **No change to the server protocol**: only the *how* (parallel) and the
*when* (retries/deadline) of the calls changed, not the payloads.
- **Push transactions**: per-server retries are intentionally kept so a real
transaction is not lost to a transient hiccup; only ping/info/download use
fast-fail. The global deadline marks unanswered servers as failed (`on_timeout`)
so the user can retry later.
- **`max_workers=8`** is conservative; with many servers (e.g. 20) it can be
raised, but 8 workers already collapse the total time to the slowest server.
- **Thread/UI**: all UI updates from workers go through `pyqtSignal`-based
dialog updates; the periodic counter is driven by `on_tick` from the calling
thread. Do **not** reintroduce a raw heartbeat thread emitting signals — it
does not repaint reliably.
- **Compatibility**: signatures are backward compatible (new parameters are
keyword-only with defaults that preserve the old behaviour).
---
## 6. Come provare
## 6. How to test
1. Installare lo zip `bal-electrum-plugin.zip` (Tools → Plugins → install from file).
2. Configurare più Will-Executor, includendone **almeno uno irraggiungibile**.
3. Lanciare invio transazioni / ping: il dialog mostra lo stato server-per-server
e **non resta più bloccato** sul server morto.
1. Install `bal-electrum-plugin.zip` (Tools → Plugins → install from file).
Fully close and reopen Electrum to avoid the cached zip import.
2. Configure several Will-Executors, including **at least one unreachable**.
3. Run push / ping / Check: each dialog shows per-server status plus a counter
`N/total (Xs / 30s)` and **no longer freezes** on the dead server — within
the global deadline the operation reports the dead server and proceeds.
SHA-256 dello zip stampato da `build_zip.py` a fine build (verificare l'integrità).
The SHA-256 of the zip is printed by `build_zip.py` at the end of the build
(use it to verify integrity).