From 5b334b4193c5c21c7a719de65c58fcfc277aea01 Mon Sep 17 00:00:00 2001 From: GenSpark AI Developer Date: Mon, 15 Jun 2026 18:50:45 +0000 Subject: [PATCH] docs: translate networking report to English and update with Check/counters/GUI Translate REPORT_NETWORKING_PARALLELO.md to English and bring it up to date: - parallel push/ping/check with fast-fail timeouts + global deadline - reliable Xs/Ns elapsed-time counter driven by on_tick from the calling thread (replacing the unreliable raw heartbeat thread) - status-bar icon restore, toolbar tooltips and reorder - updated verification section (182 official tests, ruff 0 new issues) --- REPORT_NETWORKING_PARALLELO.md | 289 +++++++++++++++++++++------------ 1 file changed, 189 insertions(+), 100 deletions(-) diff --git a/REPORT_NETWORKING_PARALLELO.md b/REPORT_NETWORKING_PARALLELO.md index 6cbfbc0..d77d737 100644 --- a/REPORT_NETWORKING_PARALLELO.md +++ b/REPORT_NETWORKING_PARALLELO.md @@ -1,133 +1,213 @@ -# Report tecnico — Networking parallelo (anti-freeze Will-Executor) +# Technical report — Parallel networking (Will-Executor anti-freeze) + UI feedback -**Destinatario:** programmatore esterno / manutentore del plugin -**Autore:** refactoring AI (lavoro su GitHub `Bitcoin-after-life/test`) -**Data:** 2026-06-15 -**Branch:** `feature/networking-parallelo` -**Repository Gitea privato `kaibot/bal-plugin-ai`: NON modificato** (per richiesta esplicita). +**Audience:** external programmer / plugin maintainer +**Author:** AI refactoring work (on GitHub `Bitcoin-after-life/test`) +**Date:** 2026-06-15 +**Branch:** `feature/networking-parallelo` (Pull Request #4) +**Private Gitea repo `kaibot/bal-plugin-ai`: NOT modified** (per explicit request; cloned read-only only to run the official tests). --- -## 1. Problema +## 1. Problem -Quando il plugin contatta i server Will-Executor (invio transazioni, ping/aggiornamento -eredità, download lista), lo fa **in sequenza**. Se un server non risponde, il thread -resta bloccato sui timeout di connessione e, peggio, sui **retry**: +When the plugin contacts the Will-Executor servers (pushing transactions, +pinging/refreshing the inheritance, downloading the list, and **checking** +transactions), it used to do it **sequentially**. If a server did not answer, +the thread stayed blocked on the connection timeouts and, worse, on the +**retries**: -- `send_request` riprovava fino a **10 volte** con `time.sleep(3)` ad ogni timeout - → circa **130 secondi per ogni server irraggiungibile**, sommati uno dopo l'altro. +- `send_request` retried up to **10 times** with `time.sleep(3)` on every + timeout → roughly **~140 seconds per unreachable server**, summed one after + another. -Conseguenze: -- Con pochi server già si avverte; con **20 server** diventa ingestibile. -- L'utente vede "Rimani in attesa — Non risponde" senza capire cosa succede. -- Un singolo server morto blocca l'intera operazione. +Consequences: +- Noticeable already with a few servers; with **20 servers** it became + unusable. +- The user saw "Stay waiting — Not responding" with no idea what was happening. +- A single dead server blocked the whole operation. --- -## 2. Soluzione (in sintesi) +## 2. Solution (overview) -1. **Parallelismo** con `ThreadPoolExecutor`: i server vengono contattati - contemporaneamente. Il tempo totale ≈ server **più lento**, non la **somma**. -2. **Fast-fail** per le operazioni interattive (ping/info/download): niente retry-storm, - un solo timeout breve e il server viene marcato "KO". -3. **Feedback live**: callback `on_each(...)` che aggiorna la finestra di attesa - server-per-server, in modo thread-safe. -4. **Push transazioni**: parallelo, ma mantiene i retry per ogni singolo server - (una transazione reale non deve andare persa per un hiccup transitorio). +1. **Parallelism** with `ThreadPoolExecutor`: servers are contacted + concurrently. Total time ≈ the **slowest** server, not the **sum**. +2. **Fast-fail** for interactive operations (ping/info/download): no retry + storm, a single short timeout and the server is marked "KO". +3. **Aggressive timeouts + global deadline** for push/check: a short per-server + retry budget is kept (a real transaction must survive a transient hiccup), + but a wall-clock **global deadline** caps the whole batch so a dialog never + freezes behind one unresponsive server. +4. **Live feedback + reliable elapsed-time counter**: `on_each(...)` updates the + dialog as results arrive, and `on_tick()` refreshes an elapsed-time counter + (`Xs / DEADLINEs`) so the user always knows progress and the maximum wait. -### Perché è thread-safe -`Network.send_http_on_proxy()` usa `asyncio.run_coroutine_threadsafe(coro, loop)` e poi -`coro.result()`: ogni chiamata schedula la propria coroutine sullo stesso loop asyncio -condiviso di Electrum e blocca **solo il proprio worker thread**. Più chiamate -concorrenti sono quindi sicure → `ThreadPoolExecutor` dà vero parallelismo. +### Why it is thread-safe +`Network.send_http_on_proxy()` uses `asyncio.run_coroutine_threadsafe(coro, +loop)` and then `coro.result()`: every call schedules its own coroutine on +Electrum's shared asyncio loop and blocks **only its own worker thread**. +Multiple concurrent calls are therefore safe → `ThreadPoolExecutor` gives true +parallelism. -Gli aggiornamenti UI passano per `BalWaitingDialog.update()` che emette un -`pyqtSignal` → marshalling automatico sul thread GUI. I callback dai worker thread -possono quindi aggiornare il dialog in sicurezza. +UI updates go through `BalWaitingDialog.update()` / the dialog's `pyqtSignal`, +which marshals to the GUI thread automatically. Callbacks from worker threads +can therefore update the dialog safely. + +### Why the counter is driven from the calling thread (important) +An earlier attempt refreshed the elapsed-time counter from a separate raw +`threading.Thread` heartbeat that emitted the `pyqtSignal`. That proved +**unreliable**: a `pyqtSignal` emitted from a raw (non-Qt) Python thread inside +the wizard's `TaskThread` was not reliably marshalled and the dialog never +repainted — the counter was invisible. + +The fix: the parallel helpers accept an **`on_tick` callback that is invoked +periodically from the CALLING thread** (the same thread that already drives +`on_each` and successfully repaints). The helpers poll the futures in short +slices (`concurrent.futures.wait(..., timeout=tick_interval)`) and call +`on_tick()` between waits. No heartbeat thread is used anymore. --- -## 3. File modificati (tutto su GitHub `Bitcoin-after-life/test`, branch `feature/networking-parallelo`) +## 3. Modified files (all on GitHub `Bitcoin-after-life/test`, branch `feature/networking-parallelo`) ### 3.1 `bal/core/willexecutors.py` -**`send_request(...)`** — aggiunti due parametri keyword-only: +**Networking constants** (module level, also exposed as `Willexecutors` class +attributes for a single source of truth in the GUI): +```python +DEFAULT_TIMEOUT = 5 # interactive ops (ping/info/list) + +PUSH_TIMEOUT = 8 # broadcast (pushtxs) +PUSH_MAX_RETRIES = 2 +PUSH_RETRY_SLEEP = 1 +PUSH_GLOBAL_DEADLINE = 30 # wall-clock cap for the whole parallel push + +CHECK_TIMEOUT = 8 # check (searchtx) +CHECK_MAX_RETRIES = 1 +CHECK_RETRY_SLEEP = 1 +CHECK_GLOBAL_DEADLINE = 30 # wall-clock cap for the whole parallel check +``` +Worst case per server is now ~26s (push) / ~17s (check) instead of ~140s, and +the global deadline guarantees the dialog proceeds within 30s regardless. + +**`send_request(...)`** — keyword-only retry controls: ```python def send_request(method, url, data=None, *, timeout=10, handle_response=None, count_reply=0, max_retries=10, retry_sleep=3): ``` -- `max_retries` / `retry_sleep` controllano i retry sui timeout. -- **Default invariato** (`10` / `3s`) → il push critico mantiene il comportamento storico. -- Le operazioni interattive passano `max_retries=0` → fast-fail. +- Defaults unchanged → callers that need the historical behaviour are + unaffected. +- Interactive callers pass `max_retries=0` → fast-fail. -**`get_info_task(...)`** — fast-fail di default: -```python -def get_info_task(url, willexecutor, *, timeout=DEFAULT_TIMEOUT, - max_retries=0, retry_sleep=0): -``` -- Se la risposta non è un `dict` (timeout/vuoto) → `status="KO"`. +**`get_info_task(...)`** — fast-fail by default (`max_retries=0`); a +timeout/empty response yields `status="KO"`. -**NUOVO `ping_servers_parallel(willexecutors, *, on_each=None, max_workers=8, timeout=DEFAULT_TIMEOUT)`** -- `ThreadPoolExecutor` + `as_completed`; un worker per server (`_ping_one`). -- Muta `willexecutors` in place (come il vecchio `ping_servers`). -- Invoca `on_each(url, we, ok)` man mano che arrivano i risultati. -- Un worker che esplode non blocca gli altri (try/except difensivo). +**`check_transaction(...)`** — now accepts `timeout`/`max_retries`/`retry_sleep` +(defaults from the `CHECK_*` constants) and forwards them to `send_request`, +replacing the old ~140s default storm. -**NUOVO `push_transactions_parallel(willexecutors, *, on_each=None, max_workers=8)`** -- Push in parallelo solo verso le voci con chiave `"txs"`. -- Ogni server mantiene i propri retry (`push_transactions_to_willexecutor`). -- `on_each(url, we, ok, exc)`; raccoglie `AlreadyPresentException` separatamente. -- Ritorna `{url: (ok, exc)}`. +**NEW `ping_servers_parallel(willexecutors, *, on_each=None, max_workers=8, +timeout=DEFAULT_TIMEOUT, on_tick=None, tick_interval=1.0)`** +- `ThreadPoolExecutor`; polls futures in slices and calls `on_tick()` from the + calling thread; mutates `willexecutors` in place; invokes + `on_each(url, we, ok)` as results arrive; a worker exception never blocks the + others (defensive try/except). -**`DEFAULT_TIMEOUT = 5`** (costante a livello di modulo). +**NEW `push_transactions_parallel(willexecutors, *, on_each=None, max_workers=8, +deadline=PUSH_GLOBAL_DEADLINE, on_timeout=None, on_tick=None, +tick_interval=1.0)`** +- Parallel push only to entries that have a `"txs"` key; each server keeps its + short retry budget. +- `on_each(url, we, ok, exc)` per server; `on_timeout(url, we)` for servers + still pending when the global deadline elapses; `on_tick()` for the counter. +- Manual pool (no `with`) so `shutdown(wait=False, cancel_futures=True)` does + not block on a hung worker once the deadline is reached. +- Returns `{url: (ok, exc)}` for the servers that answered in time. + +**NEW `check_transactions_parallel(items, *, on_each=None, max_workers=8, +deadline=CHECK_GLOBAL_DEADLINE, on_timeout=None, on_tick=None, +tick_interval=1.0)`** +- Same design as the push helper but for the **Check** (searchtx) operation. +- `items` is an iterable of `(wid, url)` pairs; `_check_one` calls + `check_transaction`. +- `on_each(wid, url, result_or_None, exc)`, `on_timeout(wid, url)`, `on_tick()`. +- Returns `{wid: (result_or_None, exc)}`. ### 3.2 `bal/gui/qt/window.py` -- **`ping_willexecutors_task(self, wes)`** riscritto su `ping_servers_parallel(...)` - con feedback live (set `pinged`/`failed`, `get_title()` mostra "Ok"/"Ko"/"waiting..."). -- **`push_transactions_to_willexecutors(self, force=False)`** riscritto su - `push_transactions_parallel(...)`. `on_each` fa book-keeping + update UI thread-safe; - i server "already present" sono raccolti in `already_present[]` e il loro - `check_transaction` viene eseguito dopo, nel task thread (logica di check originale intatta). -- **`fetch_will_executors_list(...)`** download fast-fail: - `send_request("get", url, timeout=10, max_retries=1, retry_sleep=1)`. +- **`ping_willexecutors_task(self, wes)`** rewritten on `ping_servers_parallel` + with live feedback and a counter `Ping Will-Executors: 2/3 (3s / 30s)` driven + by `on_tick` from the calling thread. +- **`push_transactions_to_willexecutors(self, force=False)`** rewritten on + `push_transactions_parallel`; `on_each` does thread-safe book-keeping + UI + update; "already present" servers are verified afterwards (original check + logic intact). +- **`check_transactions_task(self, will)`** rewritten on + `check_transactions_parallel`; shows `Checking transactions: 2/5 (4s / 30s)`, + reusing the original `set_check_willexecutor(...)` per-item logic inside + `on_each` (and `set_check_willexecutor(None)` on `on_timeout`). +- **`fetch_will_executors_list(...)`** fast-fail download + (`timeout=10, max_retries=1, retry_sleep=1`); the download dialog shows + `Downloading will-executors list... (Xs / 45s)`. -### 3.3 `bal/core/util.py` — BUGFIX (regressione pre-esistente) +### 3.3 `bal/gui/qt/dialogs.py` -In `get_value_amount` (riga 324) era stato erroneamente usato `Util.in_output(...)` -(ritorna `bool`) al posto di `Util.din_output(...)` (ritorna la tupla -`(same_amount, same_address)`), causando: +- **`BalBuildWillDialog.loop_push`** (the "Building Will" wizard broadcast step) + rewritten on `push_transactions_parallel` with the `on_tick` counter + `Broadcasting 2/3 (5s / 30s)`. The previous raw heartbeat thread was removed. + +### 3.4 `bal/gui/qt/plugin.py` — status-bar icon (restored) + +`create_status_bar` re-adds the BAL `StatusBarButton` (bottom-right of the +Electrum status bar). It shows that the plugin is installed and opens the plugin +settings on click; it also de-duplicates the button per window. (Comments in +English.) + +### 3.5 `bal/gui/qt/lists.py` and `bal/gui/qt/widgets.py` — GUI usability + +- **Tooltips** (hover) on the Will toolbar icons, all in English: + Wizard (`Wizard - Build your will`), Delivery time (truck), Check Alive + (siren), Calendar, Check (refresh). +- **Toolbar order** changed to: + `Wizard | Delivery time | Check Alive | Calendar | Check`; layout margins + tightened so everything fits the Will window. + +### 3.6 `bal/core/util.py` — BUGFIX (pre-existing regression) + +In `get_value_amount` (line 324) `Util.in_output(...)` (returns `bool`) had been +used instead of `Util.din_output(...)` (returns the tuple +`(same_amount, same_address)`), causing: ``` TypeError: cannot unpack non-iterable bool object ``` -**Corretto** ripristinando `din_output`. Bug scoperto eseguendo i test ufficiali del -repo Gitea (`tests/test_core_util.py::test_get_value_amount`). +**Fixed** by restoring `din_output`. Found by running the official Gitea tests +(`tests/test_core_util.py::test_get_value_amount`). --- -## 4. Verifica (ruff + test ufficiali) +## 4. Verification (ruff + official tests) ### 4.1 ruff (lint / PEP8) -- `ruff check` sul codice nuovo: **nessun nuovo problema** introdotto - (i `F403/F405/F401` presenti derivano dal pattern `from .common import *` - dell'originale; conteggio identico HEAD vs working tree: 121 = 121). -- Le funzioni parallele nuove rispettano il limite di 88 caratteri (0 `E501`). -- `ruff check tests/parallel_ping_test.py` → **All checks passed**. +- `ruff check` on the new code: **no new issues** introduced. The `F403/F405/ + F401` warnings come from the original `from .common import *` pattern; + per-file counts are identical between HEAD and the working tree. +- The new parallel functions add **0 `E501`** (line-length) issues; in + `window.py` the count actually decreased after the rewrite. +- `ruff check tests/parallel_ping_test.py` → no new issues. -### 4.2 Test ufficiali del repo Gitea `kaibot/bal-plugin-ai/tests` -Eseguiti contro il codice refactorizzato (con le modifiche networking): +### 4.2 Official tests from the Gitea repo `kaibot/bal-plugin-ai/tests` +Run against the refactored code (with all the networking + UI changes): -| Suite | Esito | -|-------|-------| -| `test_core_*` (pytest) | **117 passed** | -| `test_gui_*` (pytest) | **65 passed** | +| Suite | Result | +|-------|--------| +| `test_core_*` + `test_gui_*` (pytest) | **182 passed** | | `smoke_test.py` | OK | | `external_zip_test.py` | OK | | `windows_overflow_test.py` | OK | | `gui_fixes_test.py` | OK | -| `parallel_ping_test.py` (nuovo) | OK — `0.50s` per 8 server (sequenziale ~`4.00s`) | +| `parallel_ping_test.py` (new) | OK — parallel ping/push/check ~`0.50s` for 8 servers (sequential would be ~`4.00s`); global deadline enforced; `on_tick` fired from the calling thread; static checks that the dialogs use the parallel helpers + the `Xs / Ns` counter | -Comandi (come da README): +Commands (as per README): ```bash QT_QPA_PLATFORM=offscreen PYTHONPATH= \ python3 -m pytest tests/ -q @@ -135,30 +215,39 @@ QT_QPA_PLATFORM=offscreen PYTHONPATH= \ python3 tests/smoke_test.py electrum.plugins.bal QT_QPA_PLATFORM=offscreen PYTHONPATH= \ python3 tests/external_zip_test.py bal-electrum-plugin.zip +QT_QPA_PLATFORM=offscreen PYTHONPATH= \ + python3 tests/parallel_ping_test.py bal ``` --- -## 5. Note di integrazione / rischi +## 5. Integration notes / risks -- **Nessuna modifica al protocollo server**: solo il *come* (parallelo) e il *quando* - (retry) delle chiamate cambia, non i payload. -- **Push transazioni**: i retry per-server sono mantenuti apposta, per non perdere - una transazione reale per un hiccup. Solo il ping/info/download usa fast-fail. -- **`max_workers=8`** è prudente; con molti server (es. 20) si può alzare, ma 8 - worker già abbattono il tempo totale al server più lento. -- **Thread/UI**: tutti gli aggiornamenti UI dai worker passano per - `BalWaitingDialog.update()` (pyqtSignal) → safe. Non toccare quel canale. -- **Compatibilità**: firme retro-compatibili (i nuovi parametri sono keyword-only - con default che preservano il vecchio comportamento). +- **No change to the server protocol**: only the *how* (parallel) and the + *when* (retries/deadline) of the calls changed, not the payloads. +- **Push transactions**: per-server retries are intentionally kept so a real + transaction is not lost to a transient hiccup; only ping/info/download use + fast-fail. The global deadline marks unanswered servers as failed (`on_timeout`) + so the user can retry later. +- **`max_workers=8`** is conservative; with many servers (e.g. 20) it can be + raised, but 8 workers already collapse the total time to the slowest server. +- **Thread/UI**: all UI updates from workers go through `pyqtSignal`-based + dialog updates; the periodic counter is driven by `on_tick` from the calling + thread. Do **not** reintroduce a raw heartbeat thread emitting signals — it + does not repaint reliably. +- **Compatibility**: signatures are backward compatible (new parameters are + keyword-only with defaults that preserve the old behaviour). --- -## 6. Come provare +## 6. How to test -1. Installare lo zip `bal-electrum-plugin.zip` (Tools → Plugins → install from file). -2. Configurare più Will-Executor, includendone **almeno uno irraggiungibile**. -3. Lanciare invio transazioni / ping: il dialog mostra lo stato server-per-server - e **non resta più bloccato** sul server morto. +1. Install `bal-electrum-plugin.zip` (Tools → Plugins → install from file). + Fully close and reopen Electrum to avoid the cached zip import. +2. Configure several Will-Executors, including **at least one unreachable**. +3. Run push / ping / Check: each dialog shows per-server status plus a counter + `N/total (Xs / 30s)` and **no longer freezes** on the dead server — within + the global deadline the operation reports the dead server and proceeds. -SHA-256 dello zip stampato da `build_zip.py` a fine build (verificare l'integrità). +The SHA-256 of the zip is printed by `build_zip.py` at the end of the build +(use it to verify integrity).