docs: update knowledge base to match current codebase

- Fix framework references (actix-web, not hyper)
- Update all env var names (BAL_SERVER_*/BAL_PUSHER_* prefix)
- Add validation.rs module documentation
- Fix function signatures in xpub.rs and db.rs
- Update API response formats (InfoResponse, StatsResponse)
- Fix database schema (date_creation/date_update, push_err, tbl_stats)
- Mark fixed vulnerabilities with current status
- Add Docker support and actix tuning documentation
- Remove outdated references (confy, bal-stats.rs.dontcompile)
- Add regression test summary table
This commit is contained in:
2026-07-20 15:43:20 -04:00
parent 734b2ee71d
commit eacb2e1450
8 changed files with 778 additions and 598 deletions

View File

@@ -12,21 +12,19 @@ User
| HTTP POST (raw hex transactions)
v
+-----------------+
| bal-server | (hyper + tokio, async)
| bal-server | (actix-web 4.9.0 + actix-governor, async)
| (src/bin/bal-server.rs) |
+-----------------+
| SQLite insert (db.rs)
| SQLite insert (db.rs, Arc<Mutex<Connection>>)
v
bal.db
bal.db (WAL mode)
| (transactions with status=0, waiting locktime)
|
| ZMQ (hashblock / rawblock)
| ZMQ (hashblock)
v
+-----------------+
+-----------------+
| bal-pusher | (async, ZMQ + RPC + reqwest)
| bal-pusher | (tokio, ZMQ + RPC + reqwest)
| (src/bin/bal-pusher.rs)
+-----------------+
+-----------------+
| bitcoincore-rpc
| sendrawtransaction
@@ -36,13 +34,13 @@ User
## Data Flow (Transaction Lifecycle)
1. **Submission**: A client sends one or more raw hex transactions to the `pushtxs` endpoint.
2. **Validation**: The `bal-server` parses each transaction using `bitcoin::Transaction`. It checks for the fee output, extracts inputs/outputs, and validates the locktime.
3. **Storage**: Valid transactions are stored in `tbl_tx` with `status = 0` (waiting). The inputs and outputs are stored in `tbl_inp` and `tbl_out`.
4. **Monitoring**: The `bal-pusher` listens to the ZMQ `hashblock` topic. When a new block is detected, it fetches the `mediantime` via `getblockchaininfo` (or via the block's median time in the enhanced version).
5. **Evaluation**: The pusher queries the database for transactions with `status=0` and compares their locktime to the current blockchain median time.
6. **Broadcast**: If the locktime is satisfied, the pusher sends the transaction via `sendrawtransaction` and updates the status to `1` (sent) or `2` (failed if the RPC returns an error).
7. **Statistics**: The pusher periodically sends statistics to a remote server (`welist`) using a signed POST request. The server also collects stats on its own.
1. **Submission**: A client sends one or more raw hex transactions to the `pushtxs` endpoint (newline-separated).
2. **Validation**: The `bal-server` parses each transaction using `bitcoin::Transaction` via `consensus::deserialize`. It checks for the fee output, extracts inputs/outputs, and validates the locktime.
3. **Storage**: Valid transactions are stored in `tbl_tx` with `status = 0` (waiting). The inputs and outputs are stored in `tbl_inp` and `tbl_out`. Batch inserts use `UNION ALL SELECT` for efficiency.
4. **Monitoring**: The `bal-pusher` listens to the ZMQ `hashblock` topic with a 5-second receive timeout. When a new block is detected, it fetches blockchain info via RPC.
5. **Evaluation**: The pusher queries the database for transactions with `status=0` and compares their locktime against the blockchain's best block height or median time (for timestamp-based locktimes above `LOCKTIME_THRESHOLD`).
6. **Broadcast**: If the locktime is satisfied, the pusher sends the transaction via `sendrawtransaction` and updates the status to `1` (sent) or `2` (failed with error stored in `push_err`).
7. **Statistics**: The pusher periodically calculates statistics and sends them to a remote `welist` server using an Ed25519-signed POST request. The server also exposes stats via the `GET /:network/stats` endpoint if `expose_stats` is enabled.
## State Machine
@@ -57,12 +55,15 @@ User
The `status` field in `tbl_tx` is an integer:
- `0`: Waiting for locktime.
- `1`: Successfully sent to the network.
- `2`: Failed (e.g., RPC error `-25 bad-txns-inputs-missingorspent`).
- `2`: Failed (e.g., RPC error `-25 bad-txns-inputs-missingorspent`). Error details stored in `push_err`.
## Error Handling Strategy
The codebase is currently inconsistent with error handling. The `bal-server` uses `unwrap()` on many critical paths (e.g., `sqlite::open`, `Regex::new`, body parsing), which causes panics in the async runtime. The `bal-pusher` also panics on RPC connection failures (`panic!("impossible to get client {}", e)`) which crashes the entire ZMQ loop.
The codebase has been hardened with comprehensive error handling:
- **`bal-server`**: Uses `actix-web`'s built-in error handling. All `unwrap()`/`expect()` calls have been replaced with safe `match`/`if let` error propagation, returning appropriate HTTP status codes (400, 404, 500).
- **`bal-pusher`**: ZMQ `recv` uses `set_rcvtimeo(5000)` with a match/timeout handler. RPC connection failures log errors and retry with a sleep interval instead of panicking. The pusher logs warnings for consecutive ZMQ timeouts (~1 hour threshold).
- **`db.rs`**: Database operations use `Result` types. The `open_db` function validates paths before opening. WAL mode is set with retry logic.
## Logging and Monitoring
The project uses `env_logger` and `log`. By default, `RUST_LOG=info` is set. The `bal-pusher` sends signed statistics to a remote server. The server exposes a `stats` endpoint (`/<network>/stats`) if `expose_stats` is enabled.
The project uses `env_logger` and `log`. By default, `RUST_LOG=info` is set. The `bal-pusher` sends signed statistics to a remote server. The server exposes a `stats` endpoint (`/<network>/stats`) if `expose_stats` is enabled. The actix-web `Logger::default()` middleware logs all HTTP requests/responses.