From df8effcc600713b484b7e028ac9f5bf3ebdbb6b9 Mon Sep 17 00:00:00 2001 From: svatantrya Date: Thu, 16 Jul 2026 14:32:59 -0400 Subject: [PATCH] refactor: remove bal-pusher-enhanced and update docs/env - Remove src/bin/bal-pusher-enhanced.rs (synchronous pusher variant) - Remove bal-pusher.env and bal-pusher.sh from git tracking (now in .gitignore) - Update all documentation to remove references to bal-pusher-enhanced: * 01_project_overview.md * 02_glossary_and_bitcoin_domain.md * 03_architecture_and_data_flow.md * 04_modules_detail.md * 05_api_reference.md (remove rawblock ZMQ section, update references) * 08_security_audit.md (remove references to bal-pusher-enhanced in DoS and ZMQ sections) * 09_references_and_links.md - Build verified: cargo check passes for bal-pusher and bal-server binaries --- bal-pusher.env | 16 ---------------- bal-pusher.sh | 14 -------------- docs/01_project_overview.md | 1 - docs/02_glossary_and_bitcoin_domain.md | 8 ++++---- docs/03_architecture_and_data_flow.md | 3 --- docs/04_modules_detail.md | 20 +------------------- docs/05_api_reference.md | 13 +++---------- docs/08_security_audit.md | 6 +++--- docs/09_references_and_links.md | 5 ++--- 9 files changed, 13 insertions(+), 73 deletions(-) delete mode 100644 bal-pusher.env delete mode 100644 bal-pusher.sh diff --git a/bal-pusher.env b/bal-pusher.env deleted file mode 100644 index 6f77705..0000000 --- a/bal-pusher.env +++ /dev/null @@ -1,16 +0,0 @@ -RUST_LOG=info - -BAL_PUSHER_DB_FILE=/home/bal/bal.db -BAL_PUSHER_BITCOIN_COOKIE_FILE=/home/bitcoin/.bitcoin/.cookie -BAL_PUSHER_REGTEST_COOKIE_FILE=/home/bitcoin/.bitcoin/regtest/.cookie -BAL_PUSHER_TESTNET_COOKIE_FILE=/home/bitcoin/.bitcoin/testnet3/.cookie -BAL_PUSHER_SIGNET_COOKIE_FILE=/home/bitcoin/.bitcoin/signet/.cookie - -BAL_PUSHER_ZMQ_LISTENER=tcp://127.0.0.1:28332 - -BAL_PUSHER_SEND_STATS=true -WELIST_SERVER_URL=http://welist.bitcoin-after.life -SSL_KEY_PATH=/home/bal/privkey.pem - -#your server domain. do not final / only domain. -BAL_SERVER_URL="https://we.bitcoin-after.life" diff --git a/bal-pusher.sh b/bal-pusher.sh deleted file mode 100644 index 3003164..0000000 --- a/bal-pusher.sh +++ /dev/null @@ -1,14 +0,0 @@ -RUST_LOG=trace - -export BAL_PUSHER_DB_FILE="$(pwd)/bal.db" -#export BAL_PUSHER_BITCOIN_COOKIE_FILE=/~/.bitcoin/.cookie -#export BAL_PUSHER_REGTEST_COOKIE_FILE=/~/.bitcoin/regtest/.cookie -#export BAL_PUSHER_TESTNET_COOKIE_FILE=/~/.bitcoin/testnet3/.cookie -#export BAL_PUSHER_SIGNET_COOKIE_FILE=/~/.bitcoin/signet/.cookie - -export BAL_PUSHER_REGTEST_ZMQ_HASHBLOCK=tcp://127.0.0.1:21332 -export BAL_PUSHER_SEND_STATS=true -export WELIST_SERVER_URL=http://localhost:8085 -export BAL_SERVER_URL="http://127.0.0.1:9133" -export SSL_KEY_PATH="$(pwd)/private_key.pem" -cargo run --bin=bal-pusher regtest diff --git a/docs/01_project_overview.md b/docs/01_project_overview.md index a5f5b73..e24db68 100644 --- a/docs/01_project_overview.md +++ b/docs/01_project_overview.md @@ -21,7 +21,6 @@ The project consists of three primary binaries and two shared libraries: 1. **`bal-server`**: Async HTTP server (hyper + tokio) that exposes the API for receiving transactions and serving statistics. 2. **`bal-pusher`**: Async daemon that listens for `hashblock` ZMQ messages and pushes pending transactions to a Bitcoin node via RPC. -3. **`bal-pusher-enhanced`**: Synchronous variant of the pusher that listens for `rawblock` ZMQ messages and computes the block median time from the raw header without RPC calls. 4. **`lib.rs`**: Exports the shared modules `db` and `xpub`. 5. **`db.rs`**: All database operations and schema creation for SQLite `0.34.0`. 6. **`xpub.rs`**: Address derivation from xpub/zpub using BIP-84 and the `bitcoin` crate. diff --git a/docs/02_glossary_and_bitcoin_domain.md b/docs/02_glossary_and_bitcoin_domain.md index 5a0d4e3..ce84078 100644 --- a/docs/02_glossary_and_bitcoin_domain.md +++ b/docs/02_glossary_and_bitcoin_domain.md @@ -14,7 +14,7 @@ An **XPub** (Extended Public Key) is a master key that allows derivation of chil ## Locktime and nLockTime -A Bitcoin transaction can include a `nLockTime` field. If it is non-zero and below `500_000_000`, it is interpreted as a **block height** before which the transaction cannot be mined. If above, it is a **Unix timestamp**. The system evaluates whether the locktime has been met by comparing it against the blockchain's median time. See `src/bin/bal-pusher.rs` and `src/bin/bal-pusher-enhanced.rs` for the evaluation logic. +A Bitcoin transaction can include a `nLockTime` field. If it is non-zero and below `500_000_000`, it is interpreted as a **block height** before which the transaction cannot be mined. If above, it is a **Unix timestamp**. The system evaluates whether the locktime has been met by comparing it against the blockchain's median time. See `src/bin/bal-pusher.rs` for the evaluation logic. ## P2WPKH (Pay to Witness Public Key Hash) @@ -22,13 +22,13 @@ P2WPKH is a native SegWit output format that reduces transaction size and lowers ## Bitcoin Block Header (80 bytes) -A Bitcoin block header is a fixed 80-byte structure containing `version` (4 bytes), `previous block hash` (32 bytes), `merkle root` (32 bytes), `timestamp` (4 bytes), `bits` (4 bytes), and `nonce` (4 bytes). The `bal-pusher-enhanced` binary reads the raw 80-byte block from the `rawblock` ZMQ topic and extracts the timestamp from byte offset 68-72. This avoids the need for an RPC call to `getblockchaininfo`. See `src/bin/bal-pusher-enhanced.rs`. +A Bitcoin block header is a fixed 80-byte structure containing `version` (4 bytes), `previous block hash` (32 bytes), `merkle root` (32 bytes), `timestamp` (4 bytes), `bits` (4 bytes), and `nonce` (4 bytes). The `bal-pusher` binary uses the `hashblock` ZMQ topic and calls `getblockchaininfo` to get the `mediantime`. ## ZMQ Publisher Bitcoin Core can publish notifications over ZeroMQ. The project listens to two topics: - **`hashblock`**: Sends the 32-byte block hash when a new block is found. The `bal-pusher` uses this to trigger an update cycle. -- **`rawblock`**: Sends the full raw block (including the 80-byte header). The `bal-pusher-enhanced` uses this to derive the timestamp without RPC. +- **`rawblock`**: Sends the full raw block (including the 80-byte header). This is not used by the current `bal-pusher` implementation. The ZMQ endpoint is per-network: - Bitcoin: `tcp://127.0.0.1:28332` @@ -41,7 +41,7 @@ The ZMQ endpoint is per-network: The project communicates with a local Bitcoin Core node via the JSON-RPC interface. Key methods used are: - `sendrawtransaction`: To broadcast a pending transaction. -- `getblockchaininfo`: To retrieve the current block height and `mediantime` (used to evaluate locktime). `Note:` `bal-pusher-enhanced` does not use this method to avoid RPC round-trips for median time. +- `getblockchaininfo`: To retrieve the current block height and `mediantime` (used to evaluate locktime). - `getblock`: To retrieve block data in `bal-pusher` (for median time calculation). Authentication is done via `bitcoincore-rpc` using either `UserPass` or `CookieFile` (the `cookie` file is stored in `~/.bitcoin/.cookie`). See `src/bin/bal-pusher.rs`. diff --git a/docs/03_architecture_and_data_flow.md b/docs/03_architecture_and_data_flow.md index 2f4b2e8..2bde179 100644 --- a/docs/03_architecture_and_data_flow.md +++ b/docs/03_architecture_and_data_flow.md @@ -27,9 +27,6 @@ User | bal-pusher | (async, ZMQ + RPC + reqwest) | (src/bin/bal-pusher.rs) +-----------------+ -+-----------------+ -| bal-pusher-enhanced | (sync, ZMQ + raw header parsing) -| (src/bin/bal-pusher-enhanced.rs) +-----------------+ | bitcoincore-rpc | sendrawtransaction diff --git a/docs/04_modules_detail.md b/docs/04_modules_detail.md index 6b39146..c46fe5c 100644 --- a/docs/04_modules_detail.md +++ b/docs/04_modules_detail.md @@ -126,22 +126,4 @@ This is the async daemon that monitors the blockchain and pushes pending transac - `welist_url`: The URL to POST to. - `ssl_key_path`: The path to the Ed25519 private key (`privkey.pem`) for signing stats. ---- - -## `bal-pusher-enhanced.rs` (Synchronous Transaction Pusher) - -**Location:** `src/bin/bal-pusher-enhanced.rs` - -This is a synchronous variant of the pusher that does not rely on the RPC for getting the `mediantime`. - -### Architecture (Synchronous) -- **ZMQ:** It uses `zmq::Context` with a `SUB` socket but does not use `zmq` in an async context. It calls `recv_multipart(0)` in a blocking loop (`std::thread::sleep`). -- **Topic:** `rawblock` (not `hashblock`). -- **Block Header:** It extracts the first 80 bytes (the header) from the raw block. The `timestamp` field is at byte offset 4 + 32 + 32 = 68, and is 4 bytes long (little-endian). It uses `byteorder` to read this. This avoids the `getblockchaininfo` RPC call. -- **Block Median Time:** It computes the rolling median time from the timestamps of the last 1000 blocks. -- **Preload:** It fetches and sorts the pending transactions from the database at startup, keeping them in memory. This reduces the database round trip. -- **RPC:** `sendrawtransaction` is used for the pending transactions, but not for `getblockchaininfo`. - -### Design Notes -- The ZMQ socket is blocking and has no timeout. If the Bitcoin node stops sending, the thread will hang indefinitely. The sleep between attempts is `std::thread::sleep(Duration::from_secs(1))`, but this happens *after* a successful `recv_multipart`, not if `recv` blocks. This is a potential DoS vector if the ZMQ connection goes silent. -- The `main_result` function is not async and does not use a `tokio` runtime. +--- \ No newline at end of file diff --git a/docs/05_api_reference.md b/docs/05_api_reference.md index 6b53836..a4b11b3 100644 --- a/docs/05_api_reference.md +++ b/docs/05_api_reference.md @@ -107,7 +107,7 @@ --- -## ZMQ Messages (consumed by `bal-pusher` and `bal-pusher-enhanced`) +## ZMQ Messages (consumed by `bal-pusher`) ### Topic: `hashblock` (Consumed by `bal-pusher`) - **Format:** A multipart ZMQ message. The first frame is the topic name (`hashblock`), the second frame is the 32-byte block hash. @@ -115,16 +115,9 @@ - **Action:** The pusher fetches `getblockchaininfo` from the RPC, gets the updated `mediantime`, then queries and pushes pending transactions. - **Endpoint:** `tcp://127.0.0.1:28332` (or network-specific ports). -### Topic: `rawblock` (Consumed by `bal-pusher-enhanced`) -- **Format:** A multipart ZMQ message. The first frame is the topic name (`rawblock`), the second frame is the raw serialized block data. The first `80` bytes of this second frame are the block header, in which bytes `[68..72]` are the timestamp (little-endian `uint32_t`). -- **Trigger:** When a new Bitcoin block is found by the local node. -- **Action:** The pusher extracts the block header, reads the timestamp from it, computes the rolling median of the last 11 block timestamps, then evaluates and pushes pending transactions. -- **Endpoint:** `tcp://127.0.0.1:28332` (or network-specific ports). -- **Note:** This topic is much more bandwidth-intensive than `hashblock` because the entire block is sent over the wire. - --- -## Bitcoin Core RPC Usage (used by `bal-pusher` and `bal-pusher-enhanced`) +## Bitcoin Core RPC Usage (used by `bal-pusher`) ### `sendrawtransaction` (Both pushers) - **Method:** `sendrawtransaction` (RPC `2`) @@ -136,7 +129,7 @@ - **Method:** `getblockchaininfo` (RPC `1`) - **Parameters:** None. - **Description:** Returns the current blockchain state, including the `mediantime` (the median timestamp of the last 11 blocks). This is used to evaluate the `nLockTime` of pending transactions. -- **Alternative:** `bal-pusher-enhanced` does not use this method; it derives the block timestamp directly from the `rawblock` ZMQ message to avoid an RPC round-trip and a potential RPC dependency failure. + ### `getblock` (Only used by `bal-pusher` for median time) - **Method:** `getblock` (RPC `1`) diff --git a/docs/08_security_audit.md b/docs/08_security_audit.md index d5a5a74..77444d2 100644 --- a/docs/08_security_audit.md +++ b/docs/08_security_audit.md @@ -55,11 +55,11 @@ **Status:** Open. **Priority:** High. ### 4. Denial of Service (DoS) (HIGH) -**Location:** `src/bin/bal-server.rs` (HTTP request body), `src/bin/bal-pusher.rs` (ZMQ), `src/bin/bal-pusher-enhanced.rs` (ZMQ). +**Location:** `src/bin/bal-server.rs` (HTTP request body), `src/bin/bal-pusher.rs` (ZMQ). **Description:** - The `bal-server` does not limit the size of the HTTP request body. On the `POST /pushtxs` endpoint, it calls `req.collect().await?.to_bytes()` without checking for a maximum body size. A malicious client could send an unbounded or extremely large request (e.g., `100000MB`), which would consume all available memory and crash the server. - The `bal-server` regex for path matching might be expensive if the user provides a malicious path string. For a production system, the regex should be compiled only once at startup and should be very specific. -- The `bal-pusher` `recv` call is synchronous and blocking. If the ZMQ connection fails, the thread will hang without any timeout. This is a resource leak if the connection is broken. The `bal-pusher-enhanced` has `recv_multipart(0)` which is also blocking forever. If the Bitcoin Core node is not sending, the pusher will be stuck waiting forever, consuming a thread and not doing other useful work. The ZMQ socket is not reconfigured with `ZMQ_RECONNECT_IVL` or `ZMQ_MAXMSGSIZE`. +- The `bal-pusher` `recv` call is synchronous and blocking. If the ZMQ connection fails, the thread will hang without any timeout. This is a resource leak if the connection is broken. The ZMQ socket is not reconfigured with `ZMQ_RECONNECT_IVL` or `ZMQ_MAXMSGSIZE`. If the Bitcoin Core node is not sending, the pusher will be stuck waiting forever, consuming a thread and not doing other useful work. - The `bal-pusher` does not have a rate limiter for the `sendrawtransaction` call. If the database is full or the ZMQ loop is running very fast, it could send thousands of RPC requests to the `bicoind` node, overwhelming it. For example, if the node is slow, the pusher will keep sending requests, potentially blocking the RPC queue or causing a memory leak in `bitcoind`. **Impact:** The server could become unresponsive, crash, or be completely unavailable. The `bitcoind` node could be overwhelmed with `sendrawtransaction` requests, causing a chain failure in the entire Bitcoin infrastructure. **Reproduction:** @@ -98,7 +98,7 @@ **Status:** Open. **Priority:** Medium. ### 7. ZMQ Authentication and Encryption (MEDIUM) -**Location:** `src/bin/bal-pusher.rs`, `src/bin/bal-pusher-enhanced.rs`. +**Location:** `src/bin/bal-pusher.rs`. **Description:** The ZMQ connection to the `bitcoind` is a plaintext TCP connection (`zmqpubhashblock=tcp://127.0.0.1:28332`). There is no ZMQ authentication (ZAP), no username/password, and no encryption (ZMQ_CURVE or ZMQ_GSSAPI). If the ZMQ port is accessible from the network (not just `127.0.0.1`), any attacker can subscribe to the `hashblock` or `rawblock` topics. The `rawblock` topic is particularly sensitive because it sends full block data, which is large and could be used to fingerprint the `bal` system. More importantly, the pusher does not verify that the `hashblock` is from the intended `bitcoind` node. If an attacker can inject a fake ZMQ message, they could trigger the pusher to evaluate the transactions and potentially broadcast them at an incorrect time, or cause a DoS. **Impact:** - If the ZMQ port is exposed, an attacker can intercept the `rawblock` data to get the full block contents, which could be used to fingerprint the node or the system. diff --git a/docs/09_references_and_links.md b/docs/09_references_and_links.md index 4120688..ad05da0 100644 --- a/docs/09_references_and_links.md +++ b/docs/09_references_and_links.md @@ -15,7 +15,7 @@ | XPub/Address Derivation | `src/xpub.rs` | `parse_xpub`, `derive_address`, `get_descriptor`, BIP-84 | | HTTP Server | `src/bin/bal-server.rs` | Hyper + Tokio, routes, handlers, `pushtxs` logic | | Async Pusher | `src/bin/bal-pusher.rs` | ZMQ `hashblock`, RPC + Reqwest, `send_stats` | -| Synchronous Pusher Enhanced | `src/bin/bal-pusher-enhanced.rs` | ZMQ `rawblock`, raw block header parsing, no `getblockchaininfo` | + | Stats (broken) | `src/bin/bal-stats.rs.dontcompile` | Not compiled, incomplete HTML report generator | | Release script | `make_release.sh` | Hardcoded token, `cargo install` | | DB download script | `download_bal_db.sh` | `scp` from remote | @@ -25,8 +25,7 @@ | Utility scripts | `lib.sh` | Colored echo functions | | Contrib (install) | `contrib/download_and_install_bal.sh` | Nginx, Certbot, systemd setup, hardcoded xpub | | Contrib (install bitcoind) | `contrib/download_and_install_bitcoincore.sh` | Bitcoind download, GPG verify, systemd, config | -| Contrib (install Tor) | `contrib/install_tor.sh` | Tor repository, `ControlPort 9051` | -| Systemd service | `bal-server.service` | Runs as `bal` user, `ProtectSystem`, `MemoryDenyWriteExecute` | +| Contrib (install Tor) | `contrib/install_tor.sh` | Tor repository, `ControlPort 9051` || Systemd service | `bal-server.service` | Runs as `bal` user, `ProtectSystem`, `MemoryDenyWriteExecute` | | Systemd service | `bitcoind.service` | `zmqpubhashblock` setup | | Systemd service | `tbitcoind.service` | Testnet `bitcoind` |