bal-server/docs/04_modules_detail.md

Module Details

Quick Reference

• What this file contains: detailed analysis of each Rust module and binary, including source code references.
• See also: 03_architecture_and_data_flow.md, 09_references_and_links.md

---

lib.rs

Location: src/lib.rs

This is the root of the library crate. It exports three public modules:

• pub mod db; — the database interface
• pub mod validation; — SSRF URL validation
• pub mod xpub; — the extended public key utilities

It contains no application logic.

---

db.rs (Database Interface)

Location: src/db.rs

This module contains all the logic for interacting with the SQLite database.

Key Functions

Function	Signature	Purpose
open_db	pub fn open_db(path: &str) -> Result<Connection, String>	Validates path (blocks .. traversal, forbidden system dirs, symlinks), opens SQLite, sets busy_timeout=5000, retries WAL mode up to 5 times, sets synchronous=NORMAL.
create_database	pub fn create_database(db: &Connection)	Creates all tables and indexes (idempotent via IF NOT EXISTS).
check_duplicate_txids	pub fn check_duplicate_txids(db: &Connection, txids: &[String]) -> Result<HashSet<String>, Error>	Batch check which txids already exist. Chunks in groups of 500 for SQLite parameter limit safety.
insert_xpub	pub fn insert_xpub(db: &Connection, network: &str, xpub: &str)	INSERT OR IGNORE into tbl_xpub.
get_last_used_address_by_ip	pub fn get_last_used_address_by_ip(db: &Connection, network: &String, xpub: &String, address: &String) -> Option<String>	Finds most recent address previously assigned to a remote IP for an xpub.
get_next_address_index	pub fn get_next_address_index(db: &Connection, network: &String, xpub: &String) -> (i64, i64)	Atomically increments path_idx and returns (xpub_id, new_index) using RETURNING.
save_new_address	pub fn save_new_address(db: &Connection, xpub: i64, address: &String, path: &String, remote_addr: &String)	INSERT into tbl_address.
execute_insert	pub fn execute_insert(db: &Connection, sqltxs: String, ptx: Vec<(usize, Value)>, sqlinp: String, pinp: Vec<(usize, Value)>, sqlout: String, pout: Vec<(usize, Value)>) -> Result<(), Error>	Executes a transaction: BEGIN, insert txs, insert inputs, insert outputs, COMMIT (with ROLLBACK on error).
get_total_transaction_number	pub fn get_total_transaction_number(db: Connection, network: &String) -> Result<i64, Error>	Counts transactions for a network.
get_all_addresses_by_xpub	pub fn get_all_addresses_by_xpub(db: &Connection, xpub: &str) -> Result<HashSet<String>, Error>	Fetches all addresses for an xpub via JOIN on tbl_xpub/tbl_address. Used for O(1) fee validation in the push handler.

Design Notes

• All SQL queries use parameterized statements (? placeholders with bind()). No string formatting is used for user-controlled values.
• The open_db function validates paths before opening, rejecting directory traversal, system directories, and symlinks.
• WAL mode (PRAGMA journal_mode=WAL) is enabled with retry logic for concurrent access safety.

---

xpub.rs (Extended Public Key Utilities)

Location: src/xpub.rs

This module handles the derivation of Bitcoin addresses from extended public keys (xpub/zpub/ypub) and the creation of P2WPKH descriptors with Bitcoin Core checksums.

Key Functions

Function	Signature	Purpose
new_address_from_xpub	pub fn new_address_from_xpub(zpub: &str, index: i64, network: Network) -> Result<(String, String), Box<dyn std::error::Error>>	Derives a P2WPKH (native SegWit) address at path m/0/{index} from an xpub. Returns (address, path).
get_bitcoincore_descriptor	pub fn get_bitcoincore_descriptor(xpub: &str) -> String	Generates a Bitcoin Core descriptor with checksum (e.g., wpkh([fingerprint/84h/0h/0h]xpub/0/*)#checksum).
calculate_fingerprint	pub fn calculate_fingerprint(tpub: &str) -> Result<String, String>	Returns the hex fingerprint of an xpub (converts to standard xpub first).

Private Functions

• poly_mod(c, val) / calc_checksum(desc) — Bitcoin Core descriptor checksum calculation.
• convert_xpub(xpub) — Detects prefix (xpub/ypub/zpub or tpub/vpub/upub) and converts to target format.
• base58check_decode(s) / base58check_encode(data) — Base58Check encoding/decoding.
• convert_to(zpub, prefix) — Converts xpub between different prefix formats.

Supported Prefixes

Prefix	Type	Network
xpub	Legacy P2PKH	Mainnet
ypub	Nested SegWit P2SH-P2WPKH	Mainnet
zpub	Native SegWit P2WPKH	Mainnet
tpub	Legacy P2PKH	Testnet
vpub	Nested SegWit	Testnet
upub	Nested SegWit	Regtest

Dependencies

• bitcoin::bip32::{DerivationPath, Xpub}
• bitcoin::key::Secp256k1
• bitcoin::{Address, Network, ScriptBuf, WPubkeyHash}
• sha2::{Digest, Sha256}

---

validation.rs (SSRF Protection)

Location: src/validation.rs

This module provides URL validation to prevent SSRF attacks via the welist stats reporting feature.

Key Functions

Function	Signature	Purpose
is_valid_welist_url	pub fn is_valid_welist_url(url_str: &str) -> bool	Validates a URL against SSRF: checks scheme is HTTPS, blocks localhost/loopback/private/link-local/multicast/unspecified IPs for both IPv4 and IPv6.

Validation Rules

1. URL must be well-formed and parsable.
2. Scheme must be https:// (plain HTTP is rejected).
3. Host must not be localhost, 127.0.0.1, ::1, or any loopback/private/link-local/multicast/unspecified IP address.
4. IPv4 private RFC1918 ranges (10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16) and AWS metadata link-local (169.254.169.254) are blocked.
5. IPv6 Unique Local (fc00::/7) and link-local (fe80::/10) are blocked.

Inline Tests

8 unit tests cover valid domains, invalid schemes, localhost/loopback, private IPs, unspecified/multicast, IPv6 link-local, IPv6 unique local, malformed URLs, and valid public IPs.

---

bal-server.rs (HTTP Server / API)

Location: src/bin/bal-server.rs

The main application binary that provides an async HTTP server.

Architecture

• Runtime: actix-web 4.9.0 with actix-rt (#[actix_web::main]).
• Rate Limiting: actix-governor middleware with token-bucket algorithm. Uses RealIpKeyExtractor to identify clients by real IP behind reverse proxy (via X-Real-IP / X-Forwarded-For headers).
• Response Compression: actix_web::middleware::Compress.
• Request Logging: actix_web::middleware::Logger::default().
• Shared State: Arc<Mutex<Connection>> for database access, MyConfig for configuration.

Configuration Structs

MyConfig (server configuration):

• regtest, signet, testnet, testnet4, mainnet: NetConfig per network
• info, bind_address, bind_port, db_file, pub_key_path, expose_stats

NetConfig (per-network):

• address (xpub or address), fixed_fee (sats), xpub (bool), network (bitcoin::Network), name, enabled

ActixConfig (server tuning):

• max_body_size, timeout_secs, rate limits (pushtxs per sec/burst), workers, max_connections, trusted_proxy

Key Routes

Method	Path	Handler	Description
GET	/	echo_home	Returns cfg.info string
GET	/.pub_key.pem	echo_pub_key	Returns public key PEM file
GET	/version	echo_version	Returns VERSION constant (0.3.2)
GET	/{network}/info	echo_info	Returns InfoResponse JSON. In xpub mode, derives/returns per-IP address
GET	/{network}/stats	echo_stats	Returns Vec<StatsResponse> JSON (requires expose_stats=true)
POST	/{network}/pushtxs	echo_push	Accepts newline-separated raw tx hex. 3-phase: parse (no lock), check duplicates (lock), insert (lock)
POST	/searchtx	echo_search	Searches by txid (body = 64 hex chars). Returns status, tx, our_address, our_fees, reqid

Handler Details

echo_info: If xpub mode is enabled, first checks get_last_used_address_by_ip for an existing address for that IP. If none, atomically claims next index via get_next_address_index, derives address via new_address_from_xpub, and saves it. Two separate DB lock acquisitions (lookup + save) with CPU-bound derivation in between (no lock held).

echo_push: Three-phase approach:

1. Load all known addresses (for xpub validation) with DB lock, release lock
2. Parse all transactions from request body (CPU-bound, no lock) using parse_request_transactions
3. Batch check duplicates with DB lock, release lock
4. Build bulk INSERT statements using UNION ALL SELECT and execute in single transaction

parse_request_transactions: Splits body by newlines, hex-decodes each line, deserializes via consensus::deserialize, computes txid/wtxid/ntxid, checks if any output matches the expected address (or is in known_addresses for xpub mode) with amount >= fixed_fee.

Error Handling

All unwrap()/expect() calls have been replaced with safe match/if let error propagation, returning appropriate HTTP status codes (400, 404, 500). The server does not panic on untrusted input.

Static Public Key (/.pub_key.pem)

The server serves a static public_key.pem file. The corresponding privkey.pem is used by the pusher to sign statistics before sending them to the welist server.

---

bal-pusher.rs (Async Transaction Pusher)

Location: src/bin/bal-pusher.rs

This is the async daemon that monitors the blockchain and pushes pending transactions.

Architecture

• Runtime: tokio::main with rt-multi-thread.
• ZMQ: zmq::Context with a SUB socket. Subscribes to all topics. Uses set_rcvtimeo(5000) for 5-second receive timeout.
• RPC: bitcoincore-rpc client. Tries username/password auth first, falls back to cookie file auth.
• HTTP Client: reqwest with json and socks features. Sends Ed25519-signed JSON POST to the welist server.
• IPv6 Preference: Optional BAL_PUSHER_PREFER_IPV6 flag pins the HTTP connection to the first IPv6 address.

Key Logic

1. On startup and every hashblock message, it calls main_result().
2. main_result creates a bitcoincore-rpc client. If it fails, it logs an error and returns (no panic).
3. It fetches getblockchaininfo to get mediantime and blocks height.
4. It queries the database for transactions with status=0 and locktime satisfied (block height < best block, or timestamp < mediantime for timestamps > LOCKTIME_THRESHOLD).
5. For each pending transaction, it calls sendrawtransaction.
6. If send_stats is enabled, it collects statistics, signs them with privkey.pem, and sends them to the configured welist URL via reqwest.
7. It updates the database with the new status (1 = sent, 2 = failed with push_err).

Statistics Reporting

• Statistics are aggregated from the database (total, waiting, sent, failed, profits, unique inputs).
• The chain name is validated (alphanumeric, -, _ only).
• Stats are inserted into tbl_stats with ON CONFLICT(chain) DO UPDATE.
• The stats payload is signed with Ed25519 and POSTed to {welist_url}/ping.
• The WELIST_SERVER_URL is validated via is_valid_welist_url() before sending (can be bypassed with WELIST_SKIP_URL_VALIDATION=true).
• WELIST_PING_SIG_VERSION selects the signature scheme:
  • 1 (default): legacy concatenation of URL, chain, height, median time and block hash. The WeTime service verifies it against /.pub_key.pem. The signature authenticates liveness only: the pusher does not send info, base_fee or version, and the service refreshes last_update without persisting executor metadata. Use v2 when metadata must be published and authenticated.
  • 2: canonical JSON signed with the WELIST-PING-V2\n prefix. The pusher fetches info, base_fee and version from the local /{chain}/info endpoint and signs them together with the block metadata. The WeTime service discovers the public key from a DNSSEC-validated _welist-key.<host> TXT record or from a v3 onion address, so no HTTP callback is needed. The url is trimmed of trailing slashes to match the service-side normalization; absent metadata is signed as null. When onion delegation is configured, the pusher verifies at startup that the certificate is bound to the configured onion host, has not expired, and delegates to the key at SSL_KEY_PATH.
• bal-pusher.sh pins WELIST_PING_SIG_VERSION=1 for the development setup: a localhost URL cannot use DNSSEC, so the dev pusher uses the legacy scheme verified against /.pub_key.pem. v2 requires a public DNS hostname or a v3 onion address.
• To publish the v2 public key, derive the base64 SubjectPublicKeyInfo DER: openssl pkey -in private_key.pem -pubout -outform DER | base64 -w0 and publish it in a DNSSEC-signed TXT record named _welist-key.<host> with value v=we1; alg=ed25519; key=<base64>. A v3 onion URL needs no DNS record: its Ed25519 identity key is embedded in the address, and the recommended setup keeps that key offline while it certifies a dedicated ping key (WELIST_PING_DELEGATION_FILE). See 10_dnssec_key_publishing.md for the full operator guide.

ZMQ Timeout Handling

• Uses set_rcvtimeo(5000) (5-second timeout).
• Logs a warning every ~720 consecutive timeouts (~1 hour of no blocks).
• Does not block forever or panic on connection loss.

Configuration

All configuration is via environment variables (no config files):

• BAL_SERVER_DB_FILE: Path to SQLite database.
• BAL_PUSHER_BITCOIN_DIR: Bitcoin data directory (for cookie file path).
• BAL_PUSHER_SEND_STATS: Enable/disable remote stats reporting.
• BAL_SERVER_URL: URL of the bal-server for internal communication.
• SSL_KEY_PATH: Path to Ed25519 private key for signing stats.
• WELIST_SERVER_URL: URL to POST stats to (validated against SSRF).
• WELIST_SKIP_URL_VALIDATION: Bypass URL validation (for testing).
• WELIST_PING_SIG_VERSION: WeTime ping signature scheme (1 legacy, 2 canonical JSON).
• BAL_PUSHER_PREFER_IPV6: Pin HTTP connection to IPv6 address.
• Per-network: BAL_PUSHER_{NETWORK}_HOST, _PORT, _DIR_PATH, _DB_FIELD, _COOKIE_FILE, _RPC_USER, _RPC_PASSWORD, _ZMQ_HASHBLOCK.