47 Commits

Author SHA1 Message Date
b8d87ecabd fix: per-endpoint rate limits, request timeout and input hardening
bal-server:
- Wire independent actix-governor buckets per endpoint class (pushtxs,
  searchtx, info/stats, default) so polling a cheap endpoint can no longer
  exhaust the push-txs budget; previously every limit was dead config
- Enforce BAL_SERVER_ACTIX_TIMEOUT_SECS via a request-timeout middleware that
  returns 408 Request Timeout, covering slow request bodies
- De-duplicate txids within a single /pushtxs request body
- Replace the byte-slice xpub detection (panic on non-ASCII) with a safe
  prefix check

bal-pusher:
- Handle malformed/empty ZMQ frames without indexing panics or UTF-8 expect
- Return an error from sign_message and skip the stats report instead of
  panicking when the private key cannot be parsed

validation / db:
- Reject IPv4-mapped IPv6 targets (::ffff:a.b.c.d) in is_valid_welist_url
  to close an SSRF bypass, with unit tests
- Use Path::starts_with for the SQLite forbidden-directory check so paths
  like /etcfoo are not misclassified

misc:
- Add explicit secret patterns to .gitignore and sync the docs
2026-10-14 18:30:23 -04:00
457c143954 chore: unify DB env vars and harden local Postgres startup
- Use BAL_SERVER_DB_* / BAL_SERVER_PG_DSN in bal-pusher, Dockerfiles,
  scripts and docs instead of the BAL_PUSHER_DB_* duplicates
- Default BAL_SERVER_URL to http://127.0.0.1:9133
- Extract PostgreSQL container startup into a shared pg-server.sh sourced by
  bal-server.sh and bal-pusher.sh
- Read the Postgres password from BAL_PG_PASSWORD instead of hardcoding it,
  and stop printing the password-bearing DSN
- Redact a real-looking Gitea token from .env.example (revoke it if it was live)
2026-10-14 18:29:39 -04:00
ccadb14254 refactor: unify client IP extraction into extract_real_ip
Replace extract_client_ip (manual header parsing without proxy
verification) and is_valid_ip with a single extract_real_ip function
that uses the same proxy-checking logic as RealIpKeyExtractor.

- If peer IP == trusted proxy: use realip_remote_addr() (Forwarded/
  X-Forwarded-For) — safe because peer was verified
- Otherwise: use peer_addr() directly (local/direct connection)

echo_info now extracts the trusted proxy from web::Data<IpAddr>
and passes it to extract_real_ip, ensuring consistent IP resolution
across rate limiting and address derivation.
2026-08-19 03:13:01 -04:00
ed85d5a8ea docs: update rate limiting docs for RealIpKeyExtractor and trusted proxy
- Correct per-endpoint rate limit tables to reflect actual single global config
- Document RealIpKeyExtractor behavior (X-Real-IP / X-Forwarded-For behind proxy)
- Add BAL_SERVER_TRUSTED_PROXY env var to all relevant docs
- Update security audit and modules detail with proxy-aware rate limiting
2026-08-19 02:57:57 -04:00
f8b1ee167f fix: rate limit per real client IP behind reverse proxy
Replace PeerIpKeyExtractor with custom RealIpKeyExtractor that extracts
the real client IP from X-Real-IP/X-Forwarded-For headers when the
request comes from a trusted proxy (default 127.0.0.1).

This fixes rate limiting being applied to the proxy IP instead of
individual clients when deployed behind Nginx.
2026-08-19 02:54:46 -04:00
66a1273e2c fix: correct locktime threshold, PG stats SUM on TEXT, PG locktime width, and translate Italian error strings
- Bug 1: LOCKTIME_THRESHOLD was 5_000_000 instead of BIP-65 value 500_000_000
- Bug 2: PostgreSQL SUM(our_fees) failed on TEXT column; cast to BIGINT
- Bug 3: PostgreSQL locktime INTEGER (i32) widened to BIGINT (i64) with idempotent ALTER
- Translate 4 Italian error strings in xpub.rs to English
2026-08-19 02:22:14 -04:00
e7262a2cae chore: remove public_key.pem and .gitsecret/ from repo 2026-08-18 08:39:55 -04:00
7be0ef3578 chore: clean up .gitignore and remove tracked files that should not be in the repo
Removed from tracking:
- Release binaries and tarballs (releases/, bal-server-0.2.2-*)
- Environment files with secrets (bal-server.env)
- Service files (bitcoind.service, tbitcoind.service)
- Test data (invalid_txs, valid_txs, test)
- Utility scripts (sendtx.sh, lib.sh, generate_random_ascii.sh, etc.)
- Archive (contrib.tar.gz)
- Scratch files (update, update_codebase.txt, src/xpub.rs2)

NOTE: The .env file with Gitea token was never tracked (already in .gitignore).
Private keys (chiave_privata.key, ec.key, private_key.pem, privkey.pem)
were also never tracked. Rotate the Gitea token if it was ever shared.
2026-08-18 03:24:18 -04:00
b277d894dd fix: resolve PostgreSQL type mismatches and add willexecutor debug logging
- Fix get_next_address_index: use try_get::<i32> for PG SERIAL/INTEGER columns
- Fix search_tx: use try_get::<i32> for PG status column
- Fix execute_insert: parse locktime (String→i64) and in_vout (String→i32)
  before binding to PG INTEGER columns
- Fix execute_insert: bind tbl_out vout as i32, amount as String for PG
- Fix save_new_address: cast xpub i64 to i32 for PG INTEGER column
- Fix get_pending_txs: cast i64 bind params to i32, use try_get::<i32> for reads
- Fix get_stats: use try_get::<i32> for all numeric PG INTEGER columns
- Add trace logging in parse_request_transactions for xpub address matching
- Add trace logging in get_all_addresses_by_xpub for query debugging
2026-08-18 03:14:03 -04:00
90ae3b6510 docker: add release Dockerfile, fix build deps and feature-split builds
- Add Dockerfile.release: downloads latest release from Gitea, verifies SHA-256
- Fix Dockerfile: remove unused libsodium/cmake, add libsqlite3-dev
- Split build per binary with --no-default-features --features server/pusher
- Remove Cargo.lock from .dockerignore for reproducible builds
- Update README and knowledge base with Dockerfile.release docs
2026-07-20 17:36:58 -04:00
a20cc1cf7c docs: update knowledge base to match current codebase
- Fix framework references (actix-web, not hyper)
- Update all env var names (BAL_SERVER_*/BAL_PUSHER_* prefix)
- Add validation.rs module documentation
- Fix function signatures in xpub.rs and db.rs
- Update API response formats (InfoResponse, StatsResponse)
- Fix database schema (date_creation/date_update, push_err, tbl_stats)
- Mark fixed vulnerabilities with current status
- Add Docker support and actix tuning documentation
- Remove outdated references (confy, bal-stats.rs.dontcompile)
- Add regression test summary table
2026-07-20 15:43:20 -04:00
80537b548f release: bal-server-0.3.2 2026-07-20 13:37:19 -04:00
01ac26e7be fix: extract client IP correctly behind reverse proxy
Add X-Forwarded-For as fallback header when X-Real-IP is missing.
Validate IP format before storing. Log which source was used.

release: bal-server-0.3.2
2026-07-20 13:34:03 -04:00
4b9ae596ce release: bal-server-0.3.1 2026-07-20 08:53:16 -04:00
c021e3d4cc fix: bug fixes, dead code removal, improved ZMQ logging
- Fix SQL syntax error in create_database (trailing parenthesis)
- Fix typo i27.0.0.1 -> 127.0.0.1 in Testnet/Testnet4 defaults
- Replace hardcoded VERSION with CARGO_PKG_VERSION in bal-pusher
- Remove unwrap() in DB update loops (status push/invalid)
- Remove duplicate init_network call in bal-server startup
- Use INSERT OR IGNORE for idempotent xpub initialization
- Remove debug println! left in production code
- Remove dead code: check_zmq_connection, ConnectionMonitor, seq_to_str
- Remove all commented-out code blocks
- ZMQ timeout logging: trace instead of warn, error only after 1 hour
2026-07-20 08:48:54 -04:00
0a3bd34ecb release: bal-server-0.3.0 2026-07-19 21:29:18 -04:00
59774c0cbb release: bal-server-0.3.1 2026-07-19 20:16:44 -04:00
c47862a72f fix: default features for cargo build/test, build each binary separately in release
- Set default = ["server", "pusher"] so cargo build/test works without flags
- Add required-features to each [[bin]] section
- make_release.sh builds each binary with --no-default-features for optimal size
2026-07-19 19:53:41 -04:00
3f7608641d perf: reduce binary sizes from 15M/11M to 3.8M/5.5M
- Add [profile.release]: opt-level=z, lto=true, strip=true, codegen-units=1, panic=abort
- Replace vendored openssl (~5MB) with ed25519-dalek (pure Rust, ~100KB)
- Feature-gate deps: server (actix, chrono) vs pusher (zmq, reqwest, ed25519-dalek)
- Remove unused confy dependency
- bal-pusher: 15M -> 3.8M (-75%)
- bal-server: 11M -> 5.5M (-50%)
2026-07-19 19:27:42 -04:00
6cf85d4f63 Merge origin/main: resolve conflicts and add 10s timeout to welist_http_client 2026-07-19 18:13:21 -04:00
bd06140c1f fix(pusher): allow bypassing SSRF validation for local dev and fix RUST_LOG export
- Add WELIST_SKIP_URL_VALIDATION env var to bypass URL validation in dev
- Fix bal-pusher.sh: export RUST_LOG so env_logger picks up log level
- Add WELIST_SKIP_URL_VALIDATION=true to bal-pusher.sh for regtest use
2026-07-19 17:48:43 -04:00
cc39027590 fix(pusher): add ZMQ diagnostic logging and heartbeat monitoring
- Log ZMQ subscribe confirmation with address
- Add consecutive timeout counter with periodic warn every 60s
- Log ZMQ connection restored after timeouts
- Log main_result errors instead of discarding with let _=
2026-07-19 15:55:49 -04:00
5822d3e231 Merge pull request 'pusher: optional IPv6 preference for welist reports + log report failures' (#1) from SAFE21.io/bal-server:fix/pusher-welist-ipv6-preference into main
Reviewed-on: #1
2026-07-19 15:09:00 +00:00
4ed58b11c5 fix(pusher): log send_stats_report errors instead of discarding
main_result called send_stats_report/calculate_stats with 'let _ = ...',
silently dropping any failure. A broken welist route (e.g. unreachable
IPv4 path) is then invisible in the logs and can go unnoticed for a long
time. Log failures with warn! so connectivity problems are diagnosable.
2026-07-19 14:09:41 +02:00
5fbfa79c99 feat(pusher): optional IPv6 preference for welist reports (BAL_PUSHER_PREFER_IPV6)
The welist host publishes both A and AAAA records. On networks where the
IPv4 route is broken (connection stalls after the TCP handshake) while
IPv6 works, the default connector may pick the broken family and the
report request hangs.

When BAL_PUSHER_PREFER_IPV6 is truthy, the pusher now resolves the welist
host itself and pins the reqwest client to its first IPv6 address; the
original hostname is still used for the Host header and TLS SNI. When the
variable is unset (default) or no AAAA record exists, behavior is
completely unchanged.

Includes unit tests for the URL host/port parsing and documentation in
docs/07_deployment_and_ops.md.
2026-07-19 14:09:01 +02:00
94de433926 fix(pusher): improve welist response logging, error handling, and add request timeout
- Always log HTTP status code and response body from welist (info level)
- Log send_stats_report errors at call site instead of silently discarding
- Add 10s timeout to reqwest client to prevent indefinite hangs
- Apply clippy fixes (is_empty, if-let chains, dead_code, etc.)
2026-07-18 22:58:42 -04:00
84d4522ce3 fix(pushtxs): skip invalid txs instead of aborting batch, anonymize error responses
- parse_request_transactions now skips txs without willexecutor output
  instead of returning an error that aborts the entire batch
- returns 'error' (400) only when NO txs are valid
- all HTTP error bodies replaced with generic 'error' to avoid leaking
  internal details
2026-07-18 21:35:39 -04:00
5a2fc9c497 refactor: rename bal-server-actix.rs to bal-server.rs
- Rename src/bin/bal-server-actix.rs -> src/bin/bal-server.rs
- Update Cargo.toml to point to new filename
- Update Dockerfile reference
- Update docs/08_security_audit.md references
2026-07-17 10:31:56 -04:00
f28fad0194 chore: remove make_release.sh from tracking, keep local 2026-07-17 09:59:05 -04:00
53be95113e feat(release): add ASCII-armored .asc signature alongside .sig 2026-07-17 09:57:39 -04:00
1e813e1a75 fix(release): move asset paths after ASSET_PATH definition 2026-07-17 09:55:11 -04:00
2d7b767138 release: bal-server-0.3.0 2026-07-17 09:53:35 -04:00
b39397e742 feat(release): add signature, checksum and verification instructions 2026-07-17 09:51:37 -04:00
3cfb635da1 docs: fix Docker example to use regtest env vars consistently 2026-07-17 05:23:51 -04:00
cdc2d2d4c7 docs: rewrite README with Docker, per-network config, Actix DoS vars 2026-07-17 02:51:04 -04:00
40fa23b03b fix: Docker support, WAL race condition, pusher panic fixes
- Add multi-stage Dockerfile with tini, non-root user, healthcheck
- Fix SQLite WAL mode race between bal-server and bal-pusher (busy_timeout + retry)
- Fix tbl_stats missing UNIQUE index for ON CONFLICT clause
- Replace unwrap() panics in bal-pusher with graceful error handling
- Add docker/entrypoint.sh with BAL_PUSHER_NETWORK support
- cargo fmt across all files
2026-07-16 21:24:09 -04:00
3c4608bdb4 security: remove terms and personal info (Svātantrya) from files 2026-07-16 20:07:14 -04:00
94f64c6334 docs(deploy): align docs with contrib script fixes
- 07_deployment_and_ops.md: update zmqpubhashblock note, clarify certbot --nginx,
  update Nginx checklist items referencing the install script
- 09_references_and_links.md: remove 'hardcoded xpub' from install script row
2026-07-16 19:49:20 -04:00
2516fe5f61 fix(install-scripts): correct critical bugs in contrib scripts
- download_and_install_bitcoincore.sh:
  - Fix apt install with single-quoted string (was treating all deps as one package)
  - Add sudo to apt install commands
  - Fix tar extraction: now finds extracted dir and copies bitcoind/bitcoin-cli to /usr/local/bin
  - Fix zmqpubrawblock -> zmqpubhashblock (pusher expects hashblock topic)
  - Fix install check (0 was testing cd, not tar) with explicit extraction check
  - Handle existing user in adduser
  - Fix source lib.sh path

- download_and_install_bal.sh:
  - Use curl -sfL (fail on 404, follow redirects) for Gitea download
  - Add null/empty check after jq download URL extraction
  - Change certbot --standalone to --nginx (avoids port 80 conflict)
  - Enable nginx site symlink (was commented out)
  - Handle existing 'bal' user in adduser

- install_tor.sh:
  - Add sudo to all root-requiring commands (apt, tee, cp, sed, systemctl)
  - Backup torrc with timestamp before modifying
  - Add DisableDebuggerAttachment hardening
2026-07-16 19:39:10 -04:00
b0fd59e431 fix(install-scripts): harden contrib scripts and fix critical bugs
- download_and_install_bal.sh: add set -euo pipefail, fix sudo sudo typo, fix nginx path (etc/nginx/etc/nginx -> etc/nginx), add chmod 600 for env files, reduce RestartSec to 5s, fix concatenated echo line, add systemctl daemon-reload
- download_and_install_bitcoincore.sh: add set -euo pipefail, fix source lib.sh path to use script directory
- install_tor.sh: add set -euo pipefail, detect architecture dynamically (dpkg --print-architecture), detect distro dynamically (lsb_release -cs), bind ControlPort to 127.0.0.1:9051, add DisableDebuggerAttachment
2026-07-16 19:07:15 -04:00
24809b19e2 security: fix audit points 5-9 + optimize echo_push/info endpoints
- Point 5 (SSRF): Add URL validation for WELIST_SERVER_URL (src/validation.rs)
- Point 6 (DB Access): Add DB path validation, symlink check, WAL mode (open_db)
- Point 8 (HTTPS): Extract nginx config, add deployment checklist, bind warnings
- Point 9 (Input Validation): Add NETWORKS check (404 for unknown), txid 64-hex validation
- Optimize echo_push: parse transactions outside DB lock, batch duplicate check, N+1 xpub lookup eliminated via HashSet cache
- Optimize echo_info: derive BIP32 address outside DB lock, minimize lock duration
- Fix echo_stats SQL injection via parameter binding + add idx_stats_chain index
- New regression tests: ssrf_tests, db_path_validation, input_validation_tests
2026-07-16 18:59:30 -04:00
431929d37c security: fix secret leakage hardcoded tokens and credentials (Punto 3)
- .gitignore: Add protection for .env, *.pem, *.key, private_key.pem, privkey.pem, ec.key, chiave_privata.key, and shell scripts bal-*.sh
- make_release.sh: Remove hardcoded token 5cfa8c33e337ebaadb355c0ffa2d053d521ee43b
  Add loading from .env file with GITEA_API_TOKEN variable
  Add error handling if token is not set (prevents script from running without proper authentication)
- .env.example: Add template file for Gitea API token setup (not committed to git, .gitignored)
- generate_keys.sh: Add chmod 600 to protect private_key.pem permissions
- contrib/download_and_install_bal.sh: Remove hardcoded xpub and fixed_fee.
  Make all settings required as arguments or environment variables (xpub, fixed_fee, willexecutor_url, email, info)
  Add proper error handling and usage instructions if required arguments are not provided
- tests/secret_leakage_tests.rs: Add regression tests that:
  * Verify .gitignore protects .env, .pem, .key files
  * Verify no private key files are tracked in git (only public_key.pem is allowed)
  * Scan shell scripts for potential hardcoded tokens
- All tests pass: cargo test (8 tests: 3 SQL injection + 2 panic regression + 3 secret leakage)
- Build verified: cargo check (0 errors)
2026-07-16 15:25:23 -04:00
abb882bd69 security: fix unwrap/expect/panic on untrusted input (Fase 3 MEDIUM/LOW)
- xpub.rs: Replace convert_xpub() unwrap() with Result propagation
- xpub.rs: Replace calculate_fingerprint() unwrap() with Result propagation
- xpub.rs: Replace get_bitcoincore_descriptor() unwrap() with Result/match
- xpub.rs: Fix new_address_from_xpub() call in bal-server.rs to handle Result
- xpub.rs: Add prefix validation in convert_xpub (xpub/ypub/zpub/tpub/vpub/upub)
- bal-pusher.rs: Remove parse().unwrap() on env var bool, use unwrap_or(false)
- bal-pusher.rs: Replace port try_into().unwrap() with safe u16::try_from match
- bal-pusher.rs: Add error logging for invalid port values in env vars
- All tests pass: cargo test (5 tests: 3 SQL + 2 panic regression)
- Build verified: cargo check --bin=bal-server --bin=bal-pusher (0 errors)
2026-07-16 15:08:49 -04:00
604ce3ebed security: eliminate unwrap/expect/panic on untrusted input paths (Fase 2 HIGH)
- db.rs: Replace all stmt.read/bind().unwrap() with safe match/if let + error logging
- db.rs: Replace execute_insert() expect() with safe prepare + rollback on error
- db.rs: Replace get_total_transaction_number() unwrap() with safe match/Result propagation
- bal-server.rs: Replace fs::read_to_string().expect() with match + 500 error
- bal-server.rs: Replace idx/amount.try_into().unwrap() with i64::try_from(...).unwrap_or()
- bal-server.rs: echo_pub_key returns 500 instead of panic on file read failure
- docs/08_security_audit.md: Update status for panic/DoS vulnerabilities to 'Fixed'
- All tests pass: cargo test (5 tests: 3 SQL injection + 2 panic regression)
- Build verified: cargo check --bin=bal-server --bin=bal-pusher (0 errors)
2026-07-16 14:56:25 -04:00
06bec057c7 security: fix panic on untrusted input (Phase 2 Critical)
- Replace all sqlite::open().unwrap() per-request with shared Arc<Mutex<Connection>>
- Add Mutex poisoning recovery in all cfg.lock() and db.lock() calls
- Fix std::str::from_utf8().unwrap() with safe match → 400 Bad Request
- Fix timestamp_nanos_opt().unwrap() with safe match
- Fix panic on RPC client failure (bal-pusher): error log + sleep + retry
- Fix ZMQ socket connect with retry loop (bal-pusher)
- Add ZMQ_RCVTIMEO=5000 and match recv for graceful timeout (bal-pusher)
- Fix ZMQ subscribe error with match instead of unwrap (bal-pusher)
- Add unwrap_or for all tbl_stats row fields in echo_stats (prevent NULL panic)
- Add panic_regression_tests.rs: test mutex poisoning recovery and NULL unwrap_or

All tests pass: cargo test --test panic_regression_tests + sql_injection_tests
Build verified: cargo check --bin=bal-server --bin=bal-pusher
2026-07-16 14:49:31 -04:00
9e2acb78ac refactor: remove bal-pusher-enhanced and update docs/env
- Remove src/bin/bal-pusher-enhanced.rs (synchronous pusher variant)
- Remove bal-pusher.env and bal-pusher.sh from git tracking (now in .gitignore)
- Update all documentation to remove references to bal-pusher-enhanced:
  * 01_project_overview.md
  * 02_glossary_and_bitcoin_domain.md
  * 03_architecture_and_data_flow.md
  * 04_modules_detail.md
  * 05_api_reference.md (remove rawblock ZMQ section, update references)
  * 08_security_audit.md (remove references to bal-pusher-enhanced in DoS and ZMQ sections)
  * 09_references_and_links.md
- Build verified: cargo check passes for bal-pusher and bal-server binaries
2026-07-16 14:32:59 -04:00
9f308680bb docs: add comprehensive knowledge base and security audit
- Add docs/INDEX.md with navigable index and quick reference guides
- Add 9 knowledge base files covering project overview, Bitcoin domain,
  architecture, modules, API reference, database schema, deployment/security
- Update AGENTS.md with knowledge base reference and update policy
- Add tests/sql_injection_tests.rs with regression tests for SQL injection
- Fix SQL injection vulnerabilities in bal-pusher.rs:
  * Replace string-formatted UPDATE IN with loop + parameterized queries
  * Replace string-formatted UPDATE push_err with parameterized query
  * Add chain name validation in calculate_stats to prevent env var tampering
- Update .gitignore to exclude bal-pusher.env and bal-pusher.sh
2026-07-16 14:11:18 -04:00
18 changed files with 297 additions and 101 deletions

View File

@@ -3,8 +3,7 @@
# DO NOT commit the real token to git! # DO NOT commit the real token to git!
# This file is in .gitignore and should not be committed to git # This file is in .gitignore and should not be committed to git
GITEA_API_TOKEN=your_gitea_api_token_here GITEA_API_TOKEN=your_gitea_api_token_here
# Example: GITEA_API_TOKEN=5cfa8c33e337ebaadb355c0ffa2d053d521ee43b # If a token was ever committed, revoke it and set a new one via the environment.
# (replace with your actual token after revoking the old one)
# === bal-server === # === bal-server ===
# !!! WARNING: Never bind to 0.0.0.0 in production. Use 127.0.0.1 and place # !!! WARNING: Never bind to 0.0.0.0 in production. Use 127.0.0.1 and place
@@ -33,8 +32,14 @@ BAL_SERVER_ACTIX_DEFAULT_BURST=100
BAL_SERVER_ACTIX_WORKERS=4 BAL_SERVER_ACTIX_WORKERS=4
BAL_SERVER_ACTIX_MAX_CONNECTIONS=100 BAL_SERVER_ACTIX_MAX_CONNECTIONS=100
# === PostgreSQL (optional; used by pg-server.sh) ===
# pg-server.sh requires BAL_PG_PASSWORD to be set; never commit a real password.
BAL_PG_PASSWORD=change_me
BAL_PG_USER=bal
BAL_PG_DB=baldb
# === bal-pusher === # === bal-pusher ===
BAL_PUSHER_DB_FILE=/var/bal/bal.db BAL_SERVER_DB_FILE=/var/bal/bal.db
BAL_PUSHER_BITCOIN_DIR=/home/bal/.bitcoin BAL_PUSHER_BITCOIN_DIR=/home/bal/.bitcoin
BAL_PUSHER_SEND_STATS=false BAL_PUSHER_SEND_STATS=false
BAL_SERVER_URL=http://127.0.0.1:9137 BAL_SERVER_URL=http://127.0.0.1:9137

8
.gitignore vendored
View File

@@ -1,10 +1,18 @@
# Secrets and environment files - NEVER commit # Secrets and environment files - NEVER commit
.env .env
.env.* .env.*
.env.local
*.env.local
.env.production
.env.secret
!.env.example !.env.example
*.pem *.pem
private_key.pem
privkey.pem
!public_key.pem !public_key.pem
*.key *.key
ec.key
chiave_privata.key
# Databases # Databases
*.db *.db

View File

@@ -22,7 +22,7 @@ serde = { version = "1.0.152", features = ["derive"] }
serde_json = { version = "1.0.116" } serde_json = { version = "1.0.116" }
sqlx = { version = "0.8", features = ["runtime-tokio", "sqlite", "postgres", "chrono"] } sqlx = { version = "0.8", features = ["runtime-tokio", "sqlite", "postgres", "chrono"] }
regex = { version = "1.10.4" } regex = { version = "1.10.4" }
tokio = { version = "1", features = ["rt", "net","macros","rt-multi-thread"] } tokio = { version = "1", features = ["rt", "net","macros","rt-multi-thread","time"] }
url = { version = "2" } url = { version = "2" }
# server-only # server-only

View File

@@ -82,7 +82,6 @@ ENV RUST_LOG=info \
BAL_SERVER_BIND_ADDRESS=127.0.0.1 \ BAL_SERVER_BIND_ADDRESS=127.0.0.1 \
BAL_SERVER_BIND_PORT=9137 \ BAL_SERVER_BIND_PORT=9137 \
BAL_SERVER_DB_FILE=/var/bal/bal.db \ BAL_SERVER_DB_FILE=/var/bal/bal.db \
BAL_PUSHER_DB_FILE=/var/bal/bal.db \
BAL_SERVER_URL=http://127.0.0.1:9137 \ BAL_SERVER_URL=http://127.0.0.1:9137 \
BAL_SERVER_PUB_KEY_PATH=/var/bal/public_key.pem \ BAL_SERVER_PUB_KEY_PATH=/var/bal/public_key.pem \
SSL_KEY_PATH=/var/bal/private_key.pem SSL_KEY_PATH=/var/bal/private_key.pem

View File

@@ -90,7 +90,6 @@ ENV RUST_LOG=info \
BAL_SERVER_BIND_ADDRESS=127.0.0.1 \ BAL_SERVER_BIND_ADDRESS=127.0.0.1 \
BAL_SERVER_BIND_PORT=9137 \ BAL_SERVER_BIND_PORT=9137 \
BAL_SERVER_DB_FILE=/var/bal/bal.db \ BAL_SERVER_DB_FILE=/var/bal/bal.db \
BAL_PUSHER_DB_FILE=/var/bal/bal.db \
BAL_SERVER_URL=http://127.0.0.1:9137 \ BAL_SERVER_URL=http://127.0.0.1:9137 \
BAL_SERVER_PUB_KEY_PATH=/var/bal/public_key.pem \ BAL_SERVER_PUB_KEY_PATH=/var/bal/public_key.pem \
SSL_KEY_PATH=/var/bal/private_key.pem SSL_KEY_PATH=/var/bal/private_key.pem

View File

@@ -143,7 +143,7 @@ If no network is specified, defaults to `bitcoin`.
| Variable | Description | Default | | Variable | Description | Default |
| --- | --- | --- | | --- | --- | --- |
| `BAL_PUSHER_DB_FILE` | Path to the SQLite3 database file. | `bal.db` | | `BAL_SERVER_DB_FILE` | Path to the SQLite3 database file. | `bal.db` |
| `BAL_PUSHER_SEND_STATS` | Send stats to welist server. | `false` | | `BAL_PUSHER_SEND_STATS` | Send stats to welist server. | `false` |
| `BAL_SERVER_URL` | URL of bal-server (for stats reporting). | - | | `BAL_SERVER_URL` | URL of bal-server (for stats reporting). | - |
| `SSL_KEY_PATH` | Ed25519 private key for signing stats reports. | `private_key.pem` | | `SSL_KEY_PATH` | Ed25519 private key for signing stats reports. | `private_key.pem` |

18
bal-pusher.sh Normal file → Executable file
View File

@@ -1,6 +1,15 @@
#!/usr/bin/env bash
set -e
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
. "$SCRIPT_DIR/lib.sh"
WORKING_DIR=$(pwd)
export RUST_LOG=trace export RUST_LOG=trace
export BAL_PUSHER_DB_FILE="$(pwd)/bal.db" export BAL_SERVER_DB_FILE="$WORKING_DIR/bal.db"
export BAL_SERVER_DB_BACKEND="postgresql"
#export BAL_PUSHER_BITCOIN_COOKIE_FILE=/~/.bitcoin/.cookie #export BAL_PUSHER_BITCOIN_COOKIE_FILE=/~/.bitcoin/.cookie
#export BAL_PUSHER_REGTEST_COOKIE_FILE=/~/.bitcoin/regtest/.cookie #export BAL_PUSHER_REGTEST_COOKIE_FILE=/~/.bitcoin/regtest/.cookie
#export BAL_PUSHER_TESTNET_COOKIE_FILE=/~/.bitcoin/testnet3/.cookie #export BAL_PUSHER_TESTNET_COOKIE_FILE=/~/.bitcoin/testnet3/.cookie
@@ -11,5 +20,10 @@ export BAL_PUSHER_SEND_STATS=true
export WELIST_SERVER_URL=http://localhost:8086 export WELIST_SERVER_URL=http://localhost:8086
export WELIST_SKIP_URL_VALIDATION=true export WELIST_SKIP_URL_VALIDATION=true
export BAL_SERVER_URL="http://127.0.0.1:9133" export BAL_SERVER_URL="http://127.0.0.1:9133"
export SSL_KEY_PATH="$(pwd)/private_key.pem" export SSL_KEY_PATH="$WORKING_DIR/private_key.pem"
if [ "$BAL_SERVER_DB_BACKEND" = "postgresql" ]; then
. "$SCRIPT_DIR/pg-server.sh"
fi
cargo run --bin=bal-pusher regtest --features=pusher cargo run --bin=bal-pusher regtest --features=pusher

View File

@@ -31,31 +31,7 @@ export BAL_SERVER_REGTEST_FIXED_FEE=1000
export BAL_SERVER_DB_BACKEND="postgresql" export BAL_SERVER_DB_BACKEND="postgresql"
if [ "$BAL_SERVER_DB_BACKEND" = "postgresql" ]; then if [ "$BAL_SERVER_DB_BACKEND" = "postgresql" ]; then
PG_CONTAINER="bal-pg" . "$SCRIPT_DIR/pg-server.sh"
PG_DSN="${BAL_SERVER_PG_DSN:-postgres://bal:balpass@localhost:5432/baldb}"
docker rm -f "$PG_CONTAINER" 2>/dev/null || true
echo_i "Starting PostgreSQL container..."
docker run -d --name "$PG_CONTAINER" \
-e POSTGRES_USER=bal \
-e POSTGRES_PASSWORD=balpass \
-e POSTGRES_DB=baldb \
-p 5432:5432 postgres:16-alpine
echo_i "Waiting for PostgreSQL..."
until docker exec "$PG_CONTAINER" pg_isready -U bal >/dev/null 2>&1; do
sleep 1
done
echo_i "PostgreSQL ready at $PG_DSN"
export BAL_SERVER_PG_DSN="$PG_DSN"
cleanup() {
echo_i "Stopping PostgreSQL container..."
docker stop "$PG_CONTAINER" >/dev/null 2>&1
docker rm "$PG_CONTAINER" >/dev/null 2>&1
}
trap cleanup EXIT
fi fi
cargo run --bin=bal-server cargo run --bin=bal-server

View File

@@ -47,7 +47,7 @@ BAL_SERVER_INFO="$info"
EOF EOF
) )
bal_pusher_conf=$(cat << EOF bal_pusher_conf=$(cat << EOF
BAL_PUSHER_DB_FILE=/home/bal/bal.db BAL_SERVER_DB_FILE=/home/bal/bal.db
BAL_PUSHER_BITCOIN_COOKIE_FILE=/home/bitcoin/.bitcoin/.cookie BAL_PUSHER_BITCOIN_COOKIE_FILE=/home/bitcoin/.bitcoin/.cookie
EOF EOF

View File

@@ -201,7 +201,7 @@ This is the async daemon that monitors the blockchain and pushes pending transac
### Configuration ### Configuration
All configuration is via environment variables (no config files): All configuration is via environment variables (no config files):
- `BAL_PUSHER_DB_FILE`: Path to SQLite database. - `BAL_SERVER_DB_FILE`: Path to SQLite database.
- `BAL_PUSHER_BITCOIN_DIR`: Bitcoin data directory (for cookie file path). - `BAL_PUSHER_BITCOIN_DIR`: Bitcoin data directory (for cookie file path).
- `BAL_PUSHER_SEND_STATS`: Enable/disable remote stats reporting. - `BAL_PUSHER_SEND_STATS`: Enable/disable remote stats reporting.
- `BAL_SERVER_URL`: URL of the bal-server for internal communication. - `BAL_SERVER_URL`: URL of the bal-server for internal communication.

View File

@@ -8,7 +8,7 @@
## Database Technology ## Database Technology
- **Engine:** `sqlite` (Rust `sqlite` crate, version 0.34.0) - **Engine:** `sqlite` (Rust `sqlite` crate, version 0.34.0)
- **File:** `bal.db` (default, configurable via `BAL_SERVER_DB_FILE` / `BAL_PUSHER_DB_FILE`) - **File:** `bal.db` (default, configurable via `BAL_SERVER_DB_FILE`)
- **Connection Management:** Shared `Arc<Mutex<Connection>>` in `bal-server`, single connection in `bal-pusher`. - **Connection Management:** Shared `Arc<Mutex<Connection>>` in `bal-server`, single connection in `bal-pusher`.
- **WAL Mode:** Enabled via `PRAGMA journal_mode=WAL` with retry logic (up to 5 attempts) for concurrent access safety. - **WAL Mode:** Enabled via `PRAGMA journal_mode=WAL` with retry logic (up to 5 attempts) for concurrent access safety.
- **Busy Timeout:** Set to 5000ms via `PRAGMA busy_timeout=5000`. - **Busy Timeout:** Set to 5000ms via `PRAGMA busy_timeout=5000`.

View File

@@ -41,25 +41,30 @@ Example: `BAL_SERVER_REGTEST_ADDRESS=tpub...`, `BAL_SERVER_BITCOIN_FIXED_FEE=500
| `BAL_SERVER_ACTIX_WORKERS` | `4` | Number of actix-web worker threads | | `BAL_SERVER_ACTIX_WORKERS` | `4` | Number of actix-web worker threads |
| `BAL_SERVER_ACTIX_MAX_CONNECTIONS` | `100` | Maximum concurrent connections | | `BAL_SERVER_ACTIX_MAX_CONNECTIONS` | `100` | Maximum concurrent connections |
| `BAL_SERVER_TRUSTED_PROXY` | `127.0.0.1` | Trusted reverse proxy IP for rate-limiting client identification | | `BAL_SERVER_TRUSTED_PROXY` | `127.0.0.1` | Trusted reverse proxy IP for rate-limiting client identification |
| `BAL_SERVER_ACTIX_PUSHTXS_PER_SEC` | `1` | Rate limit: requests per second (applied to all endpoints) | | `BAL_SERVER_ACTIX_PUSHTXS_PER_SEC` | `1` | Rate limit: push txs requests per second |
| `BAL_SERVER_ACTIX_PUSHTXS_BURST` | `3` | Rate limit: burst size (applied to all endpoints) | | `BAL_SERVER_ACTIX_PUSHTXS_BURST` | `3` | Rate limit: push txs burst size |
| `BAL_SERVER_ACTIX_SEARCHTX_PER_SEC` | `5` | Rate limit: searchtx requests per second | | `BAL_SERVER_ACTIX_SEARCHTX_PER_SEC` | `5` | Rate limit: searchtx requests per second |
| `BAL_SERVER_ACTIX_SEARCHTX_BURST` | `10` | Rate limit: searchtx burst size | | `BAL_SERVER_ACTIX_SEARCHTX_BURST` | `10` | Rate limit: searchtx burst size |
| `BAL_SERVER_ACTIX_INFO_PER_SEC` | `20` | Rate limit: info requests per second | | `BAL_SERVER_ACTIX_INFO_PER_SEC` | `20` | Rate limit: info/stats requests per second |
| `BAL_SERVER_ACTIX_INFO_BURST` | `30` | Rate limit: info burst size | | `BAL_SERVER_ACTIX_INFO_BURST` | `30` | Rate limit: info/stats burst size |
| `BAL_SERVER_ACTIX_DEFAULT_PER_SEC` | `50` | Rate limit: default requests per second | | `BAL_SERVER_ACTIX_DEFAULT_PER_SEC` | `50` | Rate limit: default requests per second (`/`, `/.pub_key.pem`, `/version`) |
| `BAL_SERVER_ACTIX_DEFAULT_BURST` | `100` | Rate limit: default burst size | | `BAL_SERVER_ACTIX_DEFAULT_BURST` | `100` | Rate limit: default burst size |
Each endpoint class has its own independent token bucket, keyed by the real
client IP (see `BAL_SERVER_TRUSTED_PROXY`). `BAL_SERVER_ACTIX_TIMEOUT_SECS` is
enforced by a request-timeout middleware that returns `408 Request Timeout` if
the handler (including request body reading) exceeds the limit.
### `bal-pusher` (prefixed `BAL_PUSHER_`) ### `bal-pusher` (prefixed `BAL_PUSHER_`)
#### Core Settings #### Core Settings
| Variable | Default | Description | | Variable | Default | Description |
|----------|---------|-------------| |----------|---------|-------------|
| `BAL_PUSHER_DB_FILE` | `"bal.db"` | Path to the SQLite database file | | `BAL_SERVER_DB_FILE` | `"bal.db"` | Path to the SQLite database file |
| `BAL_PUSHER_BITCOIN_DIR` | `""` | Bitcoin data directory (for cookie file path resolution) | | `BAL_PUSHER_BITCOIN_DIR` | `""` | Bitcoin data directory (for cookie file path resolution) |
| `BAL_PUSHER_SEND_STATS` | `false` | Enable/disable remote stats reporting | | `BAL_PUSHER_SEND_STATS` | `false` | Enable/disable remote stats reporting |
| `BAL_SERVER_URL` | `"http://localhost/"` | URL of the bal-server for internal communication | | `BAL_SERVER_URL` | `"http://127.0.0.1:9133"` | URL of the bal-server for internal communication |
| `SSL_KEY_PATH` | `"privkey.pem"` | Path to Ed25519 private key for signing stats | | `SSL_KEY_PATH` | `"privkey.pem"` | Path to Ed25519 private key for signing stats |
| `BAL_PUSHER_PREFER_IPV6` | `false` | Pin HTTP connection to first IPv6 address (for broken IPv4 routes) | | `BAL_PUSHER_PREFER_IPV6` | `false` | Pin HTTP connection to first IPv6 address (for broken IPv4 routes) |
| `WELIST_SERVER_URL` | `"https://welist.bitcoin-after.life"` | URL to POST signed stats to (validated against SSRF) | | `WELIST_SERVER_URL` | `"https://welist.bitcoin-after.life"` | URL to POST signed stats to (validated against SSRF) |

View File

@@ -36,7 +36,7 @@
**Description:** All `unwrap()`/`expect()` calls on critical paths have been replaced with safe error handling. **Description:** All `unwrap()`/`expect()` calls on critical paths have been replaced with safe error handling.
**Mitigation Applied:** **Mitigation Applied:**
- `bal-server`: `from_utf8` returns 400, `sqlite::open` uses `open_db()` with validation, all handlers return proper HTTP status codes. - `bal-server`: `from_utf8` returns 400, `sqlite::open` uses `open_db()` with validation, all handlers return proper HTTP status codes.
- `bal-pusher`: RPC failures log errors + sleep + retry (no panic), ZMQ `recv` uses `set_rcvtimeo(5000)`, `fs::read_to_string` uses `match` + 500, `cfg.lock()` uses `poisoned.into_inner()` recovery. - `bal-pusher`: RPC failures log errors + sleep + retry (no panic), ZMQ `recv` uses `set_rcvtimeo(5000)`, malformed ZMQ frames are handled with `get()`/`first()` and `from_utf8_lossy` (no indexing or UTF-8 `expect`), stats signing returns an error that is logged and skipped rather than `expect`-panicking.
**Regression Tests:** `tests/panic_regression_tests.rs` (2 tests). **Regression Tests:** `tests/panic_regression_tests.rs` (2 tests).
### 3. Secret Leakage (FIXED) ### 3. Secret Leakage (FIXED)
@@ -52,18 +52,20 @@
**Description:** All DoS vectors mitigated via actix-web migration. **Description:** All DoS vectors mitigated via actix-web migration.
**Mitigation Applied:** **Mitigation Applied:**
- Body size limit: `PayloadConfig::default().limit(max_body_size)` via `BAL_SERVER_ACTIX_MAX_BODY_SIZE` (default 1 MiB). - Body size limit: `PayloadConfig::default().limit(max_body_size)` via `BAL_SERVER_ACTIX_MAX_BODY_SIZE` (default 1 MiB).
- Rate limiting: `actix-governor` with token-bucket per client IP (`BAL_SERVER_ACTIX_PUSHTXS_PER_SEC`/`BURST`). Uses `RealIpKeyExtractor` to extract real client IP from proxy headers. - Rate limiting: `actix-governor` with a token bucket per client IP, with an **independent bucket for each endpoint class** (push txs, searchtx, info/stats, default) so that polling a cheap endpoint cannot exhaust the push-txs budget. Uses `RealIpKeyExtractor` to extract the real client IP from proxy headers.
- Connection limits: `workers(4)` and `max_connections(100)` via `BAL_SERVER_ACTIX_WORKERS`/`MAX_CONNECTIONS`. - Connection limits: `workers(4)` and `max_connections(100)` via `BAL_SERVER_ACTIX_WORKERS`/`MAX_CONNECTIONS`.
- Body timeout: configurable via `BAL_SERVER_ACTIX_TIMEOUT_SECS`. - Request timeout: `middleware::from_fn` wraps every handler in `tokio::time::timeout`, returning `408 Request Timeout` when `BAL_SERVER_ACTIX_TIMEOUT_SECS` (default 5) elapses, covering slow request bodies.
- ZMQ timeout: `set_rcvtimeo(5000)` prevents infinite blocking. - ZMQ timeout: `set_rcvtimeo(5000)` prevents infinite blocking.
- RPC retry: sleep + retry on connection failure instead of panic. - RPC retry: sleep + retry on connection failure instead of panic.
- ZMQ message handling is panic-free: multipart frames are accessed with `get()`/`first()` and topics are decoded with `from_utf8_lossy`, so malformed frames cannot abort the pusher.
- Stats-report signing failures are logged and skipped instead of panicking the pusher when the key is unreadable.
### 5. SSRF / Network Abuse via `reqwest` (FIXED) ### 5. SSRF / Network Abuse via `reqwest` (FIXED)
**Severity:** MEDIUM | **Status:** Fixed **Severity:** MEDIUM | **Status:** Fixed
**Location:** `src/bin/bal-pusher.rs`, `src/validation.rs` **Location:** `src/bin/bal-pusher.rs`, `src/validation.rs`
**Description:** URL validation prevents redirecting requests to internal/private IPs. **Description:** URL validation prevents redirecting requests to internal/private IPs.
**Mitigation Applied:** `is_valid_welist_url()` in `src/validation.rs` blocks localhost, loopback, RFC1918, link-local, multicast, unspecified, and IPv6 unique-local addresses. HTTPS-only scheme enforced. **Mitigation Applied:** `is_valid_welist_url()` in `src/validation.rs` blocks localhost, loopback, RFC1918, link-local, multicast, unspecified, IPv6 unique-local addresses, and **IPv4-mapped IPv6 addresses** (`::ffff:a.b.c.d`) that route to loopback/private/link-local/unspecified IPv4 targets. HTTPS-only scheme enforced. Note: hostnames are not DNS-resolved before validation, so a domain that resolves to a private address is not caught by this check — restrict `WELIST_SERVER_URL` to trusted values.
**Regression Tests:** `tests/ssrf_tests.rs` (integration) + 8 unit tests in `src/validation.rs`. **Regression Tests:** `tests/ssrf_tests.rs` (integration) + 11 unit tests in `src/validation.rs`.
### 6. Insecure Database Access (FIXED) ### 6. Insecure Database Access (FIXED)
**Severity:** MEDIUM | **Status:** Fixed **Severity:** MEDIUM | **Status:** Fixed
@@ -130,16 +132,17 @@
6. Use read-only filesystem for the server binary. 6. Use read-only filesystem for the server binary.
### Application-Level ### Application-Level
1. **Rate Limiting:** Implemented via `actix-governor` with token-bucket per client IP. `RealIpKeyExtractor` identifies clients behind reverse proxy using `X-Real-IP` / `X-Forwarded-For` headers. Trusted proxy IP configurable via `BAL_SERVER_TRUSTED_PROXY`. 1. **Rate Limiting:** Implemented via `actix-governor` with token-bucket per client IP and a **separate bucket per endpoint class**. `RealIpKeyExtractor` identifies clients behind reverse proxy using `X-Real-IP` / `X-Forwarded-For` headers. Trusted proxy IP configurable via `BAL_SERVER_TRUSTED_PROXY`.
2. **Input Validation:** Network enum check, txid hex validation, body size limits. 2. **Input Validation:** Network enum check, txid hex validation, body size limits.
3. **HTTPS:** Via Nginx reverse proxy with Let's Encrypt. 3. **Request Timeout:** `tokio::time::timeout` middleware returns `408` when a handler (including body read) exceeds `BAL_SERVER_ACTIX_TIMEOUT_SECS`.
4. **WAL Mode:** Enabled with retry logic for concurrent access. 4. **HTTPS:** Via Nginx reverse proxy with Let's Encrypt.
5. **ZMQ Timeout:** 5-second receive timeout prevents infinite blocking. 5. **WAL Mode:** Enabled with retry logic for concurrent access.
6. **Transaction Size Limits:** Configurable via rate limiting and body size. 6. **ZMQ Timeout:** 5-second receive timeout prevents infinite blocking.
7. **ZMQ Retry:** Reconnect logic with timeout-based detection. 7. **Transaction Size Limits:** Configurable via rate limiting and body size.
8. **Fee Limits:** Per-network `fixed_fee` configuration. 8. **ZMQ Retry:** Reconnect logic with timeout-based detection.
9. **Network Limits:** Only known networks accepted (bitcoin, testnet, testnet4, signet, regtest). 9. **Fee Limits:** Per-network `fixed_fee` configuration.
10. **Locktime Reasonableness:** Locktime compared against blockchain height and median time with threshold-based distinction. 10. **Network Limits:** Only known networks accepted (bitcoin, testnet, testnet4, signet, regtest).
11. **Locktime Reasonableness:** Locktime compared against blockchain height and median time with threshold-based distinction.
--- ---
@@ -153,4 +156,4 @@
| `tests/secret_leakage_tests.rs` | 3 | .gitignore, no tracked secrets, no hardcoded tokens | | `tests/secret_leakage_tests.rs` | 3 | .gitignore, no tracked secrets, no hardcoded tokens |
| `tests/input_validation_tests.rs` | 4 | Address caching, network validation, txid hex validation | | `tests/input_validation_tests.rs` | 4 | Address caching, network validation, txid hex validation |
| `tests/db_path_validation.rs` | 5 | Path traversal, forbidden dirs, symlinks, WAL pragma, valid paths | | `tests/db_path_validation.rs` | 5 | Path traversal, forbidden dirs, symlinks, WAL pragma, valid paths |
| `src/validation.rs` (inline) | 8 | SSRF URL validation unit tests | | `src/validation.rs` (inline) | 11 | SSRF URL validation unit tests |

34
pg-server.sh Executable file
View File

@@ -0,0 +1,34 @@
#!/usr/bin/env bash
set -e
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
. "$SCRIPT_DIR/lib.sh"
PG_CONTAINER="bal-pg"
PG_USER="${BAL_PG_USER:-bal}"
PG_DB="${BAL_PG_DB:-baldb}"
# The password MUST come from the environment; never hardcode it.
PG_PASSWORD="${BAL_PG_PASSWORD:?BAL_PG_PASSWORD must be set (see .env.example)}"
PG_DSN="${BAL_SERVER_PG_DSN:-postgres://${PG_USER}:${PG_PASSWORD}@localhost:5432/${PG_DB}}"
if docker ps --format '{{.Names}}' | grep -q "^${PG_CONTAINER}$"; then
echo_i "PostgreSQL container '$PG_CONTAINER' is already running"
elif docker ps -a --format '{{.Names}}' | grep -q "^${PG_CONTAINER}$"; then
echo_i "Starting existing PostgreSQL container '$PG_CONTAINER'..."
docker start "$PG_CONTAINER" >/dev/null 2>&1
else
echo_i "Creating PostgreSQL container '$PG_CONTAINER'..."
docker run -d --name "$PG_CONTAINER" \
-e POSTGRES_USER="$PG_USER" \
-e POSTGRES_PASSWORD="$PG_PASSWORD" \
-e POSTGRES_DB="$PG_DB" \
-p 5432:5432 postgres:16-alpine
fi
echo_i "Waiting for PostgreSQL..."
until docker exec "$PG_CONTAINER" pg_isready -U "$PG_USER" >/dev/null 2>&1; do
sleep 1
done
# Do not echo the DSN: it embeds the password.
echo_s "PostgreSQL ready (user=$PG_USER, db=$PG_DB, host=localhost:5432)"
export BAL_SERVER_PG_DSN="$PG_DSN"

View File

@@ -46,9 +46,9 @@ struct MyConfig {
impl Default for MyConfig { impl Default for MyConfig {
fn default() -> Self { fn default() -> Self {
MyConfig { MyConfig {
db_backend: env::var("BAL_PUSHER_DB_BACKEND").unwrap_or("sqlite".to_string()), db_backend: env::var("BAL_SERVER_DB_BACKEND").unwrap_or("sqlite".to_string()),
db_file: env::var("BAL_PUSHER_DB_FILE").unwrap_or("bal.db".to_string()), db_file: env::var("BAL_SERVER_DB_FILE").unwrap_or("bal.db".to_string()),
pg_dsn: env::var("BAL_PUSHER_PG_DSN").unwrap_or_default(), pg_dsn: env::var("BAL_SERVER_PG_DSN").unwrap_or_default(),
bitcoin_dir: env::var("BAL_PUSHER_BITCOIN_DIR").unwrap_or("".to_string()), bitcoin_dir: env::var("BAL_PUSHER_BITCOIN_DIR").unwrap_or("".to_string()),
regtest: get_network_params_default(Network::Regtest), regtest: get_network_params_default(Network::Regtest),
testnet: get_network_params_default(Network::Testnet), testnet: get_network_params_default(Network::Testnet),
@@ -59,7 +59,7 @@ impl Default for MyConfig {
.unwrap_or("false".to_string()) .unwrap_or("false".to_string())
.parse::<bool>() .parse::<bool>()
.unwrap_or(false), .unwrap_or(false),
url: env::var("BAL_SERVER_URL").unwrap_or("http://localhost/".to_string()), url: env::var("BAL_SERVER_URL").unwrap_or("http://127.0.0.1:9133".to_string()),
ssl_key_path: env::var("SSL_KEY_PATH").unwrap_or("privkey.pem".to_string()), ssl_key_path: env::var("SSL_KEY_PATH").unwrap_or("privkey.pem".to_string()),
} }
} }
@@ -388,7 +388,13 @@ async fn send_stats_report(
cfg.url, chain, bcinfo.blocks, bcinfo.median_time, bcinfo.best_block_hash cfg.url, chain, bcinfo.blocks, bcinfo.median_time, bcinfo.best_block_hash
); );
trace!("message to be sent: {}", message); trace!("message to be sent: {}", message);
let sign = sign_message(cfg.ssl_key_path.as_str(), message.as_str()); let sign = match sign_message(cfg.ssl_key_path.as_str(), message.as_str()) {
Ok(signature) => signature,
Err(e) => {
error!("Cannot sign stats report, skipping: {}", e);
return Ok(());
}
};
let response = client let response = client
.post(url) .post(url)
.header("User-Agent", format!("bal-pusher/{}", VERSION)) .header("User-Agent", format!("bal-pusher/{}", VERSION))
@@ -414,22 +420,22 @@ async fn send_stats_report(
} }
Ok(()) Ok(())
} }
fn sign_message(private_key_path: &str, message: &str) -> String { fn sign_message(private_key_path: &str, message: &str) -> Result<String, Box<dyn StdError>> {
let signing_key = let signing_key = SigningKey::read_pkcs8_pem_file(private_key_path)
SigningKey::read_pkcs8_pem_file(private_key_path).expect("failed to parse private key PEM"); .map_err(|e| format!("failed to parse private key PEM at {private_key_path}: {e}"))?;
let signature = signing_key.sign(message.as_bytes()); let signature = signing_key.sign(message.as_bytes());
general_purpose::STANDARD.encode(signature.to_bytes()) Ok(general_purpose::STANDARD.encode(signature.to_bytes()))
} }
fn parse_env(cfg: &mut MyConfig) { fn parse_env(cfg: &mut MyConfig) {
if let Ok(value) = env::var("BAL_PUSHER_DB_BACKEND") { if let Ok(value) = env::var("BAL_SERVER_DB_BACKEND") {
cfg.db_backend = value; cfg.db_backend = value;
} }
if let Ok(value) = env::var("BAL_PUSHER_DB_FILE") { if let Ok(value) = env::var("BAL_SERVER_DB_FILE") {
cfg.db_file = value; cfg.db_file = value;
} }
if let Ok(value) = env::var("BAL_PUSHER_PG_DSN") { if let Ok(value) = env::var("BAL_SERVER_PG_DSN") {
cfg.pg_dsn = value; cfg.pg_dsn = value;
} }
cfg.regtest = parse_env_netconfig(cfg, "regtest"); cfg.regtest = parse_env_netconfig(cfg, "regtest");
@@ -561,12 +567,12 @@ async fn main() -> std::io::Result<()> {
); );
} }
consecutive_timeouts = 0; consecutive_timeouts = 0;
let topic = message[0].clone(); let Some(topic) = message.first().cloned() else {
let body = message[1].clone(); warn!("ZMQ: received empty multipart message, skipping");
debug!( continue;
"ZMQ:GET TOPIC: {}", };
String::from_utf8(topic.clone()).expect("invalid topic") let body = message.get(1).cloned().unwrap_or_default();
); debug!("ZMQ:GET TOPIC: {}", String::from_utf8_lossy(&topic));
trace!("ZMQ:GET BODY: {}", hex::encode(&body)); trace!("ZMQ:GET BODY: {}", hex::encode(&body));
if topic == b"hashblock" { if topic == b"hashblock" {
info!("NEW BLOCK: {}", hex::encode(&body)); info!("NEW BLOCK: {}", hex::encode(&body));

View File

@@ -1,8 +1,11 @@
use actix_governor::{Governor, GovernorConfigBuilder, KeyExtractor, SimpleKeyExtractionError}; use actix_governor::{
use actix_web::dev::ServiceRequest; Governor, GovernorConfig, GovernorConfigBuilder, KeyExtractor, SimpleKeyExtractionError,
use actix_web::middleware; };
use actix_web::body::MessageBody;
use actix_web::dev::{ServiceRequest, ServiceResponse};
use actix_web::middleware::{self, Next};
use actix_web::web::Bytes; use actix_web::web::Bytes;
use actix_web::{App, HttpResponse, HttpServer, Responder, web}; use actix_web::{App, Error, HttpResponse, HttpServer, Responder, web};
use bitcoin::{Network, Transaction, consensus}; use bitcoin::{Network, Transaction, consensus};
use chrono::Utc; use chrono::Utc;
use hex_conservative::FromHex; use hex_conservative::FromHex;
@@ -13,6 +16,7 @@ use std::env;
use std::fs; use std::fs;
use std::net::IpAddr; use std::net::IpAddr;
use std::str::FromStr; use std::str::FromStr;
use std::time::Duration;
use bal_server::db::{ use bal_server::db::{
DatabasePool, InsertInpData, InsertOutData, InsertTxData, check_duplicate_txids, DatabasePool, InsertInpData, InsertOutData, InsertTxData, check_duplicate_txids,
@@ -123,7 +127,6 @@ pub struct StatsResponse {
} }
#[derive(Debug, Clone)] #[derive(Debug, Clone)]
#[expect(dead_code)]
struct ActixConfig { struct ActixConfig {
max_body_size: usize, max_body_size: usize,
timeout_secs: u64, timeout_secs: u64,
@@ -287,6 +290,44 @@ impl KeyExtractor for RealIpKeyExtractor {
} }
} }
/// Concrete governor configuration type used by all rate-limited routes.
///
/// `NoOpMiddleware` is the default `actix-governor` middleware; it is only used
/// here as a type parameter so a single helper can build every route's config.
type GovernorConfigType =
GovernorConfig<RealIpKeyExtractor, actix_governor::governor::middleware::NoOpMiddleware>;
/// Build a rate-limiter configuration keyed by the real client IP.
///
/// A period of zero is rejected by `actix-governor`, so the values are clamped
/// to a minimum of one to avoid a `None` result.
fn build_governor_config(per_sec: u64, burst: u32) -> Result<GovernorConfigType, String> {
GovernorConfigBuilder::default()
.seconds_per_request(per_sec.max(1))
.burst_size(burst.max(1))
.key_extractor(RealIpKeyExtractor)
.finish()
.ok_or_else(|| "invalid rate limiter configuration".to_string())
}
/// Middleware that aborts a request with `408 Request Timeout` if the inner
/// handler (including request body reading) exceeds the configured duration.
///
/// The duration is read from the `web::Data<u64>` app data registered in `main`.
async fn request_timeout<B>(
timeout_secs: web::Data<u64>,
req: ServiceRequest,
next: Next<B>,
) -> Result<ServiceResponse<B>, Error>
where
B: MessageBody,
{
match tokio::time::timeout(Duration::from_secs(**timeout_secs), next.call(req)).await {
Ok(result) => result,
Err(_) => Err(actix_web::error::ErrorRequestTimeout("request timeout")),
}
}
async fn echo_info( async fn echo_info(
path: web::Path<String>, path: web::Path<String>,
data: web::Data<AppState>, data: web::Data<AppState>,
@@ -464,6 +505,7 @@ fn parse_request_transactions(
known_addresses: &HashSet<String>, known_addresses: &HashSet<String>,
) -> Vec<(ParsedTx, String, u64)> { ) -> Vec<(ParsedTx, String, u64)> {
let mut result: Vec<(ParsedTx, String, u64)> = Vec::new(); let mut result: Vec<(ParsedTx, String, u64)> = Vec::new();
let mut seen_txids: HashSet<String> = HashSet::new();
for line in strbody.split('\n') { for line in strbody.split('\n') {
if line.is_empty() { if line.is_empty() {
@@ -489,6 +531,10 @@ fn parse_request_transactions(
}; };
let txid = tx.compute_txid().to_string(); let txid = tx.compute_txid().to_string();
if !seen_txids.insert(txid.clone()) {
trace!("duplicate txid {} in request body, skipping", txid);
continue;
}
let ntxid = tx.compute_ntxid(); let ntxid = tx.compute_ntxid();
let wtxid = tx.compute_wtxid(); let wtxid = tx.compute_wtxid();
let locktime = tx.lock_time.to_string(); let locktime = tx.lock_time.to_string();
@@ -750,7 +796,10 @@ fn parse_env_netconfig(cfg: &mut MyConfig, chain: &str) {
if let Ok(value) = env::var(format!("BAL_SERVER_{}_ADDRESS", chain.to_uppercase())) { if let Ok(value) = env::var(format!("BAL_SERVER_{}_ADDRESS", chain.to_uppercase())) {
debug!("BAL_SERVER_{}_ADDRESS: {}", chain.to_uppercase(), value); debug!("BAL_SERVER_{}_ADDRESS: {}", chain.to_uppercase(), value);
c.address = value; c.address = value;
if c.address.len() > 5 && &c.address[1..4] == "pub" { let is_xpub = ["xpub", "ypub", "zpub", "tpub", "vpub", "upub"]
.iter()
.any(|prefix| c.address.starts_with(prefix));
if is_xpub {
c.xpub = true; c.xpub = true;
trace!("is_xpub"); trace!("is_xpub");
} }
@@ -813,12 +862,26 @@ async fn main() -> std::io::Result<()> {
let bind_address = data.cfg.bind_address.clone(); let bind_address = data.cfg.bind_address.clone();
let bind_port = data.cfg.bind_port; let bind_port = data.cfg.bind_port;
let governor_conf = GovernorConfigBuilder::default() // One shared limiter per endpoint class: all workers clone the same
.seconds_per_request(actix_cfg.rate_limit_pushtxs.0) // underlying state, so the rate limit is global per real client IP.
.burst_size(actix_cfg.rate_limit_pushtxs.1) let governor_pushtxs = build_governor_config(
.key_extractor(RealIpKeyExtractor) actix_cfg.rate_limit_pushtxs.0,
.finish() actix_cfg.rate_limit_pushtxs.1,
.unwrap(); )
.map_err(std::io::Error::other)?;
let governor_searchtx = build_governor_config(
actix_cfg.rate_limit_searchtx.0,
actix_cfg.rate_limit_searchtx.1,
)
.map_err(std::io::Error::other)?;
let governor_info =
build_governor_config(actix_cfg.rate_limit_info.0, actix_cfg.rate_limit_info.1)
.map_err(std::io::Error::other)?;
let governor_default = build_governor_config(
actix_cfg.rate_limit_default.0,
actix_cfg.rate_limit_default.1,
)
.map_err(std::io::Error::other)?;
println!("Starting server on http://{}:{}", bind_address, bind_port); println!("Starting server on http://{}:{}", bind_address, bind_port);
@@ -827,16 +890,45 @@ async fn main() -> std::io::Result<()> {
.app_data(web::PayloadConfig::default().limit(actix_cfg.max_body_size)) .app_data(web::PayloadConfig::default().limit(actix_cfg.max_body_size))
.app_data(data.clone()) .app_data(data.clone())
.app_data(web::Data::new(actix_cfg.trusted_proxy)) .app_data(web::Data::new(actix_cfg.trusted_proxy))
.app_data(web::Data::new(actix_cfg.timeout_secs))
.wrap(middleware::Logger::default()) .wrap(middleware::Logger::default())
.wrap(middleware::Compress::default()) .wrap(middleware::Compress::default())
.wrap(Governor::new(&governor_conf)) .wrap(middleware::from_fn(request_timeout))
.service(web::resource("/").route(web::get().to(echo_home))) .service(
.service(web::resource("/.pub_key.pem").route(web::get().to(echo_pub_key))) web::resource("/")
.service(web::resource("/version").route(web::get().to(echo_version))) .wrap(Governor::new(&governor_default))
.service(web::resource("/{network}/info").route(web::get().to(echo_info))) .route(web::get().to(echo_home)),
.service(web::resource("/{network}/stats").route(web::get().to(echo_stats))) )
.service(web::resource("/{network}/pushtxs").route(web::post().to(echo_push))) .service(
.service(web::resource("/searchtx").route(web::post().to(echo_search))) web::resource("/.pub_key.pem")
.wrap(Governor::new(&governor_default))
.route(web::get().to(echo_pub_key)),
)
.service(
web::resource("/version")
.wrap(Governor::new(&governor_default))
.route(web::get().to(echo_version)),
)
.service(
web::resource("/{network}/info")
.wrap(Governor::new(&governor_info))
.route(web::get().to(echo_info)),
)
.service(
web::resource("/{network}/stats")
.wrap(Governor::new(&governor_info))
.route(web::get().to(echo_stats)),
)
.service(
web::resource("/{network}/pushtxs")
.wrap(Governor::new(&governor_pushtxs))
.route(web::post().to(echo_push)),
)
.service(
web::resource("/searchtx")
.wrap(Governor::new(&governor_searchtx))
.route(web::post().to(echo_search)),
)
}) })
.workers(actix_cfg.workers) .workers(actix_cfg.workers)
.max_connections(actix_cfg.max_connections) .max_connections(actix_cfg.max_connections)

View File

@@ -38,13 +38,23 @@ fn validate_sqlite_path(dsn: &str) -> Result<(), String> {
// If absolute, block known sensitive system directories // If absolute, block known sensitive system directories
if p.is_absolute() { if p.is_absolute() {
let forbidden = [ let forbidden = [
"/etc", "/proc", "/sys", "/dev", "/usr", "/bin", "/sbin", "/lib", "/opt", Path::new("/etc"),
Path::new("/proc"),
Path::new("/sys"),
Path::new("/dev"),
Path::new("/usr"),
Path::new("/bin"),
Path::new("/sbin"),
Path::new("/lib"),
Path::new("/opt"),
]; ];
for prefix in &forbidden { for prefix in &forbidden {
if path_str.starts_with(prefix) { // `Path::starts_with` matches whole path components, so "/etcfoo"
// is not treated as being under "/etc".
if p.starts_with(prefix) {
return Err(format!( return Err(format!(
"Absolute database path under {} is forbidden", "Absolute database path under {} is forbidden",
prefix prefix.display()
)); ));
} }
} }

View File

@@ -65,6 +65,19 @@ pub fn is_valid_welist_url(url_str: &str) -> bool {
{ {
return false; return false;
} }
// IPv4-mapped IPv6 addresses (e.g. `::ffff:127.0.0.1`) are
// routed to the embedded IPv4 address on dual-stack hosts, so
// they must be validated against the IPv4 rules as well.
if let Some(v4) = v6.to_ipv4_mapped()
&& (v4.is_loopback()
|| v4.is_private()
|| v4.is_link_local()
|| v4.is_unspecified()
|| v4.is_multicast())
{
return false;
}
} }
} }
} }
@@ -154,6 +167,38 @@ mod tests {
); );
} }
#[test]
fn test_ipv4_mapped_ipv6() {
assert!(
!is_valid_welist_url("https://[::ffff:127.0.0.1]"),
"IPv4-mapped loopback should be blocked"
);
assert!(
!is_valid_welist_url("https://[::ffff:10.0.0.1]"),
"IPv4-mapped private address should be blocked"
);
assert!(
!is_valid_welist_url("https://[::ffff:172.16.0.1]"),
"IPv4-mapped private address should be blocked"
);
assert!(
!is_valid_welist_url("https://[::ffff:192.168.1.1]"),
"IPv4-mapped private address should be blocked"
);
assert!(
!is_valid_welist_url("https://[::ffff:169.254.169.254]"),
"IPv4-mapped link-local address should be blocked"
);
assert!(
!is_valid_welist_url("https://[::ffff:0.0.0.0]"),
"IPv4-mapped unspecified address should be blocked"
);
assert!(
is_valid_welist_url("https://[::ffff:8.8.8.8]"),
"IPv4-mapped public address should be allowed"
);
}
#[test] #[test]
fn test_malformed_urls() { fn test_malformed_urls() {
assert!(!is_valid_welist_url("not a url")); assert!(!is_valid_welist_url("not a url"));