bitcoinafterlife/bal-server
1
0
Fork 1
Code Issues Pull Requests Actions Packages Projects Releases 7 Wiki Activity

Compare commits

base: bitcoinafterlife:56696050ec059b12aaa4fa9421d13f026d51c7bb
Branches Tags
bitcoinafterlife:main
bitcoinafterlife:v0.3.2 bitcoinafterlife:v0.3.1 bitcoinafterlife:v0.3.0 bitcoinafterlife:v0.2.2-1 bitcoinafterlife:v0.2.2 bitcoinafterlife:v0.2.1 bitcoinafterlife:v0.2.0 bitcoinafterlife:v0.1.0
..
compare: bitcoinafterlife:v0.3.0
Branches Tags
bitcoinafterlife:main
bitcoinafterlife:v0.3.2 bitcoinafterlife:v0.3.1 bitcoinafterlife:v0.3.0 bitcoinafterlife:v0.2.2-1 bitcoinafterlife:v0.2.2 bitcoinafterlife:v0.2.1 bitcoinafterlife:v0.2.0 bitcoinafterlife:v0.1.0

32 Commits
56696050ec ... v0.3.0

Author SHA1 Message Date
svatantrya
0a3bd34ecb release: bal-server-0.3.0 2026-07-19 21:29:18 -04:00
svatantrya
59774c0cbb release: bal-server-0.3.1 2026-07-19 20:16:44 -04:00
svatantrya
c47862a72f fix: default features for cargo build/test, build each binary separately in release 
- Set default = ["server", "pusher"] so cargo build/test works without flags
- Add required-features to each [[bin]] section
- make_release.sh builds each binary with --no-default-features for optimal size
 2026-07-19 19:53:41 -04:00
svatantrya
3f7608641d perf: reduce binary sizes from 15M/11M to 3.8M/5.5M 
- Add [profile.release]: opt-level=z, lto=true, strip=true, codegen-units=1, panic=abort
- Replace vendored openssl (~5MB) with ed25519-dalek (pure Rust, ~100KB)
- Feature-gate deps: server (actix, chrono) vs pusher (zmq, reqwest, ed25519-dalek)
- Remove unused confy dependency
- bal-pusher: 15M -> 3.8M (-75%)
- bal-server: 11M -> 5.5M (-50%)
 2026-07-19 19:27:42 -04:00
svatantrya
6cf85d4f63 Merge origin/main: resolve conflicts and add 10s timeout to welist_http_client 2026-07-19 18:13:21 -04:00
svatantrya
bd06140c1f fix(pusher): allow bypassing SSRF validation for local dev and fix RUST_LOG export 
- Add WELIST_SKIP_URL_VALIDATION env var to bypass URL validation in dev
- Fix bal-pusher.sh: export RUST_LOG so env_logger picks up log level
- Add WELIST_SKIP_URL_VALIDATION=true to bal-pusher.sh for regtest use
 2026-07-19 17:48:43 -04:00
svatantrya
cc39027590 fix(pusher): add ZMQ diagnostic logging and heartbeat monitoring 
- Log ZMQ subscribe confirmation with address
- Add consecutive timeout counter with periodic warn every 60s
- Log ZMQ connection restored after timeouts
- Log main_result errors instead of discarding with let _=
 2026-07-19 15:55:49 -04:00
svatantrya
5822d3e231 Merge pull request 'pusher: optional IPv6 preference for welist reports + log report failures' (#1) from SAFE21.io/bal-server:fix/pusher-welist-ipv6-preference into main 
Reviewed-on: #1
 2026-07-19 15:09:00 +00:00
SAFE21
4ed58b11c5 fix(pusher): log send_stats_report errors instead of discarding 
main_result called send_stats_report/calculate_stats with 'let _ = ...',
silently dropping any failure. A broken welist route (e.g. unreachable
IPv4 path) is then invisible in the logs and can go unnoticed for a long
time. Log failures with warn! so connectivity problems are diagnosable.
 2026-07-19 14:09:41 +02:00
SAFE21
5fbfa79c99 feat(pusher): optional IPv6 preference for welist reports (BAL_PUSHER_PREFER_IPV6) 
The welist host publishes both A and AAAA records. On networks where the
IPv4 route is broken (connection stalls after the TCP handshake) while
IPv6 works, the default connector may pick the broken family and the
report request hangs.

When BAL_PUSHER_PREFER_IPV6 is truthy, the pusher now resolves the welist
host itself and pins the reqwest client to its first IPv6 address; the
original hostname is still used for the Host header and TLS SNI. When the
variable is unset (default) or no AAAA record exists, behavior is
completely unchanged.

Includes unit tests for the URL host/port parsing and documentation in
docs/07_deployment_and_ops.md.
 2026-07-19 14:09:01 +02:00
svatantrya
94de433926 fix(pusher): improve welist response logging, error handling, and add request timeout 
- Always log HTTP status code and response body from welist (info level)
- Log send_stats_report errors at call site instead of silently discarding
- Add 10s timeout to reqwest client to prevent indefinite hangs
- Apply clippy fixes (is_empty, if-let chains, dead_code, etc.)
 2026-07-18 22:58:42 -04:00
svatantrya
84d4522ce3 fix(pushtxs): skip invalid txs instead of aborting batch, anonymize error responses 
- parse_request_transactions now skips txs without willexecutor output
  instead of returning an error that aborts the entire batch
- returns 'error' (400) only when NO txs are valid
- all HTTP error bodies replaced with generic 'error' to avoid leaking
  internal details
 2026-07-18 21:35:39 -04:00
svatantrya
5a2fc9c497 refactor: rename bal-server-actix.rs to bal-server.rs 
- Rename src/bin/bal-server-actix.rs -> src/bin/bal-server.rs
- Update Cargo.toml to point to new filename
- Update Dockerfile reference
- Update docs/08_security_audit.md references
 2026-07-17 10:31:56 -04:00
svatantrya
f28fad0194 chore: remove make_release.sh from tracking, keep local 2026-07-17 09:59:05 -04:00
svatantrya
53be95113e feat(release): add ASCII-armored .asc signature alongside .sig 2026-07-17 09:57:39 -04:00
svatantrya
1e813e1a75 fix(release): move asset paths after ASSET_PATH definition 2026-07-17 09:55:11 -04:00
svatantrya
2d7b767138 release: bal-server-0.3.0 2026-07-17 09:53:35 -04:00
svatantrya
b39397e742 feat(release): add signature, checksum and verification instructions 2026-07-17 09:51:37 -04:00
svatantrya
3cfb635da1 docs: fix Docker example to use regtest env vars consistently 2026-07-17 05:23:51 -04:00
svatantrya
cdc2d2d4c7 docs: rewrite README with Docker, per-network config, Actix DoS vars 2026-07-17 02:51:04 -04:00
svatantrya
40fa23b03b fix: Docker support, WAL race condition, pusher panic fixes 
- Add multi-stage Dockerfile with tini, non-root user, healthcheck
- Fix SQLite WAL mode race between bal-server and bal-pusher (busy_timeout + retry)
- Fix tbl_stats missing UNIQUE index for ON CONFLICT clause
- Replace unwrap() panics in bal-pusher with graceful error handling
- Add docker/entrypoint.sh with BAL_PUSHER_NETWORK support
- cargo fmt across all files
 2026-07-16 21:24:09 -04:00
svatantrya
3c4608bdb4 security: remove terms and personal info (Svātantrya) from files 2026-07-16 20:07:14 -04:00
svatantrya
94f64c6334 docs(deploy): align docs with contrib script fixes 
- 07_deployment_and_ops.md: update zmqpubhashblock note, clarify certbot --nginx,
  update Nginx checklist items referencing the install script
- 09_references_and_links.md: remove 'hardcoded xpub' from install script row
 2026-07-16 19:49:20 -04:00
svatantrya
2516fe5f61 fix(install-scripts): correct critical bugs in contrib scripts 
- download_and_install_bitcoincore.sh:
  - Fix apt install with single-quoted string (was treating all deps as one package)
  - Add sudo to apt install commands
  - Fix tar extraction: now finds extracted dir and copies bitcoind/bitcoin-cli to /usr/local/bin
  - Fix zmqpubrawblock -> zmqpubhashblock (pusher expects hashblock topic)
  - Fix install check (0 was testing cd, not tar) with explicit extraction check
  - Handle existing user in adduser
  - Fix source lib.sh path

- download_and_install_bal.sh:
  - Use curl -sfL (fail on 404, follow redirects) for Gitea download
  - Add null/empty check after jq download URL extraction
  - Change certbot --standalone to --nginx (avoids port 80 conflict)
  - Enable nginx site symlink (was commented out)
  - Handle existing 'bal' user in adduser

- install_tor.sh:
  - Add sudo to all root-requiring commands (apt, tee, cp, sed, systemctl)
  - Backup torrc with timestamp before modifying
  - Add DisableDebuggerAttachment hardening
 2026-07-16 19:39:10 -04:00
svatantrya
b0fd59e431 fix(install-scripts): harden contrib scripts and fix critical bugs 
- download_and_install_bal.sh: add set -euo pipefail, fix sudo sudo typo, fix nginx path (etc/nginx/etc/nginx -> etc/nginx), add chmod 600 for env files, reduce RestartSec to 5s, fix concatenated echo line, add systemctl daemon-reload
- download_and_install_bitcoincore.sh: add set -euo pipefail, fix source lib.sh path to use script directory
- install_tor.sh: add set -euo pipefail, detect architecture dynamically (dpkg --print-architecture), detect distro dynamically (lsb_release -cs), bind ControlPort to 127.0.0.1:9051, add DisableDebuggerAttachment
 2026-07-16 19:07:15 -04:00
svatantrya
24809b19e2 security: fix audit points 5-9 + optimize echo_push/info endpoints 
- Point 5 (SSRF): Add URL validation for WELIST_SERVER_URL (src/validation.rs)
- Point 6 (DB Access): Add DB path validation, symlink check, WAL mode (open_db)
- Point 8 (HTTPS): Extract nginx config, add deployment checklist, bind warnings
- Point 9 (Input Validation): Add NETWORKS check (404 for unknown), txid 64-hex validation
- Optimize echo_push: parse transactions outside DB lock, batch duplicate check, N+1 xpub lookup eliminated via HashSet cache
- Optimize echo_info: derive BIP32 address outside DB lock, minimize lock duration
- Fix echo_stats SQL injection via parameter binding + add idx_stats_chain index
- New regression tests: ssrf_tests, db_path_validation, input_validation_tests
 2026-07-16 18:59:30 -04:00
svatantrya
431929d37c security: fix secret leakage hardcoded tokens and credentials (Punto 3) 
- .gitignore: Add protection for .env, *.pem, *.key, private_key.pem, privkey.pem, ec.key, chiave_privata.key, and shell scripts bal-*.sh
- make_release.sh: Remove hardcoded token 5cfa8c33e337ebaadb355c0ffa2d053d521ee43b
  Add loading from .env file with GITEA_API_TOKEN variable
  Add error handling if token is not set (prevents script from running without proper authentication)
- .env.example: Add template file for Gitea API token setup (not committed to git, .gitignored)
- generate_keys.sh: Add chmod 600 to protect private_key.pem permissions
- contrib/download_and_install_bal.sh: Remove hardcoded xpub and fixed_fee.
  Make all settings required as arguments or environment variables (xpub, fixed_fee, willexecutor_url, email, info)
  Add proper error handling and usage instructions if required arguments are not provided
- tests/secret_leakage_tests.rs: Add regression tests that:
  * Verify .gitignore protects .env, .pem, .key files
  * Verify no private key files are tracked in git (only public_key.pem is allowed)
  * Scan shell scripts for potential hardcoded tokens
- All tests pass: cargo test (8 tests: 3 SQL injection + 2 panic regression + 3 secret leakage)
- Build verified: cargo check (0 errors)
 2026-07-16 15:25:23 -04:00
svatantrya
abb882bd69 security: fix unwrap/expect/panic on untrusted input (Fase 3 MEDIUM/LOW) 
- xpub.rs: Replace convert_xpub() unwrap() with Result propagation
- xpub.rs: Replace calculate_fingerprint() unwrap() with Result propagation
- xpub.rs: Replace get_bitcoincore_descriptor() unwrap() with Result/match
- xpub.rs: Fix new_address_from_xpub() call in bal-server.rs to handle Result
- xpub.rs: Add prefix validation in convert_xpub (xpub/ypub/zpub/tpub/vpub/upub)
- bal-pusher.rs: Remove parse().unwrap() on env var bool, use unwrap_or(false)
- bal-pusher.rs: Replace port try_into().unwrap() with safe u16::try_from match
- bal-pusher.rs: Add error logging for invalid port values in env vars
- All tests pass: cargo test (5 tests: 3 SQL + 2 panic regression)
- Build verified: cargo check --bin=bal-server --bin=bal-pusher (0 errors)
 2026-07-16 15:08:49 -04:00
svatantrya
604ce3ebed security: eliminate unwrap/expect/panic on untrusted input paths (Fase 2 HIGH) 
- db.rs: Replace all stmt.read/bind().unwrap() with safe match/if let + error logging
- db.rs: Replace execute_insert() expect() with safe prepare + rollback on error
- db.rs: Replace get_total_transaction_number() unwrap() with safe match/Result propagation
- bal-server.rs: Replace fs::read_to_string().expect() with match + 500 error
- bal-server.rs: Replace idx/amount.try_into().unwrap() with i64::try_from(...).unwrap_or()
- bal-server.rs: echo_pub_key returns 500 instead of panic on file read failure
- docs/08_security_audit.md: Update status for panic/DoS vulnerabilities to 'Fixed'
- All tests pass: cargo test (5 tests: 3 SQL injection + 2 panic regression)
- Build verified: cargo check --bin=bal-server --bin=bal-pusher (0 errors)
 2026-07-16 14:56:25 -04:00
svatantrya
06bec057c7 security: fix panic on untrusted input (Phase 2 Critical) 
- Replace all sqlite::open().unwrap() per-request with shared Arc<Mutex<Connection>>
- Add Mutex poisoning recovery in all cfg.lock() and db.lock() calls
- Fix std::str::from_utf8().unwrap() with safe match → 400 Bad Request
- Fix timestamp_nanos_opt().unwrap() with safe match
- Fix panic on RPC client failure (bal-pusher): error log + sleep + retry
- Fix ZMQ socket connect with retry loop (bal-pusher)
- Add ZMQ_RCVTIMEO=5000 and match recv for graceful timeout (bal-pusher)
- Fix ZMQ subscribe error with match instead of unwrap (bal-pusher)
- Add unwrap_or for all tbl_stats row fields in echo_stats (prevent NULL panic)
- Add panic_regression_tests.rs: test mutex poisoning recovery and NULL unwrap_or

All tests pass: cargo test --test panic_regression_tests + sql_injection_tests
Build verified: cargo check --bin=bal-server --bin=bal-pusher
 2026-07-16 14:49:31 -04:00
svatantrya
9e2acb78ac refactor: remove bal-pusher-enhanced and update docs/env 
- Remove src/bin/bal-pusher-enhanced.rs (synchronous pusher variant)
- Remove bal-pusher.env and bal-pusher.sh from git tracking (now in .gitignore)
- Update all documentation to remove references to bal-pusher-enhanced:
  * 01_project_overview.md
  * 02_glossary_and_bitcoin_domain.md
  * 03_architecture_and_data_flow.md
  * 04_modules_detail.md
  * 05_api_reference.md (remove rawblock ZMQ section, update references)
  * 08_security_audit.md (remove references to bal-pusher-enhanced in DoS and ZMQ sections)
  * 09_references_and_links.md
- Build verified: cargo check passes for bal-pusher and bal-server binaries
 2026-07-16 14:32:59 -04:00
svatantrya
9f308680bb docs: add comprehensive knowledge base and security audit 
- Add docs/INDEX.md with navigable index and quick reference guides
- Add 9 knowledge base files covering project overview, Bitcoin domain,
  architecture, modules, API reference, database schema, deployment/security
- Update AGENTS.md with knowledge base reference and update policy
- Add tests/sql_injection_tests.rs with regression tests for SQL injection
- Fix SQL injection vulnerabilities in bal-pusher.rs:
  * Replace string-formatted UPDATE IN with loop + parameterized queries
  * Replace string-formatted UPDATE push_err with parameterized query
  * Add chain name validation in calculate_stats to prevent env var tampering
- Update .gitignore to exclude bal-pusher.env and bal-pusher.sh
 2026-07-16 14:11:18 -04:00
20 changed files with 978 additions and 735 deletions
Expand all files Collapse all files

58
.dockerignore Normal file
View File

@@ -0,0 +1,58 @@
# Git
.git
.gitignore
.gitsecret
# Build artifacts
target/
# Environment files (secrets)
*.env
.env
.env.*
!.env.example
# Private keys
*.pem
*.key
!public_key.pem
!data/public_key.pem
# Database files
*.db
*.db-shm
*.db-wal
# Documentation
docs/
*.md
!README.md
# Tests
tests/
# Scripts (local dev only)
bal-server.sh
bal-pusher.sh
download_bal_db.sh
sendtx.sh
lib.sh
contrib/
update
update_codebase.txt
# Service files
*.service
# IDE
*.swp
*.swo
.vscode/
.idea/
# Misc
Cargo.lock
generate_random_ascii.sh
test/
invalid_txs/
valid_txs/

2
.env.example
View File

@@ -3,7 +3,7 @@
# DO NOT commit the real token to git! # DO NOT commit the real token to git!
# This file is in .gitignore and should not be committed to git # This file is in .gitignore and should not be committed to git
GITEA_API_TOKEN=your_gitea_api_token_here GITEA_API_TOKEN=your_gitea_api_token_here
# Example: GITEA_API_TOKEN=5cfa8c33e337ebaadb355c0ffa2d053d521ee43b # Example: GITEA_API_TOKEN=***REMOVED***
# (replace with your actual token after revoking the old one) # (replace with your actual token after revoking the old one)
# === bal-server === # === bal-server ===

4
.gitignore vendored
View File

@@ -8,9 +8,6 @@
.env.production .env.production
.env.secret .env.secret
# Shell scripts that load env vars (contain secrets, local only)
bal-pusher.sh
bal-server.sh
# Private keys - NEVER commit to git # Private keys - NEVER commit to git
# Only public_key.pem should be tracked (if needed) # Only public_key.pem should be tracked (if needed)
@@ -39,3 +36,4 @@ Cargo.lock
!lib/ !lib/
!contrib/ !contrib/
!src/ !src/
make_release.sh

273
Cargo.lock generated
View File

@@ -325,12 +325,11 @@ dependencies = [
"byteorder", "byteorder",
"bytes", "bytes",
"chrono", "chrono",
"confy", "ed25519-dalek",
"env_logger", "env_logger",
"hex", "hex",
"hex-conservative 0.1.1", "hex-conservative 0.1.1",
"log", "log",
"openssl",
"regex", "regex",
"reqwest", "reqwest",
"serde", "serde",
@@ -364,6 +363,12 @@ version = "0.22.1"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "72b3254f16251a8381aa12e40e3c4d2f0199f8c6508fbecb9d91f575e0fbb8c6" checksum = "72b3254f16251a8381aa12e40e3c4d2f0199f8c6508fbecb9d91f575e0fbb8c6"
[[package]]
name = "base64ct"
version = "1.8.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2af50177e190e07a26ab74f8b1efbfe2ef87da2116221318cb1c2e82baf7de06"
[[package]] [[package]]
name = "bech32" name = "bech32"
version = "0.11.0" version = "0.11.0"
@@ -592,16 +597,10 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d3fd119d74b830634cea2a0f58bbd0d54540518a14397557951e79340abc28c0" checksum = "d3fd119d74b830634cea2a0f58bbd0d54540518a14397557951e79340abc28c0"
[[package]] [[package]]
name = "confy" name = "const-oid"
version = "0.6.1" version = "0.9.6"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "45b1f4c00870f07dc34adcac82bb6a72cc5aabca8536ba1797e01df51d2ce9a0" checksum = "c2459377285ad874054d797f3ccebf984978aa39129f6eafde5cdc8315b612f8"
dependencies = [
"directories",
"serde",
"thiserror",
"toml",
]
[[package]] [[package]]
name = "const-oid" name = "const-oid"
@@ -747,6 +746,33 @@ dependencies = [
"hybrid-array", "hybrid-array",
] ]
[[package]]
name = "curve25519-dalek"
version = "4.1.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "97fb8b7c4503de7d6ae7b42ab72a5a59857b4c937ec27a3d4539dba95b5ab2be"
dependencies = [
"cfg-if",
"cpufeatures 0.2.17",
"curve25519-dalek-derive",
"digest 0.10.7",
"fiat-crypto",
"rustc_version",
"subtle",
"zeroize",
]
[[package]]
name = "curve25519-dalek-derive"
version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f46882e17999c6cc590af592290432be3bce0428cb0d5f8b6715e4dc7b383eb3"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]] [[package]]
name = "dashmap" name = "dashmap"
version = "5.5.3" version = "5.5.3"
@@ -760,6 +786,17 @@ dependencies = [
"parking_lot_core", "parking_lot_core",
] ]
[[package]]
name = "der"
version = "0.7.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e7c1832837b905bbfb5101e07cc24c8deddf52f93225eee6ead5f4d63d53ddcb"
dependencies = [
"const-oid 0.9.6",
"pem-rfc7468",
"zeroize",
]
[[package]] [[package]]
name = "deranged" name = "deranged"
version = "0.5.8" version = "0.5.8"
@@ -809,7 +846,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f1dd6dbb5841937940781866fa1281a1ff7bd3bf827091440879f9994983d5c2" checksum = "f1dd6dbb5841937940781866fa1281a1ff7bd3bf827091440879f9994983d5c2"
dependencies = [ dependencies = [
"block-buffer 0.12.1", "block-buffer 0.12.1",
"const-oid", "const-oid 0.10.2",
"crypto-common 0.2.2", "crypto-common 0.2.2",
] ]
@@ -824,27 +861,6 @@ dependencies = [
"walkdir", "walkdir",
] ]
[[package]]
name = "directories"
version = "5.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9a49173b84e034382284f27f1af4dcbbd231ffa358c0fe316541a7337f376a35"
dependencies = [
"dirs-sys",
]
[[package]]
name = "dirs-sys"
version = "0.4.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "520f05a5cbd335fae5a99ff7a6ab8627577660ee5cfd6a94a6a929b52ff0321c"
dependencies = [
"libc",
"option-ext",
"redox_users",
"windows-sys 0.48.0",
]
[[package]] [[package]]
name = "displaydoc" name = "displaydoc"
version = "0.2.5" version = "0.2.5"
@@ -856,6 +872,30 @@ dependencies = [
"syn", "syn",
] ]
[[package]]
name = "ed25519"
version = "2.2.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "115531babc129696a58c64a4fef0a8bf9e9698629fb97e9e40767d235cfbcd53"
dependencies = [
"pkcs8",
"signature",
]
[[package]]
name = "ed25519-dalek"
version = "2.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "70e796c081cee67dc755e1a36a0a172b897fab85fc3f6bc48307991f64e4eca9"
dependencies = [
"curve25519-dalek",
"ed25519",
"serde",
"sha2",
"subtle",
"zeroize",
]
[[package]] [[package]]
name = "either" name = "either"
version = "1.15.0" version = "1.15.0"
@@ -916,6 +956,12 @@ version = "2.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "37909eebbb50d72f9059c3b6d82c0463f2ff062c9e95845c43a6c9c0355411be" checksum = "37909eebbb50d72f9059c3b6d82c0463f2ff062c9e95845c43a6c9c0355411be"
[[package]]
name = "fiat-crypto"
version = "0.2.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "28dea519a9695b9977216879a3ebfddf92f1c08c05d984f8996aecd6ecdc811d"
[[package]] [[package]]
name = "find-msvc-tools" name = "find-msvc-tools"
version = "0.1.4" version = "0.1.4"
@@ -1578,16 +1624,6 @@ version = "0.2.186"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "68ab91017fe16c622486840e4c83c9a37afeff978bd239b5293d61ece587de66" checksum = "68ab91017fe16c622486840e4c83c9a37afeff978bd239b5293d61ece587de66"
[[package]]
name = "libredox"
version = "0.1.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c0ff37bd590ca25063e35af745c343cb7a0271906fb7b37e4813e8f79f00268d"
dependencies = [
"bitflags 2.9.0",
"libc",
]
[[package]] [[package]]
name = "linux-raw-sys" name = "linux-raw-sys"
version = "0.11.0" version = "0.11.0"
@@ -1759,15 +1795,6 @@ version = "0.1.6"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d05e27ee213611ffe7d6348b942e8f942b37114c00cc03cec254295a4a17852e" checksum = "d05e27ee213611ffe7d6348b942e8f942b37114c00cc03cec254295a4a17852e"
[[package]]
name = "openssl-src"
version = "300.5.4+3.5.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a507b3792995dae9b0df8a1c1e3771e8418b7c2d9f0baeba32e6fe8b06c7cb72"
dependencies = [
"cc",
]
[[package]] [[package]]
name = "openssl-sys" name = "openssl-sys"
version = "0.9.110" version = "0.9.110"
@@ -1776,17 +1803,10 @@ checksum = "0a9f0075ba3c21b09f8e8b2026584b1d18d49388648f2fbbf3c97ea8deced8e2"
dependencies = [ dependencies = [
"cc", "cc",
"libc", "libc",
"openssl-src",
"pkg-config", "pkg-config",
"vcpkg", "vcpkg",
] ]
[[package]]
name = "option-ext"
version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "04744f49eae99ab78e0d5c0b603ab218f515ea8cfe5a456d7629ad883a3b6e7d"
[[package]] [[package]]
name = "parking_lot" name = "parking_lot"
version = "0.12.5" version = "0.12.5"
@@ -1810,6 +1830,15 @@ dependencies = [
"windows-link 0.2.1", "windows-link 0.2.1",
] ]
[[package]]
name = "pem-rfc7468"
version = "0.7.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "88b39c9bfcfc231068454382784bb460aae594343fb030d46e9f50a645418412"
dependencies = [
"base64ct",
]
[[package]] [[package]]
name = "percent-encoding" name = "percent-encoding"
version = "2.3.2" version = "2.3.2"
@@ -1828,6 +1857,16 @@ version = "0.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8b870d8c151b6f2fb93e84a13146138f05d02ed11c7e7c54f8826aaaf7c9f184" checksum = "8b870d8c151b6f2fb93e84a13146138f05d02ed11c7e7c54f8826aaaf7c9f184"
[[package]]
name = "pkcs8"
version = "0.10.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f950b2377845cebe5cf8b5165cb3cc1a5e0fa5cfa3e1f7f55707d8fd82e0a7b7"
dependencies = [
"der",
"spki",
]
[[package]] [[package]]
name = "pkg-config" name = "pkg-config"
version = "0.3.30" version = "0.3.30"
@@ -1994,17 +2033,6 @@ dependencies = [
"bitflags 2.9.0", "bitflags 2.9.0",
] ]
[[package]]
name = "redox_users"
version = "0.4.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bd283d9651eeda4b2a83a43c1c91b266c40fd76ecd39a50a8c630ae69dc72891"
dependencies = [
"getrandom 0.2.14",
"libredox",
"thiserror",
]
[[package]] [[package]]
name = "regex" name = "regex"
version = "1.10.4" version = "1.10.4"
@@ -2326,6 +2354,15 @@ dependencies = [
"libc", "libc",
] ]
[[package]]
name = "signature"
version = "2.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "77549399552de45a898a580c1b41d445bf730df867cc44e6c0233bbc4b8329de"
dependencies = [
"rand_core 0.6.4",
]
[[package]] [[package]]
name = "simd-adler32" name = "simd-adler32"
version = "0.3.10" version = "0.3.10"
@@ -2373,6 +2410,16 @@ dependencies = [
"lock_api", "lock_api",
] ]
[[package]]
name = "spki"
version = "0.7.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d91ed6c858b01f942cd56b37a94b3e0a1798290327d1236e4d9cf4eaca44d29d"
dependencies = [
"base64ct",
"der",
]
[[package]] [[package]]
name = "sqlite" name = "sqlite"
version = "0.34.0" version = "0.34.0"
@@ -2499,26 +2546,6 @@ dependencies = [
"windows-sys 0.61.2", "windows-sys 0.61.2",
] ]
[[package]]
name = "thiserror"
version = "1.0.59"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f0126ad08bff79f29fc3ae6a55cc72352056dfff61e3ff8bb7129476d44b23aa"
dependencies = [
"thiserror-impl",
]
[[package]]
name = "thiserror-impl"
version = "1.0.59"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d1cd413b5d558b4c5bf3680e324a6fa5014e7b7c067a51e69dbdf47eb7148b66"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]] [[package]]
name = "time" name = "time"
version = "0.3.44" version = "0.3.44"
@@ -3032,15 +3059,6 @@ dependencies = [
"windows-link 0.1.1", "windows-link 0.1.1",
] ]
[[package]]
name = "windows-sys"
version = "0.48.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "677d2418bec65e3338edb076e806bc1ec15693c5d0104683f2efe857f61056a9"
dependencies = [
"windows-targets 0.48.5",
]
[[package]] [[package]]
name = "windows-sys" name = "windows-sys"
version = "0.52.0" version = "0.52.0"
@@ -3068,21 +3086,6 @@ dependencies = [
"windows-link 0.2.1", "windows-link 0.2.1",
] ]
[[package]]
name = "windows-targets"
version = "0.48.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9a2fa6e2155d7247be68c096456083145c183cbbbc2764150dda45a87197940c"
dependencies = [
"windows_aarch64_gnullvm 0.48.5",
"windows_aarch64_msvc 0.48.5",
"windows_i686_gnu 0.48.5",
"windows_i686_msvc 0.48.5",
"windows_x86_64_gnu 0.48.5",
"windows_x86_64_gnullvm 0.48.5",
"windows_x86_64_msvc 0.48.5",
]
[[package]] [[package]]
name = "windows-targets" name = "windows-targets"
version = "0.52.5" version = "0.52.5"
@@ -3116,12 +3119,6 @@ dependencies = [
"windows_x86_64_msvc 0.53.1", "windows_x86_64_msvc 0.53.1",
] ]
[[package]]
name = "windows_aarch64_gnullvm"
version = "0.48.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2b38e32f0abccf9987a4e3079dfb67dcd799fb61361e53e2882c3cbaf0d905d8"
[[package]] [[package]]
name = "windows_aarch64_gnullvm" name = "windows_aarch64_gnullvm"
version = "0.52.5" version = "0.52.5"
@@ -3134,12 +3131,6 @@ version = "0.53.1"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a9d8416fa8b42f5c947f8482c43e7d89e73a173cead56d044f6a56104a6d1b53" checksum = "a9d8416fa8b42f5c947f8482c43e7d89e73a173cead56d044f6a56104a6d1b53"
[[package]]
name = "windows_aarch64_msvc"
version = "0.48.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "dc35310971f3b2dbbf3f0690a219f40e2d9afcf64f9ab7cc1be722937c26b4bc"
[[package]] [[package]]
name = "windows_aarch64_msvc" name = "windows_aarch64_msvc"
version = "0.52.5" version = "0.52.5"
@@ -3152,12 +3143,6 @@ version = "0.53.1"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b9d782e804c2f632e395708e99a94275910eb9100b2114651e04744e9b125006" checksum = "b9d782e804c2f632e395708e99a94275910eb9100b2114651e04744e9b125006"
[[package]]
name = "windows_i686_gnu"
version = "0.48.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a75915e7def60c94dcef72200b9a8e58e5091744960da64ec734a6c6e9b3743e"
[[package]] [[package]]
name = "windows_i686_gnu" name = "windows_i686_gnu"
version = "0.52.5" version = "0.52.5"
@@ -3182,12 +3167,6 @@ version = "0.53.1"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fa7359d10048f68ab8b09fa71c3daccfb0e9b559aed648a8f95469c27057180c" checksum = "fa7359d10048f68ab8b09fa71c3daccfb0e9b559aed648a8f95469c27057180c"
[[package]]
name = "windows_i686_msvc"
version = "0.48.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8f55c233f70c4b27f66c523580f78f1004e8b5a8b659e05a4eb49d4166cca406"
[[package]] [[package]]
name = "windows_i686_msvc" name = "windows_i686_msvc"
version = "0.52.5" version = "0.52.5"
@@ -3200,12 +3179,6 @@ version = "0.53.1"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1e7ac75179f18232fe9c285163565a57ef8d3c89254a30685b57d83a38d326c2" checksum = "1e7ac75179f18232fe9c285163565a57ef8d3c89254a30685b57d83a38d326c2"
[[package]]
name = "windows_x86_64_gnu"
version = "0.48.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "53d40abd2583d23e4718fddf1ebec84dbff8381c07cae67ff7768bbf19c6718e"
[[package]] [[package]]
name = "windows_x86_64_gnu" name = "windows_x86_64_gnu"
version = "0.52.5" version = "0.52.5"
@@ -3218,12 +3191,6 @@ version = "0.53.1"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9c3842cdd74a865a8066ab39c8a7a473c0778a3f29370b5fd6b4b9aa7df4a499" checksum = "9c3842cdd74a865a8066ab39c8a7a473c0778a3f29370b5fd6b4b9aa7df4a499"
[[package]]
name = "windows_x86_64_gnullvm"
version = "0.48.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0b7b52767868a23d5bab768e390dc5f5c55825b6d30b86c844ff2dc7414044cc"
[[package]] [[package]]
name = "windows_x86_64_gnullvm" name = "windows_x86_64_gnullvm"
version = "0.52.5" version = "0.52.5"
@@ -3236,12 +3203,6 @@ version = "0.53.1"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0ffa179e2d07eee8ad8f57493436566c7cc30ac536a3379fdf008f47f6bb7ae1" checksum = "0ffa179e2d07eee8ad8f57493436566c7cc30ac536a3379fdf008f47f6bb7ae1"
[[package]]
name = "windows_x86_64_msvc"
version = "0.48.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ed94fce61571a4006852b7389a063ab983c02eb1bb37b47f8272ce92d06d9538"
[[package]] [[package]]
name = "windows_x86_64_msvc" name = "windows_x86_64_msvc"
version = "0.52.5" version = "0.52.5"

44
Cargo.toml
View File

@@ -4,39 +4,55 @@ version = "0.3.0"
edition = "2024" edition = "2024"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html # See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[features]
default = ["server", "pusher"]
server = ["dep:actix-web", "dep:actix-governor", "dep:actix-rt", "dep:chrono", "dep:hex-conservative"]
pusher = ["dep:zmq", "dep:reqwest", "dep:byteorder", "dep:base64", "dep:ed25519-dalek"]
[dependencies] [dependencies]
base64 = { version = "0.22.1" }
bs58 = { version = "0.4.0" } bs58 = { version = "0.4.0" }
bytes = { version = "1.2" } bytes = { version = "1.2" }
bitcoin = { version = "0.32.5" } bitcoin = { version = "0.32.5" }
bitcoincore-rpc = { version = "0.19.0" } bitcoincore-rpc = { version = "0.19.0" }
bitcoincore-rpc-json = { version = "0.19.0" } bitcoincore-rpc-json = { version = "0.19.0" }
byteorder = { version = "1.5.0" }
confy = { version = "0.6.1" }
chrono = { version = "0.4.40" }
env_logger = { version = "0.11.5" } env_logger = { version = "0.11.5" }
hex = { version = "0.4.3" } hex = { version = "0.4.3" }
hex-conservative = { version = "0.1.1" }
actix-web = { version = "4.9.0" }
actix-governor = { version = "0.6.0" }
log = { version = "0.4.21" } log = { version = "0.4.21" }
openssl = { version = "0.10.74", features = ["vendored"] }
sha2 = { version = "0.10.8" }
serde = { version = "1.0.152", features = ["derive"] } serde = { version = "1.0.152", features = ["derive"] }
serde_json = { version = "1.0.116" } serde_json = { version = "1.0.116" }
sqlite = { version = "0.34.0" } sqlite = { version = "0.34.0" }
regex = { version = "1.10.4" } regex = { version = "1.10.4" }
reqwest = { version = "0.12.24", features = ["json","socks"] }
actix-rt = { version = "2.10.0" }
tokio = { version = "1", features = ["rt", "net","macros","rt-multi-thread"] } tokio = { version = "1", features = ["rt", "net","macros","rt-multi-thread"] }
url = { version = "2" } url = { version = "2" }
zmq = { version = "0.10.0" }
# server-only
actix-web = { version = "4.9.0", optional = true }
actix-governor = { version = "0.6.0", optional = true }
actix-rt = { version = "2.10.0", optional = true }
chrono = { version = "0.4.40", optional = true }
hex-conservative = { version = "0.1.1", optional = true }
# pusher-only
zmq = { version = "0.10.0", optional = true }
reqwest = { version = "0.12.24", features = ["json","socks"], optional = true }
byteorder = { version = "1.5.0", optional = true }
base64 = { version = "0.22.1", optional = true }
ed25519-dalek = { version = "2", features = ["pem", "pkcs8"], optional = true }
sha2 = { version = "0.10.8" }
[profile.release]
opt-level = "z"
lto = true
codegen-units = 1
strip = true
panic = "abort"
[[bin]] [[bin]]
name = "bal-server" name = "bal-server"
path = "src/bin/bal-server-actix.rs" path = "src/bin/bal-server.rs"
required-features = ["server"]
[[bin]] [[bin]]
name = "bal-pusher" name = "bal-pusher"
path = "src/bin/bal-pusher.rs" path = "src/bin/bal-pusher.rs"
required-features = ["pusher"]

92
Dockerfile Normal file
View File

@@ -0,0 +1,92 @@
# =============================================================================
# Multi-stage Dockerfile for bal-server + bal-pusher
# Security: non-root user, minimal runtime, tini as PID 1
# =============================================================================
# ---------------------------------------------------------------------------
# Stage 1: Builder
# ---------------------------------------------------------------------------
FROM rust:1.95-bookworm AS builder
RUN apt-get update && apt-get install -y --no-install-recommends \
pkg-config \
libssl-dev \
libsodium-dev \
libzmq5-dev \
cmake \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /build
# Cache dependencies: copy Cargo.toml first, create dummy src to build deps
COPY Cargo.toml Cargo.lock* ./
RUN mkdir -p src/bin && \
echo 'fn main() {}' > src/bin/bal-server.rs && \
echo 'fn main() {}' > src/bin/bal-pusher.rs && \
echo '' > src/lib.rs && \
echo '' > src/db.rs && \
echo '' > src/xpub.rs && \
echo '' > src/validation.rs && \
cargo build --release --bin bal-server --bin bal-pusher 2>/dev/null || true && \
rm -rf src target/release/.fingerprint target/release/deps/*bal_server*
# Copy real source and build
COPY src/ src/
RUN cargo build --release --bin bal-server --bin bal-pusher && \
strip target/release/bal-server target/release/bal-pusher
# ---------------------------------------------------------------------------
# Stage 2: Runtime
# ---------------------------------------------------------------------------
FROM debian:bookworm-slim AS runtime
# Install runtime dependencies + tini for PID 1
RUN apt-get update && apt-get install -y --no-install-recommends \
libssl3 \
libsodium23 \
libzmq5 \
libsqlite3-0 \
ca-certificates \
curl \
tini \
&& rm -rf /var/lib/apt/lists/* \
&& apt-get clean
# Copy binaries from builder
COPY --from=builder /build/target/release/bal-server /usr/local/bin/bal-server
COPY --from=builder /build/target/release/bal-pusher /usr/local/bin/bal-pusher
# Create dedicated non-root user
RUN groupadd -g 1000 bal && \
useradd -u 1000 -g bal -s /usr/sbin/nologin -M bal && \
mkdir -p /var/bal /var/bal/.bitcoin && \
chown -R bal:bal /var/bal && \
chmod 700 /var/bal
# Copy entrypoint
COPY docker/entrypoint.sh /usr/local/bin/entrypoint.sh
RUN chmod +x /usr/local/bin/entrypoint.sh
# Use tini as PID 1 for proper signal handling
ENTRYPOINT ["/usr/bin/tini", "--"]
CMD ["/usr/local/bin/entrypoint.sh"]
# Data directory (mount as volume)
VOLUME ["/var/bal"]
# bal-server port (bind to 127.0.0.1 via env, expose for reverse proxy)
EXPOSE 9137
# Default environment (override at runtime)
ENV RUST_LOG=info \
BAL_SERVER_BIND_ADDRESS=127.0.0.1 \
BAL_SERVER_BIND_PORT=9137 \
BAL_SERVER_DB_FILE=/var/bal/bal.db \
BAL_PUSHER_DB_FILE=/var/bal/bal.db \
BAL_SERVER_URL=http://127.0.0.1:9137 \
BAL_SERVER_PUB_KEY_PATH=/var/bal/public_key.pem \
SSL_KEY_PATH=/var/bal/private_key.pem
# Health check: verify bal-server is responding
HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 \
CMD curl -sf http://127.0.0.1:9137/ || exit 1

196
README.md
View File

@@ -1,85 +1,159 @@
# bal-server # bal-server
## Installation ## Installation
```bash ```bash
$ git clone .... git clone https://bitcoin-after.life/gitea/bitcoinafterlife/bal-server.git
$ cd bal-server cd bal-server
$ openssl genpkey -algorithm ED25519 -out private_key.pem openssl genpkey -algorithm ED25519 -out private_key.pem
$ openssl pkey -in private_key.pem -pubout -out public_key.pem openssl pkey -in private_key.pem -pubout -out public_key.pem
$ cargo build --release cargo build --release
$ sudo cp target/release/bal-server /usr/local/bin sudo cp target/release/bal-server target/release/bal-pusher /usr/local/bin
$ bal-server
``` ```
## Configuration ## Docker
The `bal-server` application can be configured using environment variables. The following variables are available: ### Build
```bash
docker build -t bal-server .
```
### Run
```bash
docker run -d \
--name bal-server \
--network host \
--tmpfs /tmp:rw,noexec,nosuid \
-v /path/to/data:/var/bal:rw \
-v /path/to/.bitcoin/regtest/.cookie:/var/bal/.bitcoin/regtest/.cookie:ro \
-e BAL_SERVER_REGTEST_ADDRESS="your_xpub_or_address" \
-e BAL_SERVER_REGTEST_FIXED_FEE=50000 \
-e BAL_SERVER_INFO="BAL server" \
-e BAL_PUSHER_NETWORK=regtest \
-e BAL_PUSHER_REGTEST_ZMQ_HASHBLOCK=tcp://127.0.0.1:28332 \
-e BAL_PUSHER_REGTEST_COOKIE_FILE=/var/bal/.bitcoin/regtest/.cookie \
bal-server
```
### Docker environment variables
| Variable | Description | Default | | Variable | Description | Default |
| --- | --- | --- | | --- | --- | --- |
| `BAL_SERVER_CONFIG_FILE` | Path to the configuration file. If the file does not exist, a new one will be created. | `$HOME/.config/bal-server/default-config.toml` | | `BAL_PUSHER_NETWORK` | Network to run pusher on (`bitcoin`, `testnet`, `testnet4`, `signet`, `regtest`). | `bitcoin` |
| `BAL_SERVER_DB_FILE` | Path to the SQLite3 database file. If the file does not exist, a new one will be created. | `bal.db` | | `BAL_PUSHER_REGTEST_ZMQ_HASHBLOCK` | ZMQ endpoint for regtest blocks. | `tcp://127.0.0.1:21332` |
| `BAL_SERVER_BIND_ADDRESS` | Public address for listening to requests. | `127.0.0.1` | | `BAL_PUSHER_REGTEST_COOKIE_FILE` | Absolute path to Bitcoin Core cookie file inside the container. | - |
| `BAL_SERVER_BIND_PORT` | Default port for listening to requests. | `9137` |
| `BAL_SERVER_PUB_KEY_PATH` | WillExecutor Ed25519 public key | `public_key.pem` |
| `BAL_SERVER_REGTEST_ADDRESS` | Bitcoin address for the regtest environment. | - |
| `BAL_SERVER_REGTEST_FIXED_FEE` | Fixed fee for the regtest environment. | 50000 |
| `BAL_SERVER_SIGNET_ADDRESS` | Bitcoin address for the signet environment. | - |
| `BAL_SERVER_SIGNET_FIXED_FEE` | Fixed fee for the signet environment. | 50000 |
| `BAL_SERVER_TESTNET_ADDRESS` | Bitcoin address for the testnet environment. | - |
| `BAL_SERVER_TESTNET_FIXED_FEE` | Fixed fee for the testnet environment. | 50000 |
| `BAL_SERVER_BITCOIN_ADDRESS` | Bitcoin address for the mainnet environment. | - |
| `BAL_SERVER_BITCOIN_FIXED_FEE` | Fixed fee for the mainnet environment. | 50000 |
> **Note:** The container runs as a non-root `bal` user (uid 1000) with `tini` as PID 1.
> The `/var/bal` volume stores the database. Mount Bitcoin Core's cookie file as read-only.
> When using `--network host`, ensure only `127.0.0.1` is used for internal services.
## Configuration (bal-server)
The `bal-server` application can be configured using environment variables.
### General
| Variable | Description | Default |
| --- | --- | --- |
| `BAL_SERVER_DB_FILE` | Path to the SQLite3 database file. | `bal.db` |
| `BAL_SERVER_BIND_ADDRESS` | Address to listen on. **Never bind to `0.0.0.0` in production without a reverse proxy.** | `127.0.0.1` |
| `BAL_SERVER_BIND_PORT` | Port to listen on. | `9137` |
| `BAL_SERVER_INFO` | Server info string returned by the `/` endpoint. | - |
| `BAL_SERVER_PUB_KEY_PATH` | Ed25519 public key for signature verification. | `public_key.pem` |
| `BAL_SERVER_URL` | Public URL of this server (used for stats reporting). | - |
| `SSL_KEY_PATH` | Ed25519 private key for signing stats reports. | `private_key.pem` |
| `RUST_LOG` | Log level (`error`, `warn`, `info`, `debug`, `trace`). | `info` |
### Per-network addresses and fees
| Variable | Description | Default |
| --- | --- | --- |
| `BAL_SERVER_BITCOIN_ADDRESS` | xpub or address for mainnet. | - |
| `BAL_SERVER_BITCOIN_FIXED_FEE` | Fixed fee (satoshis) for mainnet. | `50000` |
| `BAL_SERVER_REGTEST_ADDRESS` | xpub or address for regtest. | - |
| `BAL_SERVER_REGTEST_FIXED_FEE` | Fixed fee (satoshis) for regtest. | `50000` |
| `BAL_SERVER_SIGNET_ADDRESS` | xpub or address for signet. | - |
| `BAL_SERVER_SIGNET_FIXED_FEE` | Fixed fee (satoshis) for signet. | `50000` |
| `BAL_SERVER_TESTNET_ADDRESS` | xpub or address for testnet. | - |
| `BAL_SERVER_TESTNET_FIXED_FEE` | Fixed fee (satoshis) for testnet. | `50000` |
| `BAL_SERVER_TESTNET4_ADDRESS` | xpub or address for testnet4. | - |
| `BAL_SERVER_TESTNET4_FIXED_FEE` | Fixed fee (satoshis) for testnet4. | `50000` |
### DoS protection (Actix Web)
| Variable | Description | Default |
| --- | --- | --- |
| `BAL_SERVER_ACTIX_MAX_BODY_SIZE` | Maximum request body size in bytes. | `1048576` (1 MB) |
| `BAL_SERVER_ACTIX_TIMEOUT_SECS` | Request timeout in seconds. | `5` |
| `BAL_SERVER_ACTIX_PUSHTXS_PER_SEC` | Rate limit: push txs requests per second. | `1` |
| `BAL_SERVER_ACTIX_PUSHTXS_BURST` | Rate limit: push txs burst size. | `3` |
| `BAL_SERVER_ACTIX_SEARCHTX_PER_SEC` | Rate limit: search tx requests per second. | `5` |
| `BAL_SERVER_ACTIX_SEARCHTX_BURST` | Rate limit: search tx burst size. | `10` |
| `BAL_SERVER_ACTIX_INFO_PER_SEC` | Rate limit: info requests per second. | `20` |
| `BAL_SERVER_ACTIX_INFO_BURST` | Rate limit: info burst size. | `30` |
| `BAL_SERVER_ACTIX_DEFAULT_PER_SEC` | Rate limit: default requests per second. | `50` |
| `BAL_SERVER_ACTIX_DEFAULT_BURST` | Rate limit: default burst size. | `100` |
| `BAL_SERVER_ACTIX_WORKERS` | Number of Actix worker threads. | `4` |
| `BAL_SERVER_ACTIX_MAX_CONNECTIONS` | Maximum concurrent connections. | `100` |
---
# bal-pusher # bal-pusher
`bal-pusher` is a tool that retrieves Bitcoin transactions from a database and pushes them to the Bitcoin network when their **locktime** exceeds the **median time past** (MTP). It listens for Bitcoin block updates via ZMQ. `bal-pusher` monitors Bitcoin blocks via ZMQ and pushes time-locked transactions from the database to the Bitcoin network when their **locktime** exceeds the **median time past** (MTP).
## Installation ## Prerequisites
To use `bal-pusher`, you need to compile and install Bitcoin with ZMQ (ZeroMQ) support enabled. Then, configure your Bitcoin node and `bal-pusher` to push the transactions.
### Prerequisites
1. **Bitcoin with ZMQ Support**:
Ensure that Bitcoin is compiled with ZMQ support. Add the following line to your `bitcoin.conf` file:
- **Bitcoin Core** with ZMQ support enabled. Add to `bitcoin.conf`:
``` ```
zmqpubhashblock=tcp://127.0.0.1:28332 zmqpubhashblock=tcp://127.0.0.1:28332
``` ```
- **Rust and Cargo**: [Rust Installation](https://www.rust-lang.org/tools/install)
- **Libraries**: `libssl-dev`, `libsodium-dev`, `libzmq5-dev`, `libsqlite3-dev`
2. **Install Rust and Cargo**: ## Running
If you haven't already installed Rust and Cargo, you can follow the official instructions to do so: [Rust Installation](https://www.rust-lang.org/tools/install).
## Configuration
`bal-pusher` can be configured using environment variables. If no configuration file is provided, a default configuration file will be created.
### Available Configuration Variables
| Variable | Description | Default |
|---------------------------------------|------------------------------------------|----------------------------------------------|
| `BAL_PUSHER_CONFIG_FILE` | Path to the configuration file. If the file does not exist, it will be created. | `$HOME/.config/bal-pusher/default-config.toml` |
| `BAL_PUSHER_DB_FILE` | Path to the SQLite3 database file. If the file does not exist, it will be created. | `bal.db` |
| `BAL_PUSHER_ZMQ_LISTENER` | ZMQ listener for Bitcoin updates. | `tcp://127.0.0.1:28332` |
| `BAL_PUSHER_BITCOIN_HOST` | Bitcoin server host for RPC connections. | `http://127.0.0.1` |
| `BAL_PUSHER_BITCOIN_PORT` | Bitcoin RPC server port. | `8332` |
| `BAL_PUSHER_BITCOIN_COOKIE_FILE` | Path to Bitcoin RPC cookie file. | `$HOME/.bitcoin/.cookie` |
| `BAL_PUSHER_BITCOIN_RPC_USER` | Bitcoin RPC username. | - |
| `BAL_PUSHER_BITCOIN_RPC_PASSWORD` | Bitcoin RPC password. | - |
| `BAL_PUSHER_SEND_STATS` | Contact welist to provide times | false |
| `WELIST_SERVER_URL` | welist server url to provide times | https://welist.bitcoin-afer.life |
| `BAL_SERVER_URL` | WillExecutor server url | - |
| `SSL_KEY_PATH` | Ed25519 private key pem file | `private_key.pem` |
## Running `bal-pusher`
Once the application is installed and configured, you can start `bal-pusher` by running the following command:
```bash ```bash
$ bal-pusher [bitcoin|testnet|regtest|] bal-pusher [bitcoin|testnet|testnet4|signet|regtest]
``` ```
This will start the service, which will listen for Bitcoin blocks via ZMQ and push transactions from the database when their locktime exceeds the median time past. If no network is specified, defaults to `bitcoin`.
## Configuration (bal-pusher)
### General
| Variable | Description | Default |
| --- | --- | --- |
| `BAL_PUSHER_DB_FILE` | Path to the SQLite3 database file. | `bal.db` |
| `BAL_PUSHER_SEND_STATS` | Send stats to welist server. | `false` |
| `BAL_SERVER_URL` | URL of bal-server (for stats reporting). | - |
| `SSL_KEY_PATH` | Ed25519 private key for signing stats reports. | `private_key.pem` |
| `WELIST_SERVER_URL` | Welist server URL. | `https://welist.bitcoin-after.life` |
### Per-network configuration
Each network (`bitcoin`, `regtest`, `testnet`, `testnet4`, `signet`) supports the following variables.
Replace `{NETWORK}` with the uppercase network name (e.g., `REGTEST`, `BITCOIN`).
| Variable | Description | Default |
| --- | --- | --- |
| `BAL_PUSHER_{NETWORK}_ZMQ_HASHBLOCK` | ZMQ endpoint for block notifications. | `tcp://127.0.0.1:28332` (mainnet) |
| `BAL_PUSHER_{NETWORK}_COOKIE_FILE` | Absolute path to Bitcoin Core cookie file. | `$HOME/.bitcoin/{dir}/.cookie` |
| `BAL_PUSHER_{NETWORK}_RPC_USER` | Bitcoin Core RPC username (alternative to cookie auth). | - |
| `BAL_PUSHER_{NETWORK}_RPC_PASSWORD` | Bitcoin Core RPC password. | - |
| `BAL_PUSHER_{NETWORK}_HOST` | Bitcoin Core RPC host. | `http://127.0.0.1` |
| `BAL_PUSHER_{NETWORK}_PORT` | Bitcoin Core RPC port. | `8332` (mainnet) |
| `BAL_PUSHER_{NETWORK}_DIR_PATH` | Bitcoin Core data directory subfolder. | `` (mainnet) |
Default ZMQ ports per network:
| Network | ZMQ Port | RPC Port |
| --- | --- | --- |
| `bitcoin` | 28332 | 8332 |
| `regtest` | 21332 | 18443 |
| `testnet` | 23332 | 18332 |
| `testnet4` | 22332 | 48332 |
| `signet` | 24332 | 38332 |

15
bal-pusher.sh Normal file
View File

@@ -0,0 +1,15 @@
export RUST_LOG=trace
export BAL_PUSHER_DB_FILE="$(pwd)/bal.db"
#export BAL_PUSHER_BITCOIN_COOKIE_FILE=/~/.bitcoin/.cookie
#export BAL_PUSHER_REGTEST_COOKIE_FILE=/~/.bitcoin/regtest/.cookie
#export BAL_PUSHER_TESTNET_COOKIE_FILE=/~/.bitcoin/testnet3/.cookie
#export BAL_PUSHER_SIGNET_COOKIE_FILE=/~/.bitcoin/signet/.cookie
export BAL_PUSHER_REGTEST_ZMQ_HASHBLOCK=tcp://127.0.0.1:21332
export BAL_PUSHER_SEND_STATS=true
export WELIST_SERVER_URL=http://localhost:8086
export WELIST_SKIP_URL_VALIDATION=true
export BAL_SERVER_URL="http://127.0.0.1:9133"
export SSL_KEY_PATH="$(pwd)/private_key.pem"
cargo run --bin=bal-pusher regtest --features=pusher

25
docker/entrypoint.sh Normal file
View File

@@ -0,0 +1,25 @@
#!/bin/sh
set -e
mkdir -p /var/bal/.bitcoin
chown bal:bal /var/bal /var/bal/.bitcoin 2>/dev/null || true
PUSHER_NETWORK="${BAL_PUSHER_NETWORK:-bitcoin}"
echo "[entrypoint] Starting bal-server on ${BAL_SERVER_BIND_ADDRESS:-127.0.0.1}:${BAL_SERVER_BIND_PORT:-9137}"
echo "[entrypoint] Starting bal-pusher (network: ${PUSHER_NETWORK})"
su -s /bin/sh bal -c '/usr/local/bin/bal-server' &
SERVER_PID=$!
su -s /bin/sh bal -c "/usr/local/bin/bal-pusher ${PUSHER_NETWORK}" &
PUSHER_PID=$!
cleanup() {
echo "[entrypoint] Shutting down..."
kill $SERVER_PID $PUSHER_PID 2>/dev/null
wait
}
trap cleanup TERM INT
wait

11
docs/07_deployment_and_ops.md
View File

@@ -45,6 +45,7 @@ WELIST_URL=https://welist.example.com/api/stats
- `BAL_SSL_KEY_PATH`: The path to the Ed25519 private key (`private_key.pem`) used to sign the statistics payload before sending it to the `welist` server. This is a critical secret. - `BAL_SSL_KEY_PATH`: The path to the Ed25519 private key (`private_key.pem`) used to sign the statistics payload before sending it to the `welist` server. This is a critical secret.
- `SEND_STATS`: A boolean flag to enable the reporting of statistics to the remote `welist` server. - `SEND_STATS`: A boolean flag to enable the reporting of statistics to the remote `welist` server.
- `WELIST_URL`: The URL to which the statistics are sent. If `SEND_STATS` is `true`, this URL must be reachable. If the server is unreachable, the pusher will log an error but might not crash (see `08_security_audit.md` for DoS analysis). - `WELIST_URL`: The URL to which the statistics are sent. If `SEND_STATS` is `true`, this URL must be reachable. If the server is unreachable, the pusher will log an error but might not crash (see `08_security_audit.md` for DoS analysis).
- `BAL_PUSHER_PREFER_IPV6`: Optional boolean flag (default `false`). When set to `true`, the pusher resolves the `welist` host itself and pins the HTTP connection to its first IPv6 (AAAA) address, still using the hostname for the `Host` header and TLS SNI. This works around networks where the IPv4 route to the `welist` host is broken while IPv6 works — the default connector may otherwise pick the unreachable family and the request would stall. Leave unset unless you hit this specific connectivity problem.
--- ---
@@ -128,11 +129,19 @@ bitcoin-cli -regtest gettransaction ... | bitcoin-cli -regtest sendrawtransactio
- It is a helper script that wraps `bitcoin-cli` to send a pre-created transaction, get the raw bytes, and send them to a new address. It is only useful for manual testing and integration checks. - It is a helper script that wraps `bitcoin-cli` to send a pre-created transaction, get the raw bytes, and send them to a new address. It is only useful for manual testing and integration checks.
### `make_release.sh` (Release Builder) ### `make_release.sh` (Release Builder)
This script builds a release binary, creates a Git tag, and uploads the release to a Git server (Gitea). It also hardcodes a Gitea API token (`TOKEN="5cfa8c33e337ebaadb355c0ffa2d053d521ee43b"`), which is a major security risk. This script builds a release binary, creates a Git tag, and uploads the release to a Git server (Gitea). It also hardcodes a Gitea API token (`TOKEN="***REMOVED***"`), which is a major security risk.
```bash ```bash
# WARNING: This script contains a hardcoded secret token. Do not use it as-is for production. # WARNING: This script contains a hardcoded secret token. Do not use it as-is for production.
``` ```
- **Release Assets:** It generates a `.tar.gz` archive with the binaries, a `.sha256` checksum file, and both a `.sig` GPG detached binary signature and a `.asc` ASCII-armored version.
- **Signature:** The release tarball is signed with the GPG key `Svātantrya <svatantrya@bitcoin-after.life>`. The script verifies that `gpg`, `sha256sum`, and `jq` are installed before proceeding.
- **Verification:** The release body includes instructions for verifying the checksum and signature (binary or ASCII-armored):
```bash
sha256sum -c <release>.tar.gz.sha256
gpg --verify <release>.tar.gz.sig <release>.tar.gz
gpg --verify <release>.tar.gz.asc <release>.tar.gz
```
- **Security:** It also builds and uploads the binaries. The binaries should be built and signed on a separate, clean build machine, not on the production server. - **Security:** It also builds and uploads the binaries. The binaries should be built and signed on a separate, clean build machine, not on the production server.
### `download_bal_db.sh` (Database Pull Script) ### `download_bal_db.sh` (Database Pull Script)

6
docs/08_security_audit.md
View File

@@ -56,7 +56,7 @@ Regression tests: `tests/panic_regression_tests.rs` (2 tests).
### 3. Secret Leakage (HIGH) ### 3. Secret Leakage (HIGH)
**Location:** `make_release.sh`, `contrib/download_and_install_bal.sh`, `private_key.pem`, `privkey.pem`, `ec.key`, `chiave_privata.key`. **Location:** `make_release.sh`, `contrib/download_and_install_bal.sh`, `private_key.pem`, `privkey.pem`, `ec.key`, `chiave_privata.key`.
**Description:** **Description:**
- The `make_release.sh` script contains a hardcoded Gitea API token: `TOKEN="5cfa8c33e337ebaadb355c0ffa2d053d521ee43b"`. If the script is accidentally pushed to a public repository, it will be visible to everyone. - The `make_release.sh` script contains a hardcoded Gitea API token: `TOKEN="***REMOVED***"`. If the script is accidentally pushed to a public repository, it will be visible to everyone.
- The `contrib/download_and_install_bal.sh` script contains a hardcoded `xpub` address and a fixed fee. This is less critical but could be used for fingerprinting. - The `contrib/download_and_install_bal.sh` script contains a hardcoded `xpub` address and a fixed fee. This is less critical but could be used for fingerprinting.
- The `private_key.pem` and `chiave_privata.key` files are stored in the project root (and in the repository). If the repository is public, the private key is compromised. An attacker could use this to sign fake statistics or forge authentication credentials. - The `private_key.pem` and `chiave_privata.key` files are stored in the project root (and in the repository). If the repository is public, the private key is compromised. An attacker could use this to sign fake statistics or forge authentication credentials.
**Impact:** An attacker could gain unauthorized access to the CI/CD pipeline, the release server, or the `welist` statistics service. **Impact:** An attacker could gain unauthorized access to the CI/CD pipeline, the release server, or the `welist` statistics service.
@@ -87,7 +87,7 @@ Regression tests: `tests/panic_regression_tests.rs` (2 tests).
- Rate limiting: `actix-governor` middleware with token-bucket — configurable via `BAL_SERVER_ACTIX_PUSHTXS_PER_SEC`/`BURST` (default 1 req/s per IP with burst 5) - Rate limiting: `actix-governor` middleware with token-bucket — configurable via `BAL_SERVER_ACTIX_PUSHTXS_PER_SEC`/`BURST` (default 1 req/s per IP with burst 5)
- Connection limits: `workers(4)` and `max_connections(100)` — configurable via `BAL_SERVER_ACTIX_WORKERS`/`MAX_CONNECTIONS` - Connection limits: `workers(4)` and `max_connections(100)` — configurable via `BAL_SERVER_ACTIX_WORKERS`/`MAX_CONNECTIONS`
- Body timeout: configurable via `BAL_SERVER_ACTIX_TIMEOUT_SECS` (default 30s) - Body timeout: configurable via `BAL_SERVER_ACTIX_TIMEOUT_SECS` (default 30s)
**Migration:** Server replaced `hyper` custom server with `actix-web` (see `src/bin/bal-server-actix.rs`). All handlers migrated with `Arc<Mutex<Connection>>` shared DB. Old `bal-server.rs` (Hyper) removed. `bal-pusher` enhanced with ZMQ timeout (`ZMQ_RCVTIMEO` 5000ms) and RPC retry logic. **Priority:** High. (Mitigated) **Migration:** Server replaced `hyper` custom server with `actix-web` (see `src/bin/bal-server.rs`). All handlers migrated with `Arc<Mutex<Connection>>` shared DB. Old `bal-server.rs` (Hyper) removed. `bal-pusher` enhanced with ZMQ timeout (`ZMQ_RCVTIMEO` 5000ms) and RPC retry logic. **Priority:** High. (Mitigated)
### 5. SSRF / Network Abuse via `reqwest` (MEDIUM) ### 5. SSRF / Network Abuse via `reqwest` (MEDIUM)
**Location:** `src/bin/bal-pusher.rs`. **Location:** `src/bin/bal-pusher.rs`.
@@ -121,7 +121,7 @@ Regression tests: `tests/panic_regression_tests.rs` (2 tests).
- Rejects symlinks and non-regular files (directories, devices, etc.). - Rejects symlinks and non-regular files (directories, devices, etc.).
- If validation fails, the function returns `Err(String)` instead of panicking, preventing crashes or accidental access to system files. - If validation fails, the function returns `Err(String)` instead of panicking, preventing crashes or accidental access to system files.
- ✅ **WAL mode:** `db::open_db` automatically executes `PRAGMA journal_mode = WAL;` and `PRAGMA synchronous = NORMAL;` on every connection. This is a best practice for safe concurrent access when `bal-server` and `bal-pusher` share the same database file. - ✅ **WAL mode:** `db::open_db` automatically executes `PRAGMA journal_mode = WAL;` and `PRAGMA synchronous = NORMAL;` on every connection. This is a best practice for safe concurrent access when `bal-server` and `bal-pusher` share the same database file.
- ✅ **Replaced `unwrap`:** In `src/bin/bal-server-actix.rs` and `src/bin/bal-pusher.rs`, `sqlite::open(...).unwrap()` was replaced with `db::open_db(...)` with safe error handling (return `Err` in the server, `std::process::exit(1)` in the pusher with a log error). - ✅ **Replaced `unwrap`:** In `src/bin/bal-server.rs` and `src/bin/bal-pusher.rs`, `sqlite::open(...).unwrap()` was replaced with `db::open_db(...)` with safe error handling (return `Err` in the server, `std::process::exit(1)` in the pusher with a log error).
- **Remaining (ops):** Ensure the database file is owned by the `bal` user and not writable by any other user (`chmod 600`). The database file should not reside on a shared or network drive. - **Remaining (ops):** Ensure the database file is owned by the `bal` user and not writable by any other user (`chmod 600`). The database file should not reside on a shared or network drive.
**Regression tests:** `tests/db_path_validation.rs` (5 tests covering traversal, forbidden absolute paths, symlink, WAL pragma, and valid relative paths). All passing. **Regression tests:** `tests/db_path_validation.rs` (5 tests covering traversal, forbidden absolute paths, symlink, WAL pragma, and valid relative paths). All passing.
**Status:** Fixed. **Priority:** Medium. **Status:** Fixed. **Priority:** Medium.

154
make_release.sh
View File

@@ -1,154 +0,0 @@
#!/bin/bash
#author: <your-name>
source lib.sh
usage() {
echo_w "./make_release <version> <message>"
}
if [ -n "$1" ]; then release=$1; else usage; exit; fi
if [ -n "$2" ]; then message=$2; else
# Create temporary file using mktemp
TEMPFILE=$(mktemp)
vi $TEMPFILE
message=$(cat $TEMPFILE)
rm $TEMPFILE
fi
echo_i $message
# Load secrets from .env file (not committed to git)
if [ -f .env ]; then
export $(grep -v '^#' .env | xargs)
fi
TOKEN="${GITEA_API_TOKEN}"
if [ -z "$TOKEN" ]; then
echo_e "Error: GITEA_API_TOKEN is not set in .env file."
echo_e "Please create a .env file with: GITEA_API_TOKEN=your_token_here"
exit 1
fi
OWNER="bitcoinafterlife"
basename=$(basename $(pwd))
REPO=$basename
TAG="v$release"
binpath="target/release/$basename"
release_name="$basename-$release"
dest="releases/$release"
arch=$(uname -m)
platform="linux-gnu"
destbin="$dest/$arch"
destsrc="$dest/src"
assetname="$release_name""_$arch""_$platform"
asset_tar_gz="$assetname.tar.gz"
ASSET_PATH="$destbin/$assetname.tar.gz"
giteahost="https://bitcoin-after.life/gitea"
url_releases="$giteahost/api/v1/repos/$OWNER/$REPO/releases"
echo_i() {
echo -e "\033[1m==> $1\033[0m"
}
echo_e() {
echo -e "\033[31;1m$1\033[0m"
}
echo_s() {
echo -e "\033[32;1m$1\033[0m"
}
echo_w() {
echo -e "\033[33;1m$1\033[0m"
}
prepare_release(){
mkdir -p "$destbin/$assetname"
if ! cargo build --release; then
echo_w "error building release"
exit 1
fi
ls -l $binpath
cp target/release/bal-server \
target/release/bal-pusher \
README.md \
"$destbin/$assetname"
(
cd "$destbin"
echo_w $ASSET_PATH
echo "ls $(pwd)"
ls
echo "ls $(pwd)/$assetname"
ls "$(pwd)/$assetname"
ls $assetname
tar -czf "$asset_tar_gz" "$assetname"
)
}
push_tag() {
git commit -am"release: $release_name"
git push
#git tag -a "$TAG" -m"release: $release_name"
#git push origin --tags
}
# Configurazioni
post_release() {
if [ -z "$1" ]; then
echo_e "no data to release"
exit 1
else
echo "data: $1"
fi
echo "token:$TOKEN"
echo url_releases: $url_releases
RELEASE="$(curl -s -X POST \
-H "accept: application/json" \
-H "Authorization: token $TOKEN" \
-H "Content-Type: application/json" \
-d "$1" \
$url_releases
)"
echo $RELEASE
}
add_asset_release() {
if [ -z "$1" ]; then
echo_e "error add_asset_release"
exit 1
fi
echo $ASSET_PATH
ls -l $ASSET_PATH
pwd
curl -X POST \
-H "accept: application/json" \
-H "Authorization: token $TOKEN" \
-H "Content-Type: multipart/form-data" \
-F "attachment=@$ASSET_PATH" \
"$url_releases/$1/assets"
}
# Estrae l'ID della release
release_data=$(cat <<EOF
{
"tag_name":"$TAG",
"name":"$release_name",
"body":"Release: $release_name enjoy\n$message"
}
EOF
)
prepare_release
echo_s "prepare release done"
push_tag
echo_s "push tag done"
echo "$release_data"
post_release "$release_data"
echo_s "prepare release done"
echo $RELEASE
id_release=
add_asset_release $(echo $RELEASE | jq .id)
echo_s "done"

327
src/bin/bal-pusher.rs
View File

@@ -6,7 +6,7 @@ use bitcoincore_rpc::{Auth, Client, Error, RpcApi, bitcoin};
use bitcoincore_rpc_json::GetBlockchainInfoResult; use bitcoincore_rpc_json::GetBlockchainInfoResult;
use byteorder::{LittleEndian, ReadBytesExt}; use byteorder::{LittleEndian, ReadBytesExt};
use hex; use ed25519_dalek::{Signer as _, SigningKey, pkcs8::DecodePrivateKey};
use log::{debug, error, info, trace, warn}; use log::{debug, error, info, trace, warn};
use serde::Deserialize; use serde::Deserialize;
use serde::Serialize; use serde::Serialize;
@@ -23,13 +23,10 @@ use zmq::{Context, DEALER, DONTWAIT, Socket};
use bal_server::db::open_db; use bal_server::db::open_db;
use bal_server::validation::is_valid_welist_url; use bal_server::validation::is_valid_welist_url;
use base64::{Engine as _, engine::general_purpose}; use base64::{Engine as _, engine::general_purpose};
use openssl::hash::MessageDigest;
use openssl::pkey::PKey;
use openssl::sign::Signer;
use openssl::sign::Verifier;
use reqwest::Client as rClient; use reqwest::Client as rClient;
use std::fs; use std::net::SocketAddr;
use std::time::Instant; use std::time::Instant;
use url::Url;
const LOCKTIME_THRESHOLD: i64 = 5000000; const LOCKTIME_THRESHOLD: i64 = 5000000;
const VERSION: &str = "0.0.2"; const VERSION: &str = "0.0.2";
@@ -143,7 +140,7 @@ fn get_network_params_default(network: Network) -> NetworkParams {
} }
fn get_cookie_filename(network: &NetworkParams) -> Result<String, Box<dyn StdError>> { fn get_cookie_filename(network: &NetworkParams) -> Result<String, Box<dyn StdError>> {
if network.cookie_file != "" { if !network.cookie_file.is_empty() {
Ok(network.cookie_file.clone()) Ok(network.cookie_file.clone())
} else { } else {
match env::var_os("HOME") { match env::var_os("HOME") {
@@ -161,12 +158,12 @@ fn get_cookie_filename(network: &NetworkParams) -> Result<String, Box<dyn StdErr
} }
} }
fn get_client_from_username( fn get_client_from_username(
url: &String, url: &str,
network: &NetworkParams, network: &NetworkParams,
) -> Result<(Client, GetBlockchainInfoResult), Box<dyn StdError>> { ) -> Result<(Client, GetBlockchainInfoResult), Box<dyn StdError>> {
if network.rpc_user != "" { if !network.rpc_user.is_empty() {
match Client::new( match Client::new(
&url[..], url,
Auth::UserPass(network.rpc_user.to_string(), network.rpc_pass.to_string()), Auth::UserPass(network.rpc_user.to_string(), network.rpc_pass.to_string()),
) { ) {
Ok(client) => match client.get_blockchain_info() { Ok(client) => match client.get_blockchain_info() {
@@ -180,30 +177,30 @@ fn get_client_from_username(
} }
} }
fn get_client_from_cookie( fn get_client_from_cookie(
url: &String, url: &str,
network: &NetworkParams, network: &NetworkParams,
) -> Result<(Client, GetBlockchainInfoResult), Box<dyn StdError>> { ) -> Result<(Client, GetBlockchainInfoResult), Box<dyn StdError>> {
match get_cookie_filename(network) { match get_cookie_filename(network) {
Ok(cookie) => match Client::new(&url[..], Auth::CookieFile(cookie.into())) { Ok(cookie) => match Client::new(url, Auth::CookieFile(cookie.into())) {
Ok(client) => match client.get_blockchain_info() { Ok(client) => match client.get_blockchain_info() {
Ok(bcinfo) => Ok((client, bcinfo)), Ok(bcinfo) => Ok((client, bcinfo)),
Err(err) => Err(err.into()), Err(err) => Err(err.into()),
}, },
Err(err) => Err(err.into()), Err(err) => Err(err.into()),
}, },
Err(err) => Err(err.into()), Err(err) => Err(err),
} }
} }
fn get_client( fn get_client(
network: &NetworkParams, network: &NetworkParams,
) -> Result<(Client, GetBlockchainInfoResult), Box<dyn StdError>> { ) -> Result<(Client, GetBlockchainInfoResult), Box<dyn StdError>> {
let url = format!("{}:{}/", network.host, &network.port); let url = format!("{}:{}/", network.host, network.port);
debug!("trying to connect to bitcoin daemon:{url}"); debug!("trying to connect to bitcoin daemon:{url}");
match get_client_from_username(&url, network) { match get_client_from_username(&url, network) {
Ok(client) => Ok(client), Ok(client) => Ok(client),
Err(_) => match get_client_from_cookie(&url, &network) { Err(_) => match get_client_from_cookie(&url, network) {
Ok(client) => Ok(client), Ok(client) => Ok(client),
Err(err) => Err(err.into()), Err(err) => Err(err),
}, },
} }
} }
@@ -247,7 +244,13 @@ async fn main_result(cfg: &MyConfig, network_params: &NetworkParams) -> Result<(
info!("db open {}", &cfg.db_file); info!("db open {}", &cfg.db_file);
let sqlquery = "SELECT * FROM tbl_tx WHERE network = :network AND status = :status AND ( locktime < :bestblock_height OR locktime > :locktime_threshold AND locktime < :bestblock_time);"; let sqlquery = "SELECT * FROM tbl_tx WHERE network = :network AND status = :status AND ( locktime < :bestblock_height OR locktime > :locktime_threshold AND locktime < :bestblock_time);";
let query_tx = db.prepare(sqlquery).unwrap().into_iter(); let query_tx = match db.prepare(sqlquery) {
Ok(q) => q.into_iter(),
Err(e) => {
warn!("tbl_tx not ready yet (tables may not exist): {}", e);
return Ok(());
}
};
trace!("query_tx: {}", sqlquery); trace!("query_tx: {}", sqlquery);
trace!(":locktime_threshold: {}", LOCKTIME_THRESHOLD); trace!(":locktime_threshold: {}", LOCKTIME_THRESHOLD);
trace!(":bestblock_time: {}", average_time); trace!(":bestblock_time: {}", average_time);
@@ -257,19 +260,28 @@ async fn main_result(cfg: &MyConfig, network_params: &NetworkParams) -> Result<(
//let query_tx = db.prepare("SELECT * FROM tbl_tx where status = :status").unwrap().into_iter(); //let query_tx = db.prepare("SELECT * FROM tbl_tx where status = :status").unwrap().into_iter();
let mut pushed_txs: Vec<String> = Vec::new(); let mut pushed_txs: Vec<String> = Vec::new();
let mut invalid_txs: std::collections::HashMap<String, String> = HashMap::new(); let mut invalid_txs: std::collections::HashMap<String, String> = HashMap::new();
for row in query_tx for row_result in match query_tx.bind::<&[(_, Value)]>(
.bind::<&[(_, Value)]>(
&[ &[
(":locktime_threshold", (LOCKTIME_THRESHOLD as i64).into()), (":locktime_threshold", LOCKTIME_THRESHOLD.into()),
(":bestblock_time", (average_time as i64).into()), (":bestblock_time", (average_time as i64).into()),
(":bestblock_height", (bcinfo.blocks as i64).into()), (":bestblock_height", (bcinfo.blocks as i64).into()),
(":network", network_params.db_field.clone().into()), (":network", network_params.db_field.clone().into()),
(":status", 0.into()), (":status", 0.into()),
][..], ][..],
) ) {
.unwrap() Ok(bound) => bound,
.map(|row| row.unwrap()) Err(e) => {
{ error!("Failed to bind query parameters: {}", e);
return Ok(());
}
} {
let row = match row_result {
Ok(r) => r,
Err(e) => {
warn!("Failed to read row: {}", e);
continue;
}
};
let tx = row.read::<&str, _>("tx"); let tx = row.read::<&str, _>("tx");
let txid = row.read::<&str, _>("txid"); let txid = row.read::<&str, _>("txid");
let locktime = row.read::<i64, _>("locktime"); let locktime = row.read::<i64, _>("locktime");
@@ -316,8 +328,12 @@ async fn main_result(cfg: &MyConfig, network_params: &NetworkParams) -> Result<(
stmt.bind((2, Value::String(txid.clone()))).unwrap(); stmt.bind((2, Value::String(txid.clone()))).unwrap();
let _ = stmt.next(); let _ = stmt.next();
} }
let _ = send_stats_report(cfg, bcinfo).await; if let Err(e) = send_stats_report(cfg, bcinfo).await {
let _ = calculate_stats(&db, network_params.db_field.clone()).await; error!("send_stats_report failed: {}", e);
}
if let Err(e) = calculate_stats(&db, network_params.db_field.clone()).await {
warn!("calculate_stats failed: {e}");
}
} }
Err(erx) => { Err(erx) => {
error!("impossible to get client: {}, retrying on next block", erx); error!("impossible to get client: {}, retrying on next block", erx);
@@ -407,6 +423,84 @@ ON CONFLICT(chain) DO UPDATE SET
} }
Ok(()) Ok(())
} }
/// Parse the `(host, port)` pair from a base URL like `https://host[:port]`.
///
/// Falls back to the scheme's well-known default port (443 for `https`,
/// 80 for plain `http`), or to 443 when the scheme is unknown.
fn parse_host_port(base_url: &str) -> Option<(String, u16)> {
let url = Url::parse(base_url).ok()?;
let host = url
.host_str()?
.trim_start_matches('[')
.trim_end_matches(']')
.to_string();
let port = url.port_or_known_default().unwrap_or(443);
Some((host, port))
}
/// Resolve `host:port` and return the first IPv6 (AAAA) address, if any.
///
/// Returns `None` when the host has no IPv6 address.
async fn resolve_first_ipv6(host: &str, port: u16) -> Option<SocketAddr> {
use std::net::ToSocketAddrs;
let host = host.to_string();
tokio::task::spawn_blocking(move || {
format!("{}:{}", host, port)
.to_socket_addrs()
.ok()
.and_then(|mut addrs| addrs.find(|a| a.is_ipv6()))
})
.await
.ok()
.flatten()
}
/// Build the HTTP client used for welist reports.
///
/// When `BAL_PUSHER_PREFER_IPV6` is truthy, the welist host is resolved and
/// the client is pinned to its first IPv6 address (the original hostname is
/// still used for the `Host` header and TLS SNI). This works around networks
/// where the IPv4 route to the welist host is broken while IPv6 works: the
/// default connector may otherwise pick the broken family and the request
/// stalls. When the variable is unset (the default), behavior is unchanged.
async fn welist_http_client(welist_url: &str) -> rClient {
let prefer_ipv6 = env::var("BAL_PUSHER_PREFER_IPV6")
.unwrap_or("false".to_string())
.parse::<bool>()
.unwrap_or(false);
if !prefer_ipv6 {
return new_welist_client();
}
let (host, port) = match parse_host_port(welist_url) {
Some(hp) => hp,
None => {
warn!("BAL_PUSHER_PREFER_IPV6: cannot parse '{welist_url}', using default resolver");
return new_welist_client();
}
};
match resolve_first_ipv6(&host, port).await {
Some(addr) => {
debug!("BAL_PUSHER_PREFER_IPV6: pinning {host} to {addr}");
rClient::builder()
.timeout(Duration::from_secs(10))
.resolve(&host, addr)
.build()
.unwrap_or_else(|_| new_welist_client())
}
None => {
debug!("BAL_PUSHER_PREFER_IPV6: no IPv6 address for {host}, using default resolver");
new_welist_client()
}
}
}
fn new_welist_client() -> rClient {
rClient::builder()
.timeout(Duration::from_secs(10))
.build()
.unwrap_or_else(|_| rClient::new())
}
async fn send_stats_report( async fn send_stats_report(
cfg: &MyConfig, cfg: &MyConfig,
bcinfo: GetBlockchainInfoResult, bcinfo: GetBlockchainInfoResult,
@@ -415,14 +509,18 @@ async fn send_stats_report(
debug!("sending report to welist"); debug!("sending report to welist");
let welist_url = env::var("WELIST_SERVER_URL") let welist_url = env::var("WELIST_SERVER_URL")
.unwrap_or("https://welist.bitcoin-after.life".to_string()); .unwrap_or("https://welist.bitcoin-after.life".to_string());
if !is_valid_welist_url(&welist_url) { let skip_validation = env::var("WELIST_SKIP_URL_VALIDATION")
.unwrap_or("false".to_string())
.parse::<bool>()
.unwrap_or(false);
if !skip_validation && !is_valid_welist_url(&welist_url) {
warn!( warn!(
"Invalid or unsafe WELIST_SERVER_URL: {}. Skipping stats report.", "Invalid or unsafe WELIST_SERVER_URL: {}. Skipping stats report.",
welist_url welist_url
); );
return Ok(()); return Ok(());
} }
let client = rClient::new(); let client = welist_http_client(&welist_url).await;
let url = format!("{}/ping", welist_url); let url = format!("{}/ping", welist_url);
debug!("welist url: {}", url); debug!("welist url: {}", url);
let chain = bcinfo.chain.to_string().to_lowercase(); let chain = bcinfo.chain.to_string().to_lowercase();
@@ -431,7 +529,7 @@ async fn send_stats_report(
cfg.url, chain, bcinfo.blocks, bcinfo.median_time, bcinfo.best_block_hash cfg.url, chain, bcinfo.blocks, bcinfo.median_time, bcinfo.best_block_hash
); );
trace!("message to be sent: {}", message); trace!("message to be sent: {}", message);
let sign = sign_message(cfg.ssl_key_path.as_str(), &message.as_str()); let sign = sign_message(cfg.ssl_key_path.as_str(), message.as_str());
let response = client let response = client
.post(url) .post(url)
.header("User-Agent", format!("bal-pusher/{}", VERSION)) .header("User-Agent", format!("bal-pusher/{}", VERSION))
@@ -446,32 +544,23 @@ async fn send_stats_report(
})) }))
.send() .send()
.await?; .await?;
if !response.status().is_success() { let status = response.status();
warn!( let body = response.text().await?;
"Non-success response: {} {}", info!(
response.status(), "Report to welist({}) status={} body={}",
response.status().canonical_reason().unwrap_or("") welist_url, status, body
); );
}
let body = &(response.text().await?);
info!("Report to welist({})\tSent: {}", welist_url, body);
} else { } else {
debug!("Not sending stats"); debug!("Not sending stats");
} }
Ok(()) Ok(())
} }
fn sign_message(private_key_path: &str, message: &str) -> String { fn sign_message(private_key_path: &str, message: &str) -> String {
let key_data = fs::read(private_key_path).unwrap(); let signing_key =
SigningKey::read_pkcs8_pem_file(private_key_path).expect("failed to parse private key PEM");
let signature = signing_key.sign(message.as_bytes());
let private_key = PKey::private_key_from_pem(&key_data).unwrap(); general_purpose::STANDARD.encode(signature.to_bytes())
let mut signer = Signer::new_without_digest(&private_key).unwrap();
let signature = signer.sign_oneshot_to_vec(message.as_bytes()).unwrap();
let signature_b64 = general_purpose::STANDARD.encode(&signature);
signature_b64
} }
fn parse_env(cfg: &mut MyConfig) { fn parse_env(cfg: &mut MyConfig) {
@@ -490,71 +579,45 @@ fn parse_env_netconfig(cfg_lock: &mut MyConfig, chain: &str) -> NetworkParams {
"testnet4" => &mut cfg_lock.testnet4, "testnet4" => &mut cfg_lock.testnet4,
&_ => &mut cfg_lock.mainnet, &_ => &mut cfg_lock.mainnet,
}; };
match env::var(format!("BAL_PUSHER_{}_HOST", chain.to_uppercase())) { if let Ok(value) = env::var(format!("BAL_PUSHER_{}_HOST", chain.to_uppercase())) {
Ok(value) => {
cfg.host = value; cfg.host = value;
} }
Err(_) => {} if let Ok(value) = env::var(format!("BAL_PUSHER_{}_PORT", chain.to_uppercase()))
} && let Ok(port_num) = value.parse::<u64>()
match env::var(format!("BAL_PUSHER_{}_PORT", chain.to_uppercase())) { {
Ok(value) => match value.parse::<u64>() { if let Ok(port) = u16::try_from(port_num) {
Ok(value) => match u16::try_from(value) { cfg.port = port;
Ok(port) => cfg.port = port, } else {
Err(e) => {
error!( error!(
"Port value {} exceeds u16 range for chain {}: {}", "Port value {} exceeds u16 range for chain {}",
value, chain, e port_num, chain
); );
} }
},
Err(_) => {}
},
Err(_) => {}
} }
match env::var(format!("BAL_PUSHER_{}_DIR_PATH", chain.to_uppercase())) { if let Ok(value) = env::var(format!("BAL_PUSHER_{}_DIR_PATH", chain.to_uppercase())) {
Ok(value) => {
cfg.dir_path = value; cfg.dir_path = value;
} }
Err(_) => {} if let Ok(value) = env::var(format!("BAL_PUSHER_{}_DB_FIELD", chain.to_uppercase())) {
}
match env::var(format!("BAL_PUSHER_{}_DB_FIELD", chain.to_uppercase())) {
Ok(value) => {
cfg.db_field = value; cfg.db_field = value;
} }
Err(_) => {} if let Ok(value) = env::var(format!("BAL_PUSHER_{}_COOKIE_FILE", chain.to_uppercase())) {
}
match env::var(format!("BAL_PUSHER_{}_COOKIE_FILE", chain.to_uppercase())) {
Ok(value) => {
cfg.cookie_file = value; cfg.cookie_file = value;
} }
Err(_) => {} if let Ok(value) = env::var(format!("BAL_PUSHER_{}_RPC_USER", chain.to_uppercase())) {
}
match env::var(format!("BAL_PUSHER_{}_RPC_USER", chain.to_uppercase())) {
Ok(value) => {
cfg.rpc_user = value; cfg.rpc_user = value;
} }
Err(_) => {} if let Ok(value) = env::var(format!("BAL_PUSHER_{}_RPC_PASSWORD", chain.to_uppercase())) {
}
match env::var(format!("BAL_PUSHER_{}_RPC_PASSWORD", chain.to_uppercase())) {
Ok(value) => {
cfg.rpc_pass = value; cfg.rpc_pass = value;
} }
Err(_) => {} println!("BAL_PUSHER_{}_ZMQ_HASHBLOCK", chain.to_uppercase());
} if let Ok(value) = env::var(format!("BAL_PUSHER_{}_ZMQ_HASHBLOCK", chain.to_uppercase())) {
println!(
"{}",
format!("BAL_PUSHER_{}_ZMQ_HASHBLOCK", chain.to_uppercase())
);
match env::var(format!("BAL_PUSHER_{}_ZMQ_HASHBLOCK", chain.to_uppercase())) {
Ok(value) => {
println!("value:{}", value); println!("value:{}", value);
cfg.zmq_listener = value; cfg.zmq_listener = value;
} }
Err(_) => {}
}
cfg.clone() cfg.clone()
} }
#[allow(dead_code)]
fn check_zmq_connection(endpoint: &str) -> bool { fn check_zmq_connection(endpoint: &str) -> bool {
trace!("check zmq connection"); trace!("check zmq connection");
let context = Context::new(); let context = Context::new();
@@ -572,6 +635,7 @@ fn check_zmq_connection(endpoint: &str) -> bool {
} }
// Add this struct to monitor connection health // Add this struct to monitor connection health
#[allow(dead_code)]
struct ConnectionMonitor { struct ConnectionMonitor {
last_message_time: Instant, last_message_time: Instant,
timeout: Duration, timeout: Duration,
@@ -579,6 +643,7 @@ struct ConnectionMonitor {
max_consecutive_timeouts: u32, max_consecutive_timeouts: u32,
} }
#[allow(dead_code)]
impl ConnectionMonitor { impl ConnectionMonitor {
fn new(timeout_secs: u64, max_timeouts: u32) -> Self { fn new(timeout_secs: u64, max_timeouts: u32) -> Self {
Self { Self {
@@ -616,6 +681,7 @@ impl ConnectionMonitor {
} }
} }
#[allow(dead_code)]
enum ConnectionStatus { enum ConnectionStatus {
Healthy, Healthy,
Warning(Duration), Warning(Duration),
@@ -627,7 +693,6 @@ async fn main() -> std::io::Result<()> {
env_logger::init(); env_logger::init();
let mut cfg = MyConfig::default(); let mut cfg = MyConfig::default();
let dbfile = env::var("BAL_PUSHER_DB_FILE").unwrap();
parse_env(&mut cfg); parse_env(&mut cfg);
let mut args = std::env::args(); let mut args = std::env::args();
let _exe_name = args.next().unwrap(); let _exe_name = args.next().unwrap();
@@ -662,31 +727,48 @@ async fn main() -> std::io::Result<()> {
} }
match socket.set_subscribe(b"") { match socket.set_subscribe(b"") {
Ok(_) => {} Ok(_) => {
info!("ZMQ subscribed to all topics on {}", zmq_address);
}
Err(e) => { Err(e) => {
error!("ZMQ subscribe failed: {}, exiting", e); error!("ZMQ subscribe failed: {}, exiting", e);
return Ok(()); return Ok(());
} }
} }
let _ = main_result(&cfg, network_params).await; if let Err(e) = main_result(&cfg, network_params).await {
error!("main_result failed on startup: {}", e);
}
info!("waiting new blocks.."); info!("waiting new blocks..");
let mut last_seq: Vec<u8> = [0; 4].to_vec();
let mut counter = 0;
let max = 100;
socket.set_rcvtimeo(5000).unwrap(); // 5 seconds timeout socket.set_rcvtimeo(5000).unwrap(); // 5 seconds timeout
let mut consecutive_timeouts: u32 = 0;
loop { loop {
let message = match socket.recv_multipart(0) { let message = match socket.recv_multipart(0) {
Ok(m) => m, Ok(m) => m,
Err(e) => { Err(e) => {
consecutive_timeouts += 1;
if consecutive_timeouts == 1 {
warn!("ZMQ recv timeout or error: {}, retrying...", e); warn!("ZMQ recv timeout or error: {}, retrying...", e);
} else if consecutive_timeouts.is_multiple_of(12) {
warn!(
"No ZMQ messages for {}s ({} consecutive timeouts), is bitcoind ZMQ active on {}?",
consecutive_timeouts * 5,
consecutive_timeouts,
zmq_address
);
}
continue; continue;
} }
}; };
if consecutive_timeouts > 0 {
info!(
"ZMQ connection restored after {} consecutive timeouts",
consecutive_timeouts
);
}
consecutive_timeouts = 0;
let topic = message[0].clone(); let topic = message[0].clone();
let body = message[1].clone(); let body = message[1].clone();
let seq = message[2].clone();
last_seq = seq;
debug!( debug!(
"ZMQ:GET TOPIC: {}", "ZMQ:GET TOPIC: {}",
String::from_utf8(topic.clone()).expect("invalid topic") String::from_utf8(topic.clone()).expect("invalid topic")
@@ -694,12 +776,15 @@ async fn main() -> std::io::Result<()> {
trace!("ZMQ:GET BODY: {}", hex::encode(&body)); trace!("ZMQ:GET BODY: {}", hex::encode(&body));
if topic == b"hashblock" { if topic == b"hashblock" {
info!("NEW BLOCK: {}", hex::encode(&body)); info!("NEW BLOCK: {}", hex::encode(&body));
let _ = main_result(&cfg, network_params).await; if let Err(e) = main_result(&cfg, network_params).await {
error!("main_result failed on new block: {}", e);
}
} }
thread::sleep(Duration::from_millis(100)); // Sleep for 100ms thread::sleep(Duration::from_millis(100)); // Sleep for 100ms
} }
} }
fn seq_to_str(seq: &Vec<u8>) -> String { #[allow(dead_code)]
fn seq_to_str(seq: &[u8]) -> String {
if seq.len() == 4 { if seq.len() == 4 {
let mut rdr = Cursor::new(seq); let mut rdr = Cursor::new(seq);
let sequence = rdr let sequence = rdr
@@ -709,3 +794,45 @@ fn seq_to_str(seq: &Vec<u8>) -> String {
} }
"Unknown".to_string() "Unknown".to_string()
} }
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn parse_host_port_https_default_port() {
assert_eq!(
parse_host_port("https://welist.bitcoin-after.life"),
Some(("welist.bitcoin-after.life".to_string(), 443))
);
}
#[test]
fn parse_host_port_explicit_port_and_path() {
assert_eq!(
parse_host_port("https://example.com:8443/ping"),
Some(("example.com".to_string(), 8443))
);
}
#[test]
fn parse_host_port_http_default_port() {
assert_eq!(
parse_host_port("http://example.com"),
Some(("example.com".to_string(), 80))
);
}
#[test]
fn parse_host_port_ipv6_literal_brackets_stripped() {
assert_eq!(
parse_host_port("https://[2a13:2c0::1]:443"),
Some(("2a13:2c0::1".to_string(), 443))
);
}
#[test]
fn parse_host_port_invalid_url() {
assert_eq!(parse_host_port("not a url"), None);
}
}

170
src/bin/bal-server-actix.rs → src/bin/bal-server.rs
View File

@@ -7,7 +7,6 @@ use chrono::Utc;
use hex_conservative::FromHex; use hex_conservative::FromHex;
use log::{debug, error, info, trace}; use log::{debug, error, info, trace};
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
use serde_json;
use sqlite::State; use sqlite::State;
use sqlite::{Connection, Value}; use sqlite::{Connection, Value};
use std::collections::{HashMap, HashSet}; use std::collections::{HashMap, HashSet};
@@ -119,6 +118,7 @@ pub struct StatsResponse {
} }
#[derive(Debug, Clone)] #[derive(Debug, Clone)]
#[allow(dead_code)]
struct ActixConfig { struct ActixConfig {
max_body_size: usize, max_body_size: usize,
timeout_secs: u64, timeout_secs: u64,
@@ -208,7 +208,7 @@ async fn echo_pub_key(data: web::Data<AppState>) -> impl Responder {
"Failed to read public key file {}: {}", "Failed to read public key file {}: {}",
data.cfg.pub_key_path, e data.cfg.pub_key_path, e
); );
HttpResponse::InternalServerError().body("Failed to read public key file") HttpResponse::InternalServerError().body("error")
} }
} }
} }
@@ -224,13 +224,13 @@ async fn echo_info(
) -> impl Responder { ) -> impl Responder {
let param = path.into_inner(); let param = path.into_inner();
if !NETWORKS.contains(&param.as_str()) { if !NETWORKS.contains(&param.as_str()) {
return HttpResponse::NotFound().body("Unknown network"); return HttpResponse::NotFound().body("error");
} }
info!("echo info!!!{}", param); info!("echo info!!!{}", param);
let netconfig = data.cfg.get_net_config(&param); let netconfig = data.cfg.get_net_config(&param);
if !netconfig.enabled { if !netconfig.enabled {
debug!("network disabled {}", param); debug!("network disabled {}", param);
return HttpResponse::BadRequest().body("network disabled"); return HttpResponse::BadRequest().body("error");
} }
let remote_addr = req let remote_addr = req
.headers() .headers()
@@ -257,7 +257,7 @@ async fn echo_info(
Ok(g) => g, Ok(g) => g,
Err(_p) => { Err(_p) => {
error!("DB mutex poisoned in echo_info (lookup phase)"); error!("DB mutex poisoned in echo_info (lookup phase)");
return HttpResponse::InternalServerError().body("DB mutex poisoned"); return HttpResponse::InternalServerError().body("error");
} }
}; };
match get_last_used_address_by_ip( match get_last_used_address_by_ip(
@@ -266,29 +266,26 @@ async fn echo_info(
&netconfig.address, &netconfig.address,
&remote_addr, &remote_addr,
) { ) {
Some(address) => return HttpResponse::Ok().json(InfoResponse { Some(address) => {
return HttpResponse::Ok().json(InfoResponse {
address, address,
base_fee: netconfig.fixed_fee, base_fee: netconfig.fixed_fee,
chain: netconfig.network.to_string(), chain: netconfig.network.to_string(),
info: data.cfg.info.to_string(), info: data.cfg.info.to_string(),
version: VERSION.to_string(), version: VERSION.to_string(),
}), });
None => {
let next = get_next_address_index(&db, &netconfig.name, &netconfig.address);
next
} }
None => get_next_address_index(&db, &netconfig.name, &netconfig.address),
} }
}; // lock released }; // lock released
// Derive address (CPU-bound, no lock held) // Derive address (CPU-bound, no lock held)
let derived = match new_address_from_xpub( let derived =
&netconfig.address, next_idx.1, netconfig.network match new_address_from_xpub(&netconfig.address, next_idx.1, netconfig.network) {
) {
Ok(address) => address, Ok(address) => address,
Err(e) => { Err(e) => {
error!("Failed to derive address from xpub: {}", e); error!("Failed to derive address from xpub: {}", e);
return HttpResponse::BadRequest() return HttpResponse::BadRequest().body("error");
.body(format!("Failed to derive address: {}", e));
} }
}; };
@@ -298,7 +295,7 @@ async fn echo_info(
Ok(g) => g, Ok(g) => g,
Err(_p) => { Err(_p) => {
error!("DB mutex poisoned in echo_info (save phase)"); error!("DB mutex poisoned in echo_info (save phase)");
return HttpResponse::InternalServerError().body("DB mutex poisoned"); return HttpResponse::InternalServerError().body("error");
} }
}; };
save_new_address(&db, next_idx.0, &derived.0, &derived.1, &remote_addr); save_new_address(&db, next_idx.0, &derived.0, &derived.1, &remote_addr);
@@ -321,30 +318,30 @@ async fn echo_info(
debug!("echo info reply: {}", json_data); debug!("echo info reply: {}", json_data);
HttpResponse::Ok().json(info) HttpResponse::Ok().json(info)
} }
Err(err) => HttpResponse::InternalServerError().body(format!("error:{}", err)), Err(_err) => HttpResponse::InternalServerError().body("error"),
} }
} }
async fn echo_stats(path: web::Path<String>, data: web::Data<AppState>) -> impl Responder { async fn echo_stats(path: web::Path<String>, data: web::Data<AppState>) -> impl Responder {
let param = path.into_inner(); let param = path.into_inner();
if !NETWORKS.contains(&param.as_str()) { if !NETWORKS.contains(&param.as_str()) {
return HttpResponse::NotFound().body("Unknown network"); return HttpResponse::NotFound().body("error");
} }
info!("echo stats!!! {}", data.cfg.expose_stats); info!("echo stats!!! {}", data.cfg.expose_stats);
let netconfig = data.cfg.get_net_config(&param); let netconfig = data.cfg.get_net_config(&param);
if !netconfig.enabled { if !netconfig.enabled {
debug!("network disabled {}", param); debug!("network disabled {}", param);
return HttpResponse::BadRequest().body("network disabled"); return HttpResponse::BadRequest().body("error");
} }
if !data.cfg.expose_stats { if !data.cfg.expose_stats {
return HttpResponse::Forbidden().body("Stats not exposed"); return HttpResponse::Forbidden().body("error");
} }
let mut stats: Vec<StatsResponse> = vec![]; let mut stats: Vec<StatsResponse> = vec![];
let db = match data.db.lock() { let db = match data.db.lock() {
Ok(g) => g, Ok(g) => g,
Err(_p) => { Err(_p) => {
error!("DB mutex poisoned in echo_stats"); error!("DB mutex poisoned in echo_stats");
return HttpResponse::InternalServerError().body("DB mutex poisoned"); return HttpResponse::InternalServerError().body("error");
} }
}; };
let mut stmt = match db.prepare( let mut stmt = match db.prepare(
@@ -353,24 +350,56 @@ async fn echo_stats(path: web::Path<String>, data: web::Data<AppState>) -> impl
Ok(s) => s, Ok(s) => s,
Err(e) => { Err(e) => {
error!("Failed to prepare stats query: {}", e); error!("Failed to prepare stats query: {}", e);
return HttpResponse::InternalServerError().body("Database error"); return HttpResponse::InternalServerError().body("error");
} }
}; };
if let Err(e) = stmt.bind((1, Value::String(netconfig.name.clone()))) { if let Err(e) = stmt.bind((1, Value::String(netconfig.name.clone()))) {
error!("Failed to bind chain in stats query: {}", e); error!("Failed to bind chain in stats query: {}", e);
return HttpResponse::InternalServerError().body("Database error"); return HttpResponse::InternalServerError().body("error");
} }
while let Ok(State::Row) = stmt.next() { while let Ok(State::Row) = stmt.next() {
let report_date = stmt.read("report_date").unwrap_or("0".to_string()); let report_date = stmt.read("report_date").unwrap_or("0".to_string());
let chain = stmt.read("chain").unwrap_or("?".to_string()); let chain = stmt.read("chain").unwrap_or("?".to_string());
let totals = stmt.read("totals").unwrap_or("0".to_string()).parse::<i64>().unwrap_or(0); let totals = stmt
let waiting = stmt.read("waiting").unwrap_or("0".to_string()).parse::<i64>().unwrap_or(0); .read("totals")
let sent = stmt.read("sent").unwrap_or("0".to_string()).parse::<i64>().unwrap_or(0); .unwrap_or("0".to_string())
let failed = stmt.read("failed").unwrap_or("0".to_string()).parse::<i64>().unwrap_or(0); .parse::<i64>()
let waiting_profit = stmt.read("waiting_profit").unwrap_or("0".to_string()).parse::<i64>().unwrap_or(0); .unwrap_or(0);
let sent_profit = stmt.read("sent_profit").unwrap_or("0".to_string()).parse::<i64>().unwrap_or(0); let waiting = stmt
let missed_profit = stmt.read("missed_profit").unwrap_or("0".to_string()).parse::<i64>().unwrap_or(0); .read("waiting")
let unique_inputs = stmt.read("unique_inputs").unwrap_or("0".to_string()).parse::<i64>().unwrap_or(0); .unwrap_or("0".to_string())
.parse::<i64>()
.unwrap_or(0);
let sent = stmt
.read("sent")
.unwrap_or("0".to_string())
.parse::<i64>()
.unwrap_or(0);
let failed = stmt
.read("failed")
.unwrap_or("0".to_string())
.parse::<i64>()
.unwrap_or(0);
let waiting_profit = stmt
.read("waiting_profit")
.unwrap_or("0".to_string())
.parse::<i64>()
.unwrap_or(0);
let sent_profit = stmt
.read("sent_profit")
.unwrap_or("0".to_string())
.parse::<i64>()
.unwrap_or(0);
let missed_profit = stmt
.read("missed_profit")
.unwrap_or("0".to_string())
.parse::<i64>()
.unwrap_or(0);
let unique_inputs = stmt
.read("unique_inputs")
.unwrap_or("0".to_string())
.parse::<i64>()
.unwrap_or(0);
stats.push(StatsResponse { stats.push(StatsResponse {
report_date, report_date,
chain, chain,
@@ -389,7 +418,7 @@ async fn echo_stats(path: web::Path<String>, data: web::Data<AppState>) -> impl
debug!("echo info reply: {}", json_data); debug!("echo info reply: {}", json_data);
HttpResponse::Ok().json(stats) HttpResponse::Ok().json(stats)
} }
Err(err) => HttpResponse::InternalServerError().body(format!("error:{}", err)), Err(_err) => HttpResponse::InternalServerError().body("error"),
} }
} }
@@ -398,32 +427,33 @@ async fn echo_search(body: Bytes, data: web::Data<AppState>) -> impl Responder {
let strbody = match std::str::from_utf8(&body) { let strbody = match std::str::from_utf8(&body) {
Ok(s) => s, Ok(s) => s,
Err(_) => { Err(_) => {
return HttpResponse::BadRequest().body("Invalid UTF-8 body"); return HttpResponse::BadRequest().body("error");
} }
}; };
info!("{}", strbody); info!("{}", strbody);
if strbody.is_empty() || strbody.len() != 64 || !strbody.chars().all(|c| c.is_ascii_hexdigit()) { if strbody.is_empty() || strbody.len() != 64 || !strbody.chars().all(|c| c.is_ascii_hexdigit())
return HttpResponse::BadRequest().body("Invalid txid"); {
return HttpResponse::BadRequest().body("error");
} }
let db = match data.db.lock() { let db = match data.db.lock() {
Ok(g) => g, Ok(g) => g,
Err(_p) => { Err(_p) => {
error!("DB mutex poisoned in echo_search"); error!("DB mutex poisoned in echo_search");
return HttpResponse::InternalServerError().body("DB mutex poisoned"); return HttpResponse::InternalServerError().body("error");
} }
}; };
let mut statement = match db.prepare("SELECT * FROM tbl_tx WHERE txid = ? LIMIT 1") { let mut statement = match db.prepare("SELECT * FROM tbl_tx WHERE txid = ? LIMIT 1") {
Ok(s) => s, Ok(s) => s,
Err(e) => { Err(e) => {
error!("Failed to prepare statement: {}", e); error!("Failed to prepare statement: {}", e);
return HttpResponse::InternalServerError().body("Database error"); return HttpResponse::InternalServerError().body("error");
} }
}; };
if let Err(e) = statement.bind((1, strbody)) { if let Err(e) = statement.bind((1, strbody)) {
error!("Failed to bind parameter: {}", e); error!("Failed to bind parameter: {}", e);
return HttpResponse::InternalServerError().body("Database error"); return HttpResponse::InternalServerError().body("error");
} }
if let Ok(State::Row) = statement.next() { if let Ok(State::Row) = statement.next() {
@@ -469,11 +499,14 @@ async fn echo_search(body: Bytes, data: web::Data<AppState>) -> impl Responder {
} }
} }
match serde_json::to_string(&response_data) { match serde_json::to_string(&response_data) {
Ok(json_data) => HttpResponse::Ok().json(json_data), Ok(json_data) => {
Err(_) => HttpResponse::BadRequest().body("Bad data received"), debug!("echo search reply: {}", json_data);
HttpResponse::Ok().json(&response_data)
}
Err(_) => HttpResponse::BadRequest().body("error"),
} }
} else { } else {
HttpResponse::BadRequest().body("Bad data received") HttpResponse::BadRequest().body("error")
} }
} }
@@ -486,17 +519,17 @@ struct ParsedTx {
raw_hex: String, // the original line raw_hex: String, // the original line
locktime: String, locktime: String,
inputs: Vec<(String, String)>, // (in_txid, in_vout) inputs: Vec<(String, String)>, // (in_txid, in_vout)
outputs: Vec<(usize, String, u64)> // (idx, script_pubkey, amount_sat) outputs: Vec<(usize, String, u64)>, // (idx, script_pubkey, amount_sat)
} }
/// Parse all transactions from the request body **without** needing the DB lock. /// Parse all transactions from the request body **without** needing the DB lock.
/// Returns `Ok(parsed_txs)` if at least one tx was valid, or `Err(HttpResponse)` for early failure. /// Skips transactions that don't have a valid willexecutor output.
fn parse_request_transactions( fn parse_request_transactions(
strbody: &str, strbody: &str,
_req_time: i64, _req_time: i64,
netconfig: &NetConfig, netconfig: &NetConfig,
known_addresses: &HashSet<String>, known_addresses: &HashSet<String>,
) -> Result<Vec<(ParsedTx, String, u64)>, HttpResponse> { ) -> Vec<(ParsedTx, String, u64)> {
let mut result: Vec<(ParsedTx, String, u64)> = Vec::new(); let mut result: Vec<(ParsedTx, String, u64)> = Vec::new();
let mut union_tx = true; let mut union_tx = true;
@@ -560,7 +593,7 @@ fn parse_request_transactions(
if known_addresses.contains(&address) { if known_addresses.contains(&address) {
address.clone() address.clone()
} else { } else {
continue continue;
} }
} else { } else {
netconfig.address.clone() netconfig.address.clone()
@@ -579,8 +612,8 @@ fn parse_request_transactions(
} }
if !found { if !found {
error!("willexecutor output not found for tx {}", txid); trace!("willexecutor output not found for tx {}, skipping", txid);
return Err(HttpResponse::BadRequest().body("Bad data received")); continue;
} }
if !union_tx { if !union_tx {
// This is only used for SQL building later; we track it in the caller // This is only used for SQL building later; we track it in the caller
@@ -602,7 +635,7 @@ fn parse_request_transactions(
)); ));
} }
Ok(result) result
} }
async fn echo_push( async fn echo_push(
@@ -614,24 +647,24 @@ async fn echo_push(
let strbody = match std::str::from_utf8(&body) { let strbody = match std::str::from_utf8(&body) {
Ok(s) => s, Ok(s) => s,
Err(_) => { Err(_) => {
return HttpResponse::BadRequest().body("Invalid UTF-8 body"); return HttpResponse::BadRequest().body("error");
} }
}; };
let param = path.into_inner(); let param = path.into_inner();
if !NETWORKS.contains(&param.as_str()) { if !NETWORKS.contains(&param.as_str()) {
return HttpResponse::NotFound().body("Unknown network"); return HttpResponse::NotFound().body("error");
} }
let netconfig = data.cfg.get_net_config(&param); let netconfig = data.cfg.get_net_config(&param);
if !netconfig.enabled { if !netconfig.enabled {
trace!("network not enabled {}", &netconfig.name); trace!("network not enabled {}", &netconfig.name);
return HttpResponse::BadRequest().body("Network not enabled"); return HttpResponse::BadRequest().body("error");
} }
let req_time = match Utc::now().timestamp_nanos_opt() { let req_time = match Utc::now().timestamp_nanos_opt() {
Some(t) => t, Some(t) => t,
None => { None => {
error!("Invalid timestamp"); error!("Invalid timestamp");
return HttpResponse::BadRequest().body("Invalid timestamp"); return HttpResponse::BadRequest().body("error");
} }
}; };
@@ -641,7 +674,7 @@ async fn echo_push(
Ok(g) => g, Ok(g) => g,
Err(_p) => { Err(_p) => {
error!("DB mutex poisoned acquiring addresses in echo_push"); error!("DB mutex poisoned acquiring addresses in echo_push");
return HttpResponse::InternalServerError().body("DB mutex poisoned"); return HttpResponse::InternalServerError().body("error");
} }
}; };
if netconfig.xpub { if netconfig.xpub {
@@ -649,7 +682,7 @@ async fn echo_push(
Ok(addrs) => addrs, Ok(addrs) => addrs,
Err(e) => { Err(e) => {
error!("Failed to load addresses from xpub: {}", e); error!("Failed to load addresses from xpub: {}", e);
return HttpResponse::InternalServerError().body("Database error"); return HttpResponse::InternalServerError().body("error");
} }
} }
} else { } else {
@@ -658,14 +691,9 @@ async fn echo_push(
}; // lock released here }; // lock released here
// Parse all transactions (CPU-bound, no DB needed) // Parse all transactions (CPU-bound, no DB needed)
let parsed = match parse_request_transactions( let parsed = parse_request_transactions(strbody, req_time, netconfig, &known_addresses);
strbody, req_time, netconfig, &known_addresses,
) {
Ok(v) => v,
Err(resp) => return resp,
};
if parsed.is_empty() { if parsed.is_empty() {
return HttpResponse::Ok().body("thx"); return HttpResponse::BadRequest().body("error");
} }
let all_txids: Vec<String> = parsed.iter().map(|(p, _, _)| p.txid.clone()).collect(); let all_txids: Vec<String> = parsed.iter().map(|(p, _, _)| p.txid.clone()).collect();
@@ -676,14 +704,14 @@ async fn echo_push(
Ok(g) => g, Ok(g) => g,
Err(_p) => { Err(_p) => {
error!("DB mutex poisoned in echo_push duplicate check"); error!("DB mutex poisoned in echo_push duplicate check");
return HttpResponse::InternalServerError().body("DB mutex poisoned"); return HttpResponse::InternalServerError().body("error");
} }
}; };
match check_duplicate_txids(&db, &all_txids) { match check_duplicate_txids(&db, &all_txids) {
Ok(dups) => dups, Ok(dups) => dups,
Err(e) => { Err(e) => {
error!("Duplicate check failed: {}", e); error!("Duplicate check failed: {}", e);
return HttpResponse::InternalServerError().body("Database error"); return HttpResponse::InternalServerError().body("error");
} }
} }
}; // lock released here }; // lock released here
@@ -699,7 +727,7 @@ async fn echo_push(
Ok(g) => g, Ok(g) => g,
Err(_p) => { Err(_p) => {
error!("DB mutex poisoned in echo_push insert phase"); error!("DB mutex poisoned in echo_push insert phase");
return HttpResponse::InternalServerError().body("DB mutex poisoned"); return HttpResponse::InternalServerError().body("error");
} }
}; };
@@ -763,9 +791,15 @@ async fn echo_push(
} }
sqlouts.push_str(" SELECT ?, ?, ?, ?"); sqlouts.push_str(" SELECT ?, ?, ?, ?");
pouts.push((lineout, Value::String(parsed.txid.clone()))); pouts.push((lineout, Value::String(parsed.txid.clone())));
pouts.push((lineout + 1, Value::Integer(i64::try_from(*idx).unwrap_or(-1)))); pouts.push((
lineout + 1,
Value::Integer(i64::try_from(*idx).unwrap_or(-1)),
));
pouts.push((lineout + 2, Value::String(script.clone()))); pouts.push((lineout + 2, Value::String(script.clone())));
pouts.push((lineout + 3, Value::Integer(i64::try_from(*amount).unwrap_or(0)))); pouts.push((
lineout + 3,
Value::Integer(i64::try_from(*amount).unwrap_or(0)),
));
lineout += 4; lineout += 4;
} }
} }
@@ -780,7 +814,7 @@ async fn echo_push(
if let Err(err) = execute_insert(&db, sqltxs, ptx, sqlinps, pinps, sqlouts, pouts) { if let Err(err) = execute_insert(&db, sqltxs, ptx, sqlinps, pinps, sqlouts, pouts) {
error!("execute_insert failed: {}", err); error!("execute_insert failed: {}", err);
return HttpResponse::BadRequest().body("Bad data received"); return HttpResponse::BadRequest().body("error");
} }
} // lock released } // lock released
@@ -862,7 +896,7 @@ async fn main() -> std::io::Result<()> {
let db = match open_db(&cfg.db_file) { let db = match open_db(&cfg.db_file) {
Ok(c) => c, Ok(c) => c,
Err(e) => { Err(e) => {
return Err(std::io::Error::new(std::io::ErrorKind::Other, e)); return Err(std::io::Error::other(e));
} }
}; };

64
src/db.rs
View File

@@ -1,7 +1,9 @@
use log::{error, info, trace}; use log::{error, info, trace, warn};
use sqlite::{Connection, Error, State, Value}; use sqlite::{Connection, Error, State, Value};
use std::collections::HashSet; use std::collections::HashSet;
use std::path::Path; use std::path::Path;
use std::thread;
use std::time::Duration;
/// Check which txids are already present in the database in a single batch query. /// Check which txids are already present in the database in a single batch query.
/// Returns a HashSet of txids that already exist (duplicates). /// Returns a HashSet of txids that already exist (duplicates).
@@ -61,12 +63,10 @@ pub fn open_db(path: &str) -> Result<Connection, String> {
]; ];
for prefix in &forbidden { for prefix in &forbidden {
if path_str.starts_with(prefix) { if path_str.starts_with(prefix) {
return Err( return Err(format!(
format!(
"Absolute database path under {} is forbidden", "Absolute database path under {} is forbidden",
prefix prefix
) ));
);
} }
} }
} }
@@ -74,24 +74,48 @@ pub fn open_db(path: &str) -> Result<Connection, String> {
// If file exists, must be a regular file (not a symlink, device, etc.) // If file exists, must be a regular file (not a symlink, device, etc.)
if p.exists() { if p.exists() {
if p.is_symlink() { if p.is_symlink() {
return Err( return Err("Database path must not be a symlink".to_string());
"Database path must not be a symlink".to_string()
);
} }
let metadata = std::fs::metadata(p) let metadata = std::fs::metadata(p)
.map_err(|e| format!("Cannot access database file metadata: {}", e))?; .map_err(|e| format!("Cannot access database file metadata: {}", e))?;
if !metadata.is_file() { if !metadata.is_file() {
return Err( return Err(
"Database path must point to a regular file, not a directory or device".to_string() "Database path must point to a regular file, not a directory or device".to_string(),
); );
} }
} }
let conn = sqlite::open(path) let conn = sqlite::open(path).map_err(|e| format!("Failed to open SQLite database: {}", e))?;
.map_err(|e| format!("Failed to open SQLite database: {}", e))?;
// Set busy timeout BEFORE WAL mode so SQLite waits instead of failing immediately.
// This handles the race where two processes (server + pusher) open the same DB
// and both try to enable WAL mode concurrently.
conn.execute("PRAGMA busy_timeout = 5000;")
.map_err(|e| format!("Failed to set busy_timeout: {}", e))?;
// Retry WAL mode up to 5 times (handles concurrent open from bal-pusher).
let mut wal_ok = false;
for attempt in 0..5 {
match conn.execute("PRAGMA journal_mode = WAL;") {
Ok(_) => {
wal_ok = true;
break;
}
Err(e) => {
warn!(
"WAL mode attempt {}/5 failed: {}, retrying in 100ms...",
attempt + 1,
e
);
thread::sleep(Duration::from_millis(100));
}
}
}
if !wal_ok {
// WAL might already be enabled by another process; this is not fatal.
warn!("Could not set WAL mode after retries — may already be enabled by another process");
}
conn.execute("PRAGMA journal_mode = WAL;")
.map_err(|e| format!("Failed to enable WAL mode: {}", e))?;
conn.execute("PRAGMA synchronous = NORMAL;") conn.execute("PRAGMA synchronous = NORMAL;")
.map_err(|e| format!("Failed to set synchronous NORMAL: {}", e))?; .map_err(|e| format!("Failed to set synchronous NORMAL: {}", e))?;
@@ -102,13 +126,15 @@ pub fn open_db(path: &str) -> Result<Connection, String> {
/// in-memory lookup during transaction validation (replaces N+1 query). /// in-memory lookup during transaction validation (replaces N+1 query).
pub fn get_all_addresses_by_xpub(db: &Connection, xpub: &str) -> Result<HashSet<String>, Error> { pub fn get_all_addresses_by_xpub(db: &Connection, xpub: &str) -> Result<HashSet<String>, Error> {
let mut stmt = db.prepare( let mut stmt = db.prepare(
"SELECT a.address FROM tbl_address a JOIN tbl_xpub x ON a.xpub = x.id WHERE x.xpub = ?" "SELECT a.address FROM tbl_address a JOIN tbl_xpub x ON a.xpub = x.id WHERE x.xpub = ?",
)?; )?;
stmt.bind((1, Value::String(xpub.to_string())))?; stmt.bind((1, Value::String(xpub.to_string())))?;
let mut addresses = HashSet::new(); let mut addresses = HashSet::new();
while let Ok(State::Row) = stmt.next() { while let Ok(State::Row) = stmt.next() {
match stmt.read::<String, _>("address") { match stmt.read::<String, _>("address") {
Ok(addr) => { addresses.insert(addr); } Ok(addr) => {
addresses.insert(addr);
}
Err(e) => { Err(e) => {
error!("Failed to read address column: {}", e); error!("Failed to read address column: {}", e);
} }
@@ -134,7 +160,9 @@ pub fn create_database(db: &Connection) {
let _ = db.execute("CREATE TABLE IF NOT EXISTS tbl_address (address TEXT PRIMARY_KEY, path TEXT NOT NULL, date_create TIMESTAMP DEFAULT CURRENT_TIMESTAMP, xpub INTEGER,remote_address TEXT);"); let _ = db.execute("CREATE TABLE IF NOT EXISTS tbl_address (address TEXT PRIMARY_KEY, path TEXT NOT NULL, date_create TIMESTAMP DEFAULT CURRENT_TIMESTAMP, xpub INTEGER,remote_address TEXT);");
let _ = db.execute("CREATE TABLE IF NOT EXISTS tbl_stats (report_date TEXT, chain TEXT, totals INTEGER, waiting INTEGER, sent INTEGER, failed INTEGER, waiting_profit INTEGER, sent_profit INTEGER, missed_profit INTEGER, unique_inputs INTEGER);"); let _ = db.execute("CREATE TABLE IF NOT EXISTS tbl_stats (report_date TEXT, chain TEXT, totals INTEGER, waiting INTEGER, sent INTEGER, failed INTEGER, waiting_profit INTEGER, sent_profit INTEGER, missed_profit INTEGER, unique_inputs INTEGER);");
let _ = db.execute("CREATE INDEX IF NOT EXISTS idx_stats_chain ON tbl_stats(chain);"); // UNIQUE index required for ON CONFLICT(chain) DO UPDATE in calculate_stats
let _ = db.execute("DROP INDEX IF EXISTS idx_stats_chain;");
let _ = db.execute("CREATE UNIQUE INDEX IF NOT EXISTS idx_stats_chain ON tbl_stats(chain);");
let _ = db.execute("UPDATE tbl_tx set network='bitcoin' where network='mainnet');"); let _ = db.execute("UPDATE tbl_tx set network='bitcoin' where network='mainnet');");
} }
@@ -150,8 +178,8 @@ pub fn create_database(db: &Connection) {
} }
} }
*/ */
pub fn insert_xpub(db: &Connection, network: &String, xpub: &String) { pub fn insert_xpub(db: &Connection, network: &str, xpub: &str) {
if xpub != "" { if !xpub.is_empty() {
trace!("going to insert: {} xpub:{}", network, xpub); trace!("going to insert: {} xpub:{}", network, xpub);
let mut stmt = match db.prepare("INSERT INTO tbl_xpub(network,xpub) VALUES(?, ?);") { let mut stmt = match db.prepare("INSERT INTO tbl_xpub(network,xpub) VALUES(?, ?);") {
Ok(s) => s, Ok(s) => s,

32
src/xpub.rs
View File

@@ -11,6 +11,7 @@ use sha2::{Digest, Sha256};
use std::str::FromStr; use std::str::FromStr;
// Mainnet (BIP44/BIP49/BIP84) // Mainnet (BIP44/BIP49/BIP84)
#[allow(dead_code)]
enum BS58Prefix { enum BS58Prefix {
Xpub, Xpub,
Ypub, Ypub,
@@ -102,44 +103,29 @@ fn calc_checksum(desc: &str) -> Result<String, String> {
Ok(checksum) Ok(checksum)
} }
pub fn get_bitcoincore_descriptor(xpub: &String) -> String { pub fn get_bitcoincore_descriptor(xpub: &str) -> String {
let fingerprint = match calculate_fingerprint(xpub) { let fingerprint = match calculate_fingerprint(xpub) {
Ok(f) => f, Ok(f) => f,
Err(_) => return String::new(), // Invalid xpub, return empty descriptor Err(_) => return String::new(), // Invalid xpub, return empty descriptor
}; };
let mut bip = 84;
let cpub = xpub.to_string();
match &xpub[0..4] {
"vpub" => {
bip = 84;
}
"zpub" => {
bip = 84;
}
&_ => {
bip = 84;
}
};
let xpub_converted = match convert_xpub(xpub) { let xpub_converted = match convert_xpub(xpub) {
Ok(c) => c, Ok(c) => c,
Err(_) => return String::new(), // Invalid xpub, return empty descriptor Err(_) => return String::new(), // Invalid xpub, return empty descriptor
}; };
let descriptor = format!("wpkh([{}/84h/0h/0h]{}/0/*)", fingerprint, xpub_converted); let descriptor = format!("wpkh([{}/84h/0h/0h]{}/0/*)", fingerprint, xpub_converted);
let descriptor = match calc_checksum(&descriptor) { match calc_checksum(&descriptor) {
Ok(checksum) => { Ok(checksum) => {
let clean_descriptor = descriptor.split('#').next().unwrap_or(&descriptor); let clean_descriptor = descriptor.split('#').next().unwrap_or(&descriptor);
format!("{}#{}", clean_descriptor, checksum) format!("{}#{}", clean_descriptor, checksum)
} }
Err(err) => { Err(err) => {
eprintln!("Error: {}", err); eprintln!("Error: {}", err);
"".to_string() String::new()
} }
};
descriptor
//format!("{}#{}",descriptor,checksum)
} }
fn convert_xpub(xpub: &String) -> Result<String, String> { }
fn convert_xpub(xpub: &str) -> Result<String, String> {
if xpub.len() >= 4 && (&xpub[0..4] == "xpub" || &xpub[0..4] == "ypub" || &xpub[0..4] == "zpub") if xpub.len() >= 4 && (&xpub[0..4] == "xpub" || &xpub[0..4] == "ypub" || &xpub[0..4] == "zpub")
{ {
convert_to(xpub, BS58Prefix::Xpub) convert_to(xpub, BS58Prefix::Xpub)
@@ -165,7 +151,7 @@ fn base58check_decode(s: &str) -> Result<Vec<u8>, String> {
return Err("Data troppo corta".to_string()); return Err("Data troppo corta".to_string());
} }
let (payload, checksum) = data.split_at(data.len() - 4); let (payload, checksum) = data.split_at(data.len() - 4);
let hash = Sha256::digest(&Sha256::digest(payload)); let hash = Sha256::digest(Sha256::digest(payload));
if hash[0..4] != checksum[..] { if hash[0..4] != checksum[..] {
return Err("Checksum invalido".to_string()); return Err("Checksum invalido".to_string());
} }
@@ -173,7 +159,7 @@ fn base58check_decode(s: &str) -> Result<Vec<u8>, String> {
} }
fn base58check_encode(data: &[u8]) -> String { fn base58check_encode(data: &[u8]) -> String {
let checksum = &Sha256::digest(&Sha256::digest(data))[0..4]; let checksum = &Sha256::digest(Sha256::digest(data))[0..4];
let full = [data, checksum].concat(); let full = [data, checksum].concat();
bs58::encode(full).into_string() bs58::encode(full).into_string()
} }
@@ -205,7 +191,7 @@ pub fn new_address_from_xpub(
) -> Result<(String, String), Box<dyn std::error::Error>> { ) -> Result<(String, String), Box<dyn std::error::Error>> {
let xpub = Xpub::from_str(&convert_to(zpub, BS58Prefix::Xpub)?)?; let xpub = Xpub::from_str(&convert_to(zpub, BS58Prefix::Xpub)?)?;
let path = format!("m/0/{}", index); let path = format!("m/0/{}", index);
let derivation_path = DerivationPath::from_str(&path.as_str())?; let derivation_path = DerivationPath::from_str(path.as_str())?;
let secp = Secp256k1::new(); let secp = Secp256k1::new();
let derived_xpub = xpub.derive_pub(&secp, &derivation_path)?; let derived_xpub = xpub.derive_pub(&secp, &derivation_path)?;
let public_key = derived_xpub.public_key; let public_key = derived_xpub.public_key;

40
tests/db_path_validation.rs
View File

@@ -1,37 +1,35 @@
use bal_server::db::open_db; use bal_server::db::open_db;
use sqlite::State; use sqlite::State;
use std::fs; use std::fs;
use std::path::Path;
#[test] #[test]
fn test_open_db_blocks_traversal() { fn test_open_db_blocks_traversal() {
let res = open_db("../etc/passwd"); let res = open_db("../etc/passwd");
assert!( assert!(res.is_err(), "Path with '..' should be rejected");
res.is_err(),
"Path with '..' should be rejected"
);
let err = match res { let err = match res {
Err(e) => e, Err(e) => e,
Ok(_) => panic!("Expected error for traversal path"), Ok(_) => panic!("Expected error for traversal path"),
}; };
assert!(err.contains("'..'"), "Error should mention directory traversal: {}", err); assert!(
err.contains("'..'"),
"Error should mention directory traversal: {}",
err
);
} }
#[test] #[test]
fn test_open_db_blocks_forbidden_absolute() { fn test_open_db_blocks_forbidden_absolute() {
for path in ["/etc/passwd", "/proc/self/mem", "/dev/null", "/usr/bin/ls"] { for path in ["/etc/passwd", "/proc/self/mem", "/dev/null", "/usr/bin/ls"] {
let res = open_db(path); let res = open_db(path);
assert!( assert!(res.is_err(), "Absolute path {} should be rejected", path);
res.is_err(),
"Absolute path {} should be rejected", path
);
let err = match res { let err = match res {
Err(e) => e, Err(e) => e,
Ok(_) => panic!("Expected error for forbidden path {}", path), Ok(_) => panic!("Expected error for forbidden path {}", path),
}; };
assert!( assert!(
err.contains("forbidden"), err.contains("forbidden"),
"Error should mention forbidden prefix: {}", err "Error should mention forbidden prefix: {}",
err
); );
} }
} }
@@ -41,10 +39,7 @@ fn test_open_db_allows_relative() {
let test_path = "tmp_test_bal.db"; let test_path = "tmp_test_bal.db";
let _ = fs::remove_file(test_path); let _ = fs::remove_file(test_path);
let res = open_db(test_path); let res = open_db(test_path);
assert!( assert!(res.is_ok(), "Valid relative path should be allowed");
res.is_ok(),
"Valid relative path should be allowed"
);
let db = res.unwrap(); let db = res.unwrap();
drop(db); drop(db);
let _ = fs::remove_file(test_path); let _ = fs::remove_file(test_path);
@@ -61,10 +56,7 @@ fn test_open_db_wal_pragmas_set() {
let mut stmt = db.prepare("PRAGMA journal_mode;").unwrap(); let mut stmt = db.prepare("PRAGMA journal_mode;").unwrap();
if let Ok(State::Row) = stmt.next() { if let Ok(State::Row) = stmt.next() {
let mode: String = stmt.read(0).unwrap(); let mode: String = stmt.read(0).unwrap();
assert_eq!( assert_eq!(mode, "wal", "SQLite journal mode should be WAL");
mode, "wal",
"SQLite journal mode should be WAL"
);
} else { } else {
panic!("Could not read journal_mode pragma"); panic!("Could not read journal_mode pragma");
} }
@@ -81,20 +73,18 @@ fn test_open_db_rejects_symlink() {
let _ = fs::remove_file(real); let _ = fs::remove_file(real);
let _ = fs::remove_file(link); let _ = fs::remove_file(link);
fs::File::create(real).unwrap(); fs::File::create(real).unwrap();
fs::soft_link(real, link).unwrap(); std::os::unix::fs::symlink(real, link).unwrap();
let res = open_db(link); let res = open_db(link);
assert!( assert!(res.is_err(), "Symlink DB path should be rejected");
res.is_err(),
"Symlink DB path should be rejected"
);
let err = match res { let err = match res {
Err(e) => e, Err(e) => e,
Ok(_) => panic!("Expected error for symlink"), Ok(_) => panic!("Expected error for symlink"),
}; };
assert!( assert!(
err.contains("symlink"), err.contains("symlink"),
"Error should mention symlink: {}", err "Error should mention symlink: {}",
err
); );
let _ = fs::remove_file(real); let _ = fs::remove_file(real);

19
tests/input_validation_tests.rs
View File

@@ -10,15 +10,21 @@ fn setup_db_with_xpub() -> sqlite::Connection {
"CREATE TABLE tbl_address (address TEXT PRIMARY KEY, path TEXT, xpub INTEGER, remote_address TEXT);" "CREATE TABLE tbl_address (address TEXT PRIMARY KEY, path TEXT, xpub INTEGER, remote_address TEXT);"
); );
// Insert test xpub // Insert test xpub
let mut stmt = db.prepare("INSERT INTO tbl_xpub(id, network, xpub) VALUES(?, ?, ?);").unwrap(); let mut stmt = db
.prepare("INSERT INTO tbl_xpub(id, network, xpub) VALUES(?, ?, ?);")
.unwrap();
stmt.bind((1, Value::Integer(1))).unwrap(); stmt.bind((1, Value::Integer(1))).unwrap();
stmt.bind((2, Value::String("testnet".to_string()))).unwrap(); stmt.bind((2, Value::String("testnet".to_string())))
stmt.bind((3, Value::String("tpub_test".to_string()))).unwrap(); .unwrap();
stmt.bind((3, Value::String("tpub_test".to_string())))
.unwrap();
let _ = stmt.next(); let _ = stmt.next();
drop(stmt); drop(stmt);
// Insert test addresses // Insert test addresses
for addr in ["addr1", "addr2", "addr3"] { for addr in ["addr1", "addr2", "addr3"] {
let mut stmt = db.prepare("INSERT INTO tbl_address(address, path, xpub) VALUES(?, ?, ?);").unwrap(); let mut stmt = db
.prepare("INSERT INTO tbl_address(address, path, xpub) VALUES(?, ?, ?);")
.unwrap();
stmt.bind((1, Value::String(addr.to_string()))).unwrap(); stmt.bind((1, Value::String(addr.to_string()))).unwrap();
stmt.bind((2, Value::String("m/0/1".to_string()))).unwrap(); stmt.bind((2, Value::String("m/0/1".to_string()))).unwrap();
stmt.bind((3, Value::Integer(1))).unwrap(); stmt.bind((3, Value::Integer(1))).unwrap();
@@ -53,7 +59,10 @@ fn test_network_unknown_returns_404() {
for n in networks { for n in networks {
assert!(networks.contains(&n), "{} should be a valid network", n); assert!(networks.contains(&n), "{} should be a valid network", n);
} }
assert!(!networks.contains(&"attacker"), "attacker should not be a valid network"); assert!(
!networks.contains(&"attacker"),
"attacker should not be a valid network"
);
} }
#[test] #[test]

7
tests/panic_regression_tests.rs
View File

@@ -36,11 +36,8 @@ fn test_db_null_unwrap_or() {
let mut found_value = None; let mut found_value = None;
let _ = db.iterate("SELECT * FROM test_stats;", |pairs| { let _ = db.iterate("SELECT * FROM test_stats;", |pairs| {
let row: HashMap<_, _> = pairs let row: HashMap<_, _> = pairs.iter().map(|(k, v)| (k.to_string(), *v)).collect();
.into_iter() let totals = row["totals"].unwrap_or("0").to_string();
.map(|(k, v)| (k.to_string(), v.map(|s| s)))
.collect();
let totals = row["totals"].clone().unwrap_or("0").to_string();
found_value = Some(totals); found_value = Some(totals);
true true
}); });

58
tests/secret_leakage_tests.rs
View File

@@ -1,5 +1,4 @@
use std::fs; use std::fs;
use std::path::Path;
#[test] #[test]
fn test_gitignore_protection_env() { fn test_gitignore_protection_env() {
@@ -23,30 +22,20 @@ fn test_gitignore_protection_env() {
]; ];
for pattern in required_patterns { for pattern in required_patterns {
let has_exact = gitignore.contains(&pattern); let has_wildcard = gitignore.contains("*.env.local") || gitignore.contains(".env.local");
let has_wildcard = gitignore.contains(&format!("*.env.local"))
|| gitignore.contains(&format!(".env.local"));
let has_env = gitignore.contains("*.env") || gitignore.contains(".env"); let has_env = gitignore.contains("*.env") || gitignore.contains(".env");
// For .env.local, either .env.local or *.env.local is acceptable
let is_env_local = pattern == "*.env.local" || pattern == ".env.local"; let is_env_local = pattern == "*.env.local" || pattern == ".env.local";
if is_env_local { if is_env_local {
assert!( assert!(
has_wildcard, has_wildcard,
".gitignore must contain pattern '*.env.local' or '.env.local' to protect secrets", ".gitignore must contain pattern '*.env.local' or '.env.local' to protect secrets",
); );
} else if pattern == ".env.production" || pattern == ".env.secret" { } else if pattern == ".env.production"
assert!( || pattern == ".env.secret"
gitignore.contains(pattern), || pattern == "*.env"
".gitignore must contain pattern '{}' to protect secrets", || pattern == ".env"
pattern {
);
} else if pattern == "*.env" {
assert!(
has_env,
".gitignore must contain pattern '*.env' or '.env' to protect secrets",
);
} else if pattern == ".env" {
assert!( assert!(
has_env, has_env,
".gitignore must contain pattern '*.env' or '.env' to protect secrets", ".gitignore must contain pattern '*.env' or '.env' to protect secrets",
@@ -65,7 +54,6 @@ fn test_gitignore_protection_env() {
#[test] #[test]
fn test_no_private_key_in_git() { fn test_no_private_key_in_git() {
// Check that .gitignore includes private_key.pem
let gitignore = match fs::read_to_string(".gitignore") { let gitignore = match fs::read_to_string(".gitignore") {
Ok(c) => c, Ok(c) => c,
Err(e) => { Err(e) => {
@@ -88,20 +76,17 @@ fn test_no_private_key_in_git() {
".gitignore must block chiave_privata.key" ".gitignore must block chiave_privata.key"
); );
// Check that no private key files are tracked by git
let output = std::process::Command::new("git") let output = std::process::Command::new("git")
.args(&["ls-files", "*.pem", "*.key"]) .args(["ls-files", "*.pem", "*.key"])
.output() .output()
.expect("Failed to run git ls-files"); .expect("Failed to run git ls-files");
let tracked_keys = String::from_utf8(output.stdout).unwrap(); let tracked_keys = String::from_utf8(output.stdout).unwrap();
let tracked_keys: Vec<&str> = tracked_keys.lines().collect(); let tracked_keys: Vec<&str> = tracked_keys.lines().collect();
// Only non-empty entries and only public_key.pem should be tracked
for tracked in tracked_keys.iter().filter(|s| !s.is_empty()) { for tracked in tracked_keys.iter().filter(|s| !s.is_empty()) {
if !tracked.contains("public_key.pem") { if !tracked.contains("public_key.pem") {
assert!( panic!(
false,
"Private key file is tracked by git: {}. Remove it with git rm --cached", "Private key file is tracked by git: {}. Remove it with git rm --cached",
tracked tracked
); );
@@ -113,38 +98,34 @@ fn test_no_private_key_in_git() {
#[test] #[test]
fn test_no_token_in_source_files() { fn test_no_token_in_source_files() {
// Scan source files for hardcoded tokens
let mut found_issues = Vec::new(); let mut found_issues = Vec::new();
// Scan .sh files for hardcoded 40-char hex strings
for entry in fs::read_dir(".").unwrap().filter_map(|e| e.ok()) { for entry in fs::read_dir(".").unwrap().filter_map(|e| e.ok()) {
let path = entry.path(); let path = entry.path();
if !path.is_file() { if !path.is_file() {
continue; continue;
} }
if let Some(ext) = path.extension() { if let Some(ext) = path.extension()
if ext == "sh" { && ext == "sh"
{
let content = fs::read_to_string(&path).unwrap(); let content = fs::read_to_string(&path).unwrap();
for (line_num, line) in content.lines().enumerate() { for (line_num, line) in content.lines().enumerate() {
// Skip comments and example/template files if line.trim().starts_with('#')
if line.trim().starts_with("#")
|| line.to_lowercase().contains("example") || line.to_lowercase().contains("example")
|| line.to_lowercase().contains("template") || line.to_lowercase().contains("template")
{ {
continue; continue;
} }
// Check for 40-64 hex chars that could be API tokens (not in .env.example comments)
if line.trim().len() >= 40 { if line.trim().len() >= 40 {
let hex_chars = line let hex_chars: Vec<_> = line
.trim() .trim()
.chars() .chars()
.filter(|c| c.is_ascii_hexdigit()) .filter(|c| c.is_ascii_hexdigit())
.collect::<Vec<_>>(); .collect();
if hex_chars.len() >= 40 && hex_chars.len() <= 64 { if (40..=64).contains(&hex_chars.len())
// Check if it looks like it's part of a TOKEN assignment && (line.to_lowercase().contains("token")
if line.to_lowercase().contains("token")
|| line.to_lowercase().contains("api") || line.to_lowercase().contains("api")
|| line.to_lowercase().contains("secret") || line.to_lowercase().contains("secret"))
{ {
found_issues.push(format!( found_issues.push(format!(
"Potential hardcoded token in {}: line {}: {}", "Potential hardcoded token in {}: line {}: {}",
@@ -157,16 +138,13 @@ fn test_no_token_in_source_files() {
} }
} }
} }
}
}
if !found_issues.is_empty() { if !found_issues.is_empty() {
println!("FAIL: Found potential hardcoded tokens:"); println!("FAIL: Found potential hardcoded tokens:");
for issue in &found_issues { for issue in &found_issues {
println!(" {}", issue); println!(" {}", issue);
} }
assert!( panic!(
false,
"Found potential hardcoded tokens in shell scripts: {:?}", "Found potential hardcoded tokens in shell scripts: {:?}",
found_issues found_issues
); );