7 Commits

Author SHA1 Message Date
6690343aad refactor: unify client IP extraction into extract_real_ip
Replace extract_client_ip (manual header parsing without proxy
verification) and is_valid_ip with a single extract_real_ip function
that uses the same proxy-checking logic as RealIpKeyExtractor.

- If peer IP == trusted proxy: use realip_remote_addr() (Forwarded/
  X-Forwarded-For) — safe because peer was verified
- Otherwise: use peer_addr() directly (local/direct connection)

echo_info now extracts the trusted proxy from web::Data<IpAddr>
and passes it to extract_real_ip, ensuring consistent IP resolution
across rate limiting and address derivation.
2026-08-19 03:13:01 -04:00
c957dfff6b docs: update rate limiting docs for RealIpKeyExtractor and trusted proxy
- Correct per-endpoint rate limit tables to reflect actual single global config
- Document RealIpKeyExtractor behavior (X-Real-IP / X-Forwarded-For behind proxy)
- Add BAL_SERVER_TRUSTED_PROXY env var to all relevant docs
- Update security audit and modules detail with proxy-aware rate limiting
2026-08-19 02:57:57 -04:00
8e5b921771 fix: rate limit per real client IP behind reverse proxy
Replace PeerIpKeyExtractor with custom RealIpKeyExtractor that extracts
the real client IP from X-Real-IP/X-Forwarded-For headers when the
request comes from a trusted proxy (default 127.0.0.1).

This fixes rate limiting being applied to the proxy IP instead of
individual clients when deployed behind Nginx.
2026-08-19 02:54:46 -04:00
f106beeb62 fix: correct locktime threshold, PG stats SUM on TEXT, PG locktime width, and translate Italian error strings
- Bug 1: LOCKTIME_THRESHOLD was 5_000_000 instead of BIP-65 value 500_000_000
- Bug 2: PostgreSQL SUM(our_fees) failed on TEXT column; cast to BIGINT
- Bug 3: PostgreSQL locktime INTEGER (i32) widened to BIGINT (i64) with idempotent ALTER
- Translate 4 Italian error strings in xpub.rs to English
2026-08-19 02:22:14 -04:00
5e18a2e06c chore: remove public_key.pem and .gitsecret/ from repo 2026-08-18 08:39:55 -04:00
7f62ffaf25 chore: clean up .gitignore and remove tracked files that should not be in the repo
Removed from tracking:
- Release binaries and tarballs (releases/, bal-server-0.2.2-*)
- Environment files with secrets (bal-server.env)
- Service files (bitcoind.service, tbitcoind.service)
- Test data (invalid_txs, valid_txs, test)
- Utility scripts (sendtx.sh, lib.sh, generate_random_ascii.sh, etc.)
- Archive (contrib.tar.gz)
- Scratch files (update, update_codebase.txt, src/xpub.rs2)

NOTE: The .env file with Gitea token was never tracked (already in .gitignore).
Private keys (chiave_privata.key, ec.key, private_key.pem, privkey.pem)
were also never tracked. Rotate the Gitea token if it was ever shared.
2026-08-18 03:24:18 -04:00
8f764f06b2 fix: resolve PostgreSQL type mismatches and add willexecutor debug logging
- Fix get_next_address_index: use try_get::<i32> for PG SERIAL/INTEGER columns
- Fix search_tx: use try_get::<i32> for PG status column
- Fix execute_insert: parse locktime (String→i64) and in_vout (String→i32)
  before binding to PG INTEGER columns
- Fix execute_insert: bind tbl_out vout as i32, amount as String for PG
- Fix save_new_address: cast xpub i64 to i32 for PG INTEGER column
- Fix get_pending_txs: cast i64 bind params to i32, use try_get::<i32> for reads
- Fix get_stats: use try_get::<i32> for all numeric PG INTEGER columns
- Add trace logging in parse_request_transactions for xpub address matching
- Add trace logging in get_all_addresses_by_xpub for query debugging
2026-08-18 03:14:03 -04:00
31 changed files with 3420 additions and 1296 deletions

72
.gitignore vendored
View File

@@ -1,39 +1,51 @@
.gitsecret/keys/random_seed # Secrets and environment files - NEVER commit
!*.secret
# Environment files - NEVER commit tokens or secrets
*.env
.env .env
.env.local .env.*
.env.production !.env.example
.env.secret
# Private keys - NEVER commit to git
# Only public_key.pem should be tracked (if needed)
*.pem *.pem
!public_key.pem !public_key.pem
data/*.pem
!data/public_key.pem
*.key *.key
!*.secret
private_key.pem
privkey.pem
ec.key
chiave_privata.key
# Other sensitive files # Databases
bal.db *.db
.bal.db *.db-shm
download_bal_db.sh *.db-wal
# IDE files
*.swp
*.swo
# Rust build artifacts # Rust build artifacts
/target /target
Cargo.lock
!lib/ # Release artifacts
!contrib/ /releases/
!src/ *.tar.gz
*.tar.gz.asc
*.tar.gz.sig
*.tar.gz.sha256
bal-server-*/
# Deployment configs (keep in deploy scripts, not in repo)
bal-server.env
bal-pusher.env
bitcoind.service
tbitcoind.service
# Test data and scratch files
invalid_txs
valid_txs
test
update
update_codebase.txt
# Utility scripts (keep local, not in repo)
sendtx.sh
lib.sh
generate_random_ascii.sh
start_postgres_docker.sh
make_release.sh make_release.sh
download_bal_db.sh
# Misc
*.swp
*.swo
src/xpub.rs2
public_key.pem
.gitsecret/

715
Cargo.lock generated

File diff suppressed because it is too large Load Diff

View File

@@ -20,7 +20,7 @@ hex = { version = "0.4.3" }
log = { version = "0.4.21" } log = { version = "0.4.21" }
serde = { version = "1.0.152", features = ["derive"] } serde = { version = "1.0.152", features = ["derive"] }
serde_json = { version = "1.0.116" } serde_json = { version = "1.0.116" }
sqlite = { version = "0.34.0" } sqlx = { version = "0.8", features = ["runtime-tokio", "sqlite", "postgres", "chrono"] }
regex = { version = "1.10.4" } regex = { version = "1.10.4" }
tokio = { version = "1", features = ["rt", "net","macros","rt-multi-thread"] } tokio = { version = "1", features = ["rt", "net","macros","rt-multi-thread"] }
url = { version = "2" } url = { version = "2" }

View File

@@ -112,6 +112,7 @@ The `bal-server` application can be configured using environment variables.
| `BAL_SERVER_ACTIX_DEFAULT_BURST` | Rate limit: default burst size. | `100` | | `BAL_SERVER_ACTIX_DEFAULT_BURST` | Rate limit: default burst size. | `100` |
| `BAL_SERVER_ACTIX_WORKERS` | Number of Actix worker threads. | `4` | | `BAL_SERVER_ACTIX_WORKERS` | Number of Actix worker threads. | `4` |
| `BAL_SERVER_ACTIX_MAX_CONNECTIONS` | Maximum concurrent connections. | `100` | | `BAL_SERVER_ACTIX_MAX_CONNECTIONS` | Maximum concurrent connections. | `100` |
| `BAL_SERVER_TRUSTED_PROXY` | Trusted reverse proxy IP for rate-limiting client identification. | `127.0.0.1` |
--- ---

View File

@@ -1,30 +0,0 @@
RUST_LOG=info
BAL_SERVER_DB_FILE="/home/bal/bal.db"
BAL_SERVER_INFO="BAL server test willexecutor"
# !!! WARNING: Never bind to 0.0.0.0 in production. Use 127.0.0.1 and place Nginx with TLS in front.
BAL_SERVER_BIND_ADDRESS=127.0.0.1
BAL_SERVER_BIND_PORT=9133
BAL_SERVER_BITCOIN_ADDRESS="your bitcoin or xpub to recive payments here"
BAL_SERVER_BITCOIN_FIXED_FEE=50000
BAL_SERVER_PUB_KEY_PATH="/home/bal/public_key.pem"
BAL_SERVER_REGTEST_ADDRESS="vpub5UhLrYG1qQjnJhvJgBdqgpznyH11mxW9hwBYxf3KhfdjiupCFPUVDvgwpeZ9Wj5YUJXjKjXjy7DSbJNBW1sXbKwARiaphm1UjHYy3mKvTG4"
BAL_SERVER_REGTEST_FEE=5000
#BAL_SERVER_TESTNET_ADDRESS=
#BAL_SERVER_TESTNET_FEE=100000
#BAL_SERVER_SIGNET_ADDRESS=
#BAL_SERVER_SIGNET_FEE=100000
# Actix Web DoS Protection Settings (added with migration to Actix Web)
BAL_SERVER_ACTIX_MAX_BODY_SIZE=1048576
BAL_SERVER_ACTIX_TIMEOUT_SECS=5
BAL_SERVER_ACTIX_PUSHTXS_PER_SEC=1
BAL_SERVER_ACTIX_PUSHTXS_BURST=3
BAL_SERVER_ACTIX_SEARCHTX_PER_SEC=5
BAL_SERVER_ACTIX_SEARCHTX_BURST=10
BAL_SERVER_ACTIX_INFO_PER_SEC=20
BAL_SERVER_ACTIX_INFO_BURST=30
BAL_SERVER_ACTIX_DEFAULT_PER_SEC=50
BAL_SERVER_ACTIX_DEFAULT_BURST=100
BAL_SERVER_ACTIX_WORKERS=4
BAL_SERVER_ACTIX_MAX_CONNECTIONS=100

40
bal-server.sh Normal file → Executable file
View File

@@ -1,6 +1,12 @@
#!/usr/bin/env bash
set -e
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
. "$SCRIPT_DIR/lib.sh"
WORKING_DIR=$(pwd) WORKING_DIR=$(pwd)
if [ ! -f "$WORKING_DIR/public_key.pem" ]; then if [ ! -f "$WORKING_DIR/public_key.pem" ]; then
echo "creating keypairs" echo_i "creating keypairs"
openssl genpkey -algorithm ED25519 -out private_key.pem openssl genpkey -algorithm ED25519 -out private_key.pem
openssl pkey -in private_key.pem -pubout -out public_key.pem openssl pkey -in private_key.pem -pubout -out public_key.pem
fi fi
@@ -11,7 +17,7 @@ export BAL_SERVER_INFO="BAL devel willexecutor server"
export BAL_SERVER_BIND_ADDRESS="127.0.0.1" export BAL_SERVER_BIND_ADDRESS="127.0.0.1"
export BAL_SERVER_BIND_PORT=9133 export BAL_SERVER_BIND_PORT=9133
export BAL_SERVER_PUB_KEY_PATH="$WORKING_DIR/public_key.pem" export BAL_SERVER_PUB_KEY_PATH="$WORKING_DIR/public_key.pem"
export BAL_SERVER_EXPOSE_STATS=true; export BAL_SERVER_EXPOSE_STATS=true
#export BAL_SERVER_BITCOIN_ADDRESS="your bitcoin address or xpub to recive payments here" #export BAL_SERVER_BITCOIN_ADDRESS="your bitcoin address or xpub to recive payments here"
#export BAL_SERVER_BITCOIN_FIXED_FEE=50000 #export BAL_SERVER_BITCOIN_FIXED_FEE=50000
@@ -22,4 +28,34 @@ export BAL_SERVER_REGTEST_FIXED_FEE=1000
#export BAL_SERVER_TESTNET_FEE=100000 #export BAL_SERVER_TESTNET_FEE=100000
#export BAL_SERVER_SIGNET_ADDRESS= #export BAL_SERVER_SIGNET_ADDRESS=
#export BAL_SERVER_SIGNET_FEE=100000 #export BAL_SERVER_SIGNET_FEE=100000
export BAL_SERVER_DB_BACKEND="postgresql"
if [ "$BAL_SERVER_DB_BACKEND" = "postgresql" ]; then
PG_CONTAINER="bal-pg"
PG_DSN="${BAL_SERVER_PG_DSN:-postgres://bal:balpass@localhost:5432/baldb}"
docker rm -f "$PG_CONTAINER" 2>/dev/null || true
echo_i "Starting PostgreSQL container..."
docker run -d --name "$PG_CONTAINER" \
-e POSTGRES_USER=bal \
-e POSTGRES_PASSWORD=balpass \
-e POSTGRES_DB=baldb \
-p 5432:5432 postgres:16-alpine
echo_i "Waiting for PostgreSQL..."
until docker exec "$PG_CONTAINER" pg_isready -U bal >/dev/null 2>&1; do
sleep 1
done
echo_i "PostgreSQL ready at $PG_DSN"
export BAL_SERVER_PG_DSN="$PG_DSN"
cleanup() {
echo_i "Stopping PostgreSQL container..."
docker stop "$PG_CONTAINER" >/dev/null 2>&1
docker rm "$PG_CONTAINER" >/dev/null 2>&1
}
trap cleanup EXIT
fi
cargo run --bin=bal-server cargo run --bin=bal-server

0
contrib/.config_init Normal file
View File

325
contrib/deploy.sh Executable file
View File

@@ -0,0 +1,325 @@
#!/bin/bash
set -euo pipefail
# deploy.sh — Deploy the latest bal-server release on a remote server.
#
# Downloads, verifies (SHA-256 + GPG), and installs the release entirely
# on the remote server via SSH. No file transfer (scp/rsync) needed.
#
# Usage:
# ./contrib/deploy.sh # deploy latest
# ./contrib/deploy.sh --dry-run # show plan without executing
# ./contrib/deploy.sh v0.3.1 # deploy a specific tag
# DEPLOY_DRY_RUN=1 ./contrib/deploy.sh # same as --dry-run
#
# Prerequisites:
# - ssh key access to debian@bitcoin-after.life:47081
# - The server must have: curl, jq, sha256sum, gpg, systemctl
#
# Services managed:
# bal-server, bal-pusher, tbal-pusher, t4bal-pusher
REMOTE_USER="debian"
REMOTE_HOST="bitcoin-after.life"
REMOTE_PORT="47081"
SSH_OPTS="-p ${REMOTE_PORT} -o ConnectTimeout=15 -o BatchMode=yes"
GITEA_API="https://bitcoin-after.life/gitea/api/v1/repos/bitcoinafterlife/bal-server"
GPG_SIGNER="svatantrya@bitcoin-after.life"
GPG_KEY_ID="A847D004DB91610711CA6A0DFE756706E833E0D1"
SERVICES=("bal-server" "bal-pusher" "tbal-pusher" "t4bal-pusher")
INSTALL_DIR="/usr/local/bin"
# ── Parse arguments ─────────────────────────────────────────────────
DRY_RUN=0
DEPLOY_TAG=""
for arg in "$@"; do
case "$arg" in
--dry-run) DRY_RUN=1 ;;
--help|-h)
echo "Usage: $0 [--dry-run] [TAG]"
echo " TAG Deploy a specific release tag (e.g. v0.3.1). Default: latest."
exit 0
;;
*) DEPLOY_TAG="$arg" ;;
esac
done
[[ "${DEPLOY_DRY_RUN:-}" == "1" ]] && DRY_RUN=1
# ── Helpers ──────────────────────────────────────────────────────────
info() { echo -e "\033[1m==> $1\033[0m"; }
ok() { echo -e "\033[32;1m ✔ $1\033[0m"; }
warn() { echo -e "\033[33;1m ⚠ $1\033[0m"; }
err() { echo -e "\033[31;1m ✖ $1\033[0m"; }
die() { err "$1"; exit "${2:-1}"; }
remote_exec() {
if [[ $DRY_RUN -eq 1 ]]; then
info "[dry-run] remote: $1"
return 0
fi
ssh $SSH_OPTS "${REMOTE_USER}@${REMOTE_HOST}" "$@"
}
# ── Step 0: Verify SSH connectivity ────────────────────────────────
info "Verifying SSH connection to ${REMOTE_HOST} ..."
if ! remote_exec "echo ok" >/dev/null 2>&1; then
die "Cannot connect via SSH. Check your key and network."
fi
ok "SSH connection OK"
# ── Step 1: Resolve release to deploy ──────────────────────────────
if [[ -n "$DEPLOY_TAG" ]]; then
TAG="$DEPLOY_TAG"
info "Deploying explicit tag: $TAG"
RELEASE_JSON=$(curl -sfL "${GITEA_API}/releases/tags/${TAG}") \
|| die "Release $TAG not found at $GITEA_API/releases/tags/$TAG"
else
info "Fetching latest release metadata ..."
RELEASE_JSON=$(curl -sfL "${GITEA_API}/releases/latest") \
|| die "Failed to fetch latest release from $GITEA_API"
TAG=$(echo "$RELEASE_JSON" | jq -r '.tag_name // empty')
fi
RELEASE_NAME=$(echo "$RELEASE_JSON" | jq -r '.name // empty')
if [[ -z "$TAG" ]]; then
die "Could not determine release tag"
fi
info "Release: $RELEASE_NAME (tag: $TAG)"
TARBALL_URL=$(echo "$RELEASE_JSON" | jq -r \
'.assets[] | select(.name | test("\\.tar\\.gz$")) | .browser_download_url' | head -1)
[[ -z "$TARBALL_URL" ]] && die "No .tar.gz asset found in release $TAG"
ASSET_NAME=$(basename "$TARBALL_URL")
info "Asset: $ASSET_NAME"
# Collect sidecar URLs
SHA256_URL="${TARBALL_URL}.sha256"
SIG_URL="${TARBALL_URL}.sig"
ASC_URL="${TARBALL_URL}.asc"
# ── Step 2: Check current version on server ─────────────────────────
info "Checking current version on server ..."
CURRENT=$(remote_exec "$INSTALL_DIR/bal-server --version" 2>/dev/null \
|| remote_exec "strings $INSTALL_DIR/bal-server 2>/dev/null | head -1" 2>/dev/null \
|| echo "unknown")
info "Current version on server: $CURRENT"
# ── Step 2b: Export GPG key for remote import ────────────────────────
info "Exporting GPG public key for $GPG_SIGNER ..."
GPG_PUBKEY_B64=""
if gpg --list-keys "$GPG_SIGNER" &>/dev/null; then
GPG_PUBKEY_B64=$(gpg --armor --export "$GPG_SIGNER" | base64 -w 0)
ok "GPG public key exported (${#GPG_PUBKEY_B64} chars base64)"
else
warn "GPG key for $GPG_SIGNER not found locally — remote import will try keyservers"
fi
# ── Step 3: Deploy on remote ───────────────────────────────────────
# Build a single shell script that runs entirely on the server.
# This avoids scp/rsync issues and ensures atomic, auditable deployment.
DEPLOY_SCRIPT=$(cat << 'REMOTE_EOF'
#!/bin/bash
set -euo pipefail
TAG="__TAG__"
ASSET_NAME="__ASSET_NAME__"
TARBALL_URL="__TARBALL_URL__"
SHA256_URL="__SHA256_URL__"
SIG_URL="__SIG_URL__"
ASC_URL="__ASC_URL__"
GPG_SIGNER="__GPG_SIGNER__"
GPG_KEY_ID="__GPG_KEY_ID__"
INSTALL_DIR="__INSTALL_DIR__"
SERVICES="__SERVICES__"
DRY_RUN="__DRY_RUN__"
info() { echo -e "\033[1m==> $1\033[0m"; }
ok() { echo -e "\033[32;1m ✔ $1\033[0m"; }
warn() { echo -e "\033[33;1m ⚠ $1\033[0m"; }
err() { echo -e "\033[31;1m ✖ $1\033[0m"; }
die() { err "$1"; exit "${2:-1}"; }
WORKDIR=$(mktemp -d /tmp/bal-deploy.XXXXXX)
trap 'rm -rf "$WORKDIR"' EXIT
# ── Ensure GPG is available ─────────────────────────────────────────
if ! command -v gpg &>/dev/null; then
info "Installing gnupg ..."
sudo apt-get update -qq && sudo apt-get install -y -qq gnupg
fi
# ── Import GPG key ──────────────────────────────────────────────────
info "Importing GPG key $GPG_KEY_ID ..."
if ! gpg --list-keys "$GPG_SIGNER" &>/dev/null; then
imported=0
# Try embedded key first (base64-encoded, pushed from deployer)
if [[ -n "__GPG_PUBKEY_B64__" ]]; then
echo "__GPG_PUBKEY_B64__" | base64 -d | gpg --batch --import 2>/dev/null && imported=1 && ok "Key imported from deployer"
fi
# Fallback to keyservers
if [[ $imported -eq 0 ]]; then
for ks in keyserver.ubuntu.com keys.openpgp.org pgp.mit.edu; do
if gpg --batch --keyserver "$ks" --recv-keys "$GPG_KEY_ID" 2>/dev/null; then
ok "Key imported from $ks"
imported=1
break
fi
done
fi
if [[ $imported -eq 0 ]]; then
die "Cannot import GPG key $GPG_KEY_ID. Import it manually and re-run."
fi
else
ok "GPG key already present"
fi
# ── Download release assets ─────────────────────────────────────────
info "Downloading $ASSET_NAME ..."
curl -sfL -o "$WORKDIR/$ASSET_NAME" "$TARBALL_URL" || die "Download failed"
for sidecar in "sha256:$SHA256_URL:.sha256" "sig:$SIG_URL:.sig" "asc:$ASC_URL:.asc"; do
IFS=: read -r label url suffix <<< "$sidecar"
if curl -sfL -o "$WORKDIR/$(basename "$ASSET_NAME")$suffix" "$url" 2>/dev/null; then
ok "Downloaded $ASSET_NAME$suffix"
else
warn "$label file not available — skipping"
fi
done
# ── Verify SHA-256 ──────────────────────────────────────────────────
SHA_FILE="$WORKDIR/${ASSET_NAME}.sha256"
if [[ -f "$SHA_FILE" ]]; then
info "Verifying SHA-256 checksum ..."
(cd "$WORKDIR" && sha256sum -c "$SHA_FILE") || die "SHA-256 verification FAILED"
ok "SHA-256 OK"
else
warn "No .sha256 file — skipping checksum"
fi
# ── Verify GPG signature ────────────────────────────────────────────
SIG_FILE="$WORKDIR/${ASSET_NAME}.sig"
ASC_FILE="$WORKDIR/${ASSET_NAME}.asc"
verified=0
if [[ -f "$SIG_FILE" ]]; then
info "Verifying GPG signature (binary) ..."
gpg --batch --verify "$SIG_FILE" "$WORKDIR/$ASSET_NAME" 2>&1 && verified=1
fi
if [[ $verified -eq 0 ]] && [[ -f "$ASC_FILE" ]]; then
info "Verifying GPG signature (ASCII-armored) ..."
gpg --batch --verify "$ASC_FILE" "$WORKDIR/$ASSET_NAME" 2>&1 && verified=1
fi
if [[ $verified -eq 0 ]]; then
warn "No GPG signature available — skipping signature verification"
fi
# ── Extract ──────────────────────────────────────────────────────────
info "Extracting tarball ..."
tar -xzf "$WORKDIR/$ASSET_NAME" -C "$WORKDIR"
EXTRACTED="$WORKDIR/$(basename "$ASSET_NAME" .tar.gz)"
for bin in bal-server bal-pusher; do
if [[ ! -f "$EXTRACTED/$bin" ]]; then
die "Binary '$bin' not found in archive"
fi
ok "Found $bin ($(stat -c%s "$EXTRACTED/$bin") bytes)"
done
# ── Stop services ────────────────────────────────────────────────────
info "Stopping services ..."
IFS=',' read -ra SVC_LIST <<< "$SERVICES"
for svc in "${SVC_LIST[@]}"; do
svc=$(echo "$svc" | xargs) # trim whitespace
if systemctl is-active --quiet "$svc" 2>/dev/null; then
sudo systemctl stop "$svc" || warn "Failed to stop $svc"
ok "Stopped $svc"
else
warn "$svc is not running"
fi
done
# ── Backup current binaries ─────────────────────────────────────────
BACKUP_DIR="$WORKDIR/backup"
mkdir -p "$BACKUP_DIR"
for bin in bal-server bal-pusher; do
if [[ -f "$INSTALL_DIR/$bin" ]]; then
cp "$INSTALL_DIR/$bin" "$BACKUP_DIR/$bin" 2>/dev/null || true
fi
done
ok "Current binaries backed up"
# ── Install new binaries ────────────────────────────────────────────
info "Installing binaries to $INSTALL_DIR ..."
sudo install -m 0755 -o root -g root "$EXTRACTED/bal-server" "$EXTRACTED/bal-pusher" "$INSTALL_DIR/"
ok "Installed bal-server and bal-pusher"
# ── Restart services ────────────────────────────────────────────────
info "Restarting services ..."
sudo systemctl daemon-reload
for svc in "${SVC_LIST[@]}"; do
svc=$(echo "$svc" | xargs)
if systemctl is-enabled --quiet "$svc" 2>/dev/null; then
sudo systemctl restart "$svc" || warn "Failed to restart $svc"
ok "Restarted $svc"
else
warn "$svc is not enabled — skipping restart"
fi
done
# ── Verify ──────────────────────────────────────────────────────────
echo ""
info "Checking service status ..."
sleep 2
for svc in "${SVC_LIST[@]}"; do
svc=$(echo "$svc" | xargs)
status=$(systemctl is-active "$svc" 2>/dev/null || echo "inactive")
if [[ "$status" == "active" ]]; then
ok "$svc: active"
else
warn "$svc: $status"
fi
done
echo ""
ok "Deploy of $TAG completed successfully!"
REMOTE_EOF
)
# ── Substitute variables into the remote script ─────────────────────
DEPLOY_SCRIPT="${DEPLOY_SCRIPT//__TAG__/$TAG}"
DEPLOY_SCRIPT="${DEPLOY_SCRIPT//__ASSET_NAME__/$ASSET_NAME}"
DEPLOY_SCRIPT="${DEPLOY_SCRIPT//__TARBALL_URL__/$TARBALL_URL}"
DEPLOY_SCRIPT="${DEPLOY_SCRIPT//__SHA256_URL__/$SHA256_URL}"
DEPLOY_SCRIPT="${DEPLOY_SCRIPT//__SIG_URL__/$SIG_URL}"
DEPLOY_SCRIPT="${DEPLOY_SCRIPT//__ASC_URL__/$ASC_URL}"
DEPLOY_SCRIPT="${DEPLOY_SCRIPT//__GPG_SIGNER__/$GPG_SIGNER}"
DEPLOY_SCRIPT="${DEPLOY_SCRIPT//__GPG_KEY_ID__/$GPG_KEY_ID}"
DEPLOY_SCRIPT="${DEPLOY_SCRIPT//__INSTALL_DIR__/$INSTALL_DIR}"
DEPLOY_SCRIPT="${DEPLOY_SCRIPT//__DRY_RUN__/$DRY_RUN}"
# Join SERVICES array into comma-separated string for the remote script
SERVICES_STR=$(IFS=,; echo "${SERVICES[*]}")
DEPLOY_SCRIPT="${DEPLOY_SCRIPT//__SERVICES__/$SERVICES_STR}"
DEPLOY_SCRIPT="${DEPLOY_SCRIPT//__GPG_PUBKEY_B64__/${GPG_PUBKEY_B64:-}}"
# ── Execute ──────────────────────────────────────────────────────────
if [[ $DRY_RUN -eq 1 ]]; then
info "=== DRY RUN — would execute the following on remote: ==="
echo "$DEPLOY_SCRIPT"
echo "=== end dry run ==="
exit 0
fi
REMOTE_SCRIPT="/tmp/bal-deploy-$(date +%s).sh"
info "Uploading deploy script to remote ..."
ssh $SSH_OPTS "${REMOTE_USER}@${REMOTE_HOST}" "cat > '$REMOTE_SCRIPT'" <<< "$DEPLOY_SCRIPT" \
|| die "Failed to upload deploy script"
info "Executing deploy on remote server ..."
ssh $SSH_OPTS "${REMOTE_USER}@${REMOTE_HOST}" "chmod +x '$REMOTE_SCRIPT' && bash '$REMOTE_SCRIPT'; rc=\$?; rm -f '$REMOTE_SCRIPT'; exit \$rc" \
|| die "Deploy script failed on remote (exit $?)"
echo ""
ok "Deploy finished!"

19
contrib/lib.sh Normal file
View File

@@ -0,0 +1,19 @@
echo_i() {
echo -e "\033[1m==> $1\033[0m"
}
echo_e() {
echo -e "\033[31;1m$1\033[0m"
}
echo_s() {
echo -e "\033[32;1m$1\033[0m"
}
echo_w() {
echo -e "\033[33;1m$1\033[0m"
}
alias echo_i=echo_i
alias echo_e=echo_e
alias echo_s=echo_s
alias echo_w=echo_w

3
docker/s6-rc.d/bal-pusher/run Executable file
View File

@@ -0,0 +1,3 @@
#!/command/with-contenv sh
exec s6-setsid logutil-service bal-pusher \
/usr/local/bin/bal-pusher

View File

@@ -0,0 +1 @@
longrun

3
docker/s6-rc.d/bal-server/run Executable file
View File

@@ -0,0 +1,3 @@
#!/command/with-contenv sh
exec s6-setsid logutil-service bal-server \
/usr/local/bin/bal-server

View File

@@ -0,0 +1 @@
longrun

View File

@@ -117,7 +117,7 @@ The main application binary that provides an async HTTP server.
### Architecture ### Architecture
- **Runtime:** `actix-web 4.9.0` with `actix-rt` (`#[actix_web::main]`). - **Runtime:** `actix-web 4.9.0` with `actix-rt` (`#[actix_web::main]`).
- **Rate Limiting:** `actix-governor` middleware with token-bucket algorithm per endpoint. - **Rate Limiting:** `actix-governor` middleware with token-bucket algorithm. Uses `RealIpKeyExtractor` to identify clients by real IP behind reverse proxy (via `X-Real-IP` / `X-Forwarded-For` headers).
- **Response Compression:** `actix_web::middleware::Compress`. - **Response Compression:** `actix_web::middleware::Compress`.
- **Request Logging:** `actix_web::middleware::Logger::default()`. - **Request Logging:** `actix_web::middleware::Logger::default()`.
- **Shared State:** `Arc<Mutex<Connection>>` for database access, `MyConfig` for configuration. - **Shared State:** `Arc<Mutex<Connection>>` for database access, `MyConfig` for configuration.
@@ -132,7 +132,7 @@ The main application binary that provides an async HTTP server.
- `address` (xpub or address), `fixed_fee` (sats), `xpub` (bool), `network` (bitcoin::Network), `name`, `enabled` - `address` (xpub or address), `fixed_fee` (sats), `xpub` (bool), `network` (bitcoin::Network), `name`, `enabled`
**`ActixConfig`** (server tuning): **`ActixConfig`** (server tuning):
- `max_body_size`, `timeout_secs`, per-endpoint rate limits (`pushtxs`, `searchtx`, `info`, `default`), `workers`, `max_connections` - `max_body_size`, `timeout_secs`, rate limits (`pushtxs` per sec/burst), `workers`, `max_connections`, `trusted_proxy`
### Key Routes ### Key Routes
| Method | Path | Handler | Description | | Method | Path | Handler | Description |

View File

@@ -10,16 +10,11 @@
### Rate Limiting ### Rate Limiting
All endpoints are rate-limited via `actix-governor` with a token-bucket algorithm. Defaults: All endpoints are rate-limited via `actix-governor` with a token-bucket algorithm. The rate limit key is the **real client IP address**, extracted from `X-Real-IP` / `X-Forwarded-For` headers when the request comes from a trusted proxy (default `127.0.0.1`, configurable via `BAL_SERVER_TRUSTED_PROXY`). Direct connections (non-proxy) use the TCP peer IP.
| Endpoint | Rate (req/s) | Burst | Default: 1 req/s with burst of 3 (configurable via `BAL_SERVER_ACTIX_PUSHTXS_PER_SEC` / `BAL_SERVER_ACTIX_PUSHTXS_BURST`).
|----------|-------------|-------|
| `POST /{network}/pushtxs` | 1 | 3 |
| `POST /searchtx` | 5 | 10 |
| `GET /{network}/info` | 20 | 30 |
| All others | 50 | 100 |
Rate limits are configurable via `BAL_SERVER_ACTIX_*` environment variables. When behind Nginx, ensure `proxy_set_header X-Real-IP $remote_addr` is set so the server can identify individual clients.
### `GET /` ### `GET /`
- **Description:** Returns a static identification string (default: "Will Executor Server"). - **Description:** Returns a static identification string (default: "Will Executor Server").

View File

@@ -40,8 +40,9 @@ Example: `BAL_SERVER_REGTEST_ADDRESS=tpub...`, `BAL_SERVER_BITCOIN_FIXED_FEE=500
| `BAL_SERVER_ACTIX_TIMEOUT_SECS` | `5` | Request timeout in seconds | | `BAL_SERVER_ACTIX_TIMEOUT_SECS` | `5` | Request timeout in seconds |
| `BAL_SERVER_ACTIX_WORKERS` | `4` | Number of actix-web worker threads | | `BAL_SERVER_ACTIX_WORKERS` | `4` | Number of actix-web worker threads |
| `BAL_SERVER_ACTIX_MAX_CONNECTIONS` | `100` | Maximum concurrent connections | | `BAL_SERVER_ACTIX_MAX_CONNECTIONS` | `100` | Maximum concurrent connections |
| `BAL_SERVER_ACTIX_PUSHTXS_PER_SEC` | `1` | Rate limit: pushtxs requests per second | | `BAL_SERVER_TRUSTED_PROXY` | `127.0.0.1` | Trusted reverse proxy IP for rate-limiting client identification |
| `BAL_SERVER_ACTIX_PUSHTXS_BURST` | `3` | Rate limit: pushtxs burst size | | `BAL_SERVER_ACTIX_PUSHTXS_PER_SEC` | `1` | Rate limit: requests per second (applied to all endpoints) |
| `BAL_SERVER_ACTIX_PUSHTXS_BURST` | `3` | Rate limit: burst size (applied to all endpoints) |
| `BAL_SERVER_ACTIX_SEARCHTX_PER_SEC` | `5` | Rate limit: searchtx requests per second | | `BAL_SERVER_ACTIX_SEARCHTX_PER_SEC` | `5` | Rate limit: searchtx requests per second |
| `BAL_SERVER_ACTIX_SEARCHTX_BURST` | `10` | Rate limit: searchtx burst size | | `BAL_SERVER_ACTIX_SEARCHTX_BURST` | `10` | Rate limit: searchtx burst size |
| `BAL_SERVER_ACTIX_INFO_PER_SEC` | `20` | Rate limit: info requests per second | | `BAL_SERVER_ACTIX_INFO_PER_SEC` | `20` | Rate limit: info requests per second |

View File

@@ -52,7 +52,7 @@
**Description:** All DoS vectors mitigated via actix-web migration. **Description:** All DoS vectors mitigated via actix-web migration.
**Mitigation Applied:** **Mitigation Applied:**
- Body size limit: `PayloadConfig::default().limit(max_body_size)` via `BAL_SERVER_ACTIX_MAX_BODY_SIZE` (default 1 MiB). - Body size limit: `PayloadConfig::default().limit(max_body_size)` via `BAL_SERVER_ACTIX_MAX_BODY_SIZE` (default 1 MiB).
- Rate limiting: `actix-governor` with token-bucket per endpoint (`BAL_SERVER_ACTIX_PUSHTXS_PER_SEC`/`BURST`). - Rate limiting: `actix-governor` with token-bucket per client IP (`BAL_SERVER_ACTIX_PUSHTXS_PER_SEC`/`BURST`). Uses `RealIpKeyExtractor` to extract real client IP from proxy headers.
- Connection limits: `workers(4)` and `max_connections(100)` via `BAL_SERVER_ACTIX_WORKERS`/`MAX_CONNECTIONS`. - Connection limits: `workers(4)` and `max_connections(100)` via `BAL_SERVER_ACTIX_WORKERS`/`MAX_CONNECTIONS`.
- Body timeout: configurable via `BAL_SERVER_ACTIX_TIMEOUT_SECS`. - Body timeout: configurable via `BAL_SERVER_ACTIX_TIMEOUT_SECS`.
- ZMQ timeout: `set_rcvtimeo(5000)` prevents infinite blocking. - ZMQ timeout: `set_rcvtimeo(5000)` prevents infinite blocking.
@@ -130,7 +130,7 @@
6. Use read-only filesystem for the server binary. 6. Use read-only filesystem for the server binary.
### Application-Level ### Application-Level
1. **Rate Limiting:** Implemented via `actix-governor` with per-endpoint token-bucket configuration. 1. **Rate Limiting:** Implemented via `actix-governor` with token-bucket per client IP. `RealIpKeyExtractor` identifies clients behind reverse proxy using `X-Real-IP` / `X-Forwarded-For` headers. Trusted proxy IP configurable via `BAL_SERVER_TRUSTED_PROXY`.
2. **Input Validation:** Network enum check, txid hex validation, body size limits. 2. **Input Validation:** Network enum check, txid hex validation, body size limits.
3. **HTTPS:** Via Nginx reverse proxy with Let's Encrypt. 3. **HTTPS:** Via Nginx reverse proxy with Let's Encrypt.
4. **WAL Mode:** Enabled with retry logic for concurrent access. 4. **WAL Mode:** Enabled with retry logic for concurrent access.

View File

@@ -10,7 +10,6 @@ use log::{debug, error, info, trace, warn};
use serde::Deserialize; use serde::Deserialize;
use serde::Serialize; use serde::Serialize;
use serde_json::json; use serde_json::json;
use sqlite::{Connection, Value};
use std::collections::HashMap; use std::collections::HashMap;
use std::env; use std::env;
use std::error::Error as StdError; use std::error::Error as StdError;
@@ -18,18 +17,21 @@ use std::str;
use std::{thread, time::Duration}; use std::{thread, time::Duration};
use zmq::{Context, Socket}; use zmq::{Context, Socket};
use bal_server::db::open_db; use bal_server::db::{calculate_and_upsert_stats, get_pending_txs, open_database};
use bal_server::validation::is_valid_welist_url; use bal_server::validation::is_valid_welist_url;
use base64::{Engine as _, engine::general_purpose}; use base64::{Engine as _, engine::general_purpose};
use reqwest::Client as rClient; use reqwest::Client as rClient;
use std::net::SocketAddr; use std::net::SocketAddr;
use url::Url; use url::Url;
const LOCKTIME_THRESHOLD: i64 = 5000000; // BIP-65: locktime values below this are block heights, at or above are UNIX timestamps.
const LOCKTIME_THRESHOLD: i64 = 500_000_000;
const VERSION: &str = env!("CARGO_PKG_VERSION"); const VERSION: &str = env!("CARGO_PKG_VERSION");
#[derive(Debug, Clone, Serialize, Deserialize)] #[derive(Debug, Clone, Serialize, Deserialize)]
struct MyConfig { struct MyConfig {
db_backend: String,
db_file: String, db_file: String,
pg_dsn: String,
bitcoin_dir: String, bitcoin_dir: String,
regtest: NetworkParams, regtest: NetworkParams,
testnet: NetworkParams, testnet: NetworkParams,
@@ -44,7 +46,9 @@ struct MyConfig {
impl Default for MyConfig { impl Default for MyConfig {
fn default() -> Self { fn default() -> Self {
MyConfig { MyConfig {
db_backend: env::var("BAL_PUSHER_DB_BACKEND").unwrap_or("sqlite".to_string()),
db_file: env::var("BAL_PUSHER_DB_FILE").unwrap_or("bal.db".to_string()), db_file: env::var("BAL_PUSHER_DB_FILE").unwrap_or("bal.db".to_string()),
pg_dsn: env::var("BAL_PUSHER_PG_DSN").unwrap_or_default(),
bitcoin_dir: env::var("BAL_PUSHER_BITCOIN_DIR").unwrap_or("".to_string()), bitcoin_dir: env::var("BAL_PUSHER_BITCOIN_DIR").unwrap_or("".to_string()),
regtest: get_network_params_default(Network::Regtest), regtest: get_network_params_default(Network::Regtest),
testnet: get_network_params_default(Network::Testnet), testnet: get_network_params_default(Network::Testnet),
@@ -206,114 +210,79 @@ async fn main_result(cfg: &MyConfig, network_params: &NetworkParams) -> Result<(
Ok((rpc, bcinfo)) => { Ok((rpc, bcinfo)) => {
info!("connected"); info!("connected");
info!("median time: {}", bcinfo.median_time); info!("median time: {}", bcinfo.median_time);
//info!("height time: {}",bcinfo.median_time);
info!("blocks: {}", bcinfo.blocks); info!("blocks: {}", bcinfo.blocks);
debug!("best block hash: {}", bcinfo.best_block_hash); debug!("best block hash: {}", bcinfo.best_block_hash);
let average_time = bcinfo.median_time; let average_time = bcinfo.median_time;
let db = match open_db(&cfg.db_file) {
Ok(c) => c, let connection_string = match cfg.db_backend.as_str() {
"sqlite" => cfg.db_file.clone(),
"postgresql" => cfg.pg_dsn.clone(),
other => {
error!("Unknown DB backend: {}", other);
return Ok(());
}
};
let db = match open_database(&cfg.db_backend, &connection_string).await {
Ok(pool) => pool,
Err(e) => { Err(e) => {
error!("Fatal: {}", e); error!("Fatal: {}", e);
std::process::exit(1); std::process::exit(1);
} }
}; };
info!("db open {}", &cfg.db_file); info!("db open {}", &connection_string);
let sqlquery = "SELECT * FROM tbl_tx WHERE network = :network AND status = :status AND ( locktime < :bestblock_height OR locktime > :locktime_threshold AND locktime < :bestblock_time);"; let pending_txs = match get_pending_txs(
let query_tx = match db.prepare(sqlquery) { &db,
Ok(q) => q.into_iter(), &network_params.db_field,
LOCKTIME_THRESHOLD,
bcinfo.blocks as i64,
average_time as i64,
)
.await
{
Ok(txs) => txs,
Err(e) => { Err(e) => {
warn!("tbl_tx not ready yet (tables may not exist): {}", e); warn!("Failed to query pending transactions: {}", e);
return Ok(()); return Ok(());
} }
}; };
trace!("query_tx: {}", sqlquery);
trace!(":locktime_threshold: {}", LOCKTIME_THRESHOLD);
trace!(":bestblock_time: {}", average_time);
trace!(":bestblock_height: {}", bcinfo.blocks);
trace!(":network: {}", network_params.db_field.clone());
trace!(":status: {}", 0);
//let query_tx = db.prepare("SELECT * FROM tbl_tx where status = :status").unwrap().into_iter();
let mut pushed_txs: Vec<String> = Vec::new(); let mut pushed_txs: Vec<String> = Vec::new();
let mut invalid_txs: std::collections::HashMap<String, String> = HashMap::new(); let mut invalid_txs: HashMap<String, String> = HashMap::new();
for row_result in match query_tx.bind::<&[(_, Value)]>(
&[ for row in &pending_txs {
(":locktime_threshold", LOCKTIME_THRESHOLD.into()), let txid = &row.txid;
(":bestblock_time", (average_time as i64).into()), let tx = &row.tx;
(":bestblock_height", (bcinfo.blocks as i64).into()), let locktime = row.locktime;
(":network", network_params.db_field.clone().into()),
(":status", 0.into()),
][..],
) {
Ok(bound) => bound,
Err(e) => {
error!("Failed to bind query parameters: {}", e);
return Ok(());
}
} {
let row = match row_result {
Ok(r) => r,
Err(e) => {
warn!("Failed to read row: {}", e);
continue;
}
};
let tx = row.read::<&str, _>("tx");
let txid = row.read::<&str, _>("txid");
let locktime = row.read::<i64, _>("locktime");
info!("to be pushed: {}: {}", txid, locktime); info!("to be pushed: {}: {}", txid, locktime);
match rpc.send_raw_transaction(tx) { match rpc.send_raw_transaction(tx.as_str()) {
Ok(o) => { Ok(o) => {
info!("tx: {} pusshata PUSHED\n{}", txid, o); info!("tx: {} pusshata PUSHED\n{}", txid, o);
pushed_txs.push(txid.to_string()); pushed_txs.push(txid.clone());
} }
Err(err) => { Err(err) => {
warn!("Error: {}\n{}", err, txid); warn!("Error: {}\n{}", err, txid);
//store err in invalid_txs invalid_txs.insert(txid.clone(), err.to_string());
invalid_txs.insert(txid.to_string(), err.to_string());
} }
}; };
} }
for txid in &pushed_txs { for txid in &pushed_txs {
let sql = "UPDATE tbl_tx SET status = 1 WHERE txid = ?"; if let Err(e) = bal_server::db::update_tx_status(&db, txid, 1, None).await {
match db.prepare(sql) { error!("Failed to update tx status: {}", e);
Ok(mut stmt) => {
if let Err(e) = stmt.bind((1, Value::String(txid.clone()))) {
error!("Failed to bind txid for status update: {}", e);
continue;
}
let _ = stmt.next();
}
Err(e) => {
error!("Failed to prepare status update: {}", e);
}
} }
} }
for (txid, txerr) in &invalid_txs { for (txid, txerr) in &invalid_txs {
let sql = "UPDATE tbl_tx SET status = 2, push_err = ? WHERE txid = ?"; if let Err(e) = bal_server::db::update_tx_status(&db, txid, 2, Some(txerr)).await {
match db.prepare(sql) { error!("Failed to update tx status: {}", e);
Ok(mut stmt) => {
if let Err(e) = stmt.bind((1, Value::String(txerr.clone()))) {
error!("Failed to bind txerr for error update: {}", e);
continue;
}
if let Err(e) = stmt.bind((2, Value::String(txid.clone()))) {
error!("Failed to bind txid for error update: {}", e);
continue;
}
let _ = stmt.next();
}
Err(e) => {
error!("Failed to prepare error update: {}", e);
}
} }
} }
if let Err(e) = send_stats_report(cfg, bcinfo).await { if let Err(e) = send_stats_report(cfg, bcinfo).await {
error!("send_stats_report failed: {}", e); error!("send_stats_report failed: {}", e);
} }
if let Err(e) = calculate_stats(&db, network_params.db_field.clone()).await { if let Err(e) = calculate_and_upsert_stats(&db, &network_params.db_field).await {
warn!("calculate_stats failed: {e}"); warn!("calculate_stats failed: {e}");
} }
} }
@@ -325,65 +294,8 @@ async fn main_result(cfg: &MyConfig, network_params: &NetworkParams) -> Result<(
} }
Ok(()) Ok(())
} }
async fn calculate_stats(db: &Connection, chain: String) -> Result<(), reqwest::Error> {
// Validate chain to prevent SQL injection via environment variable tampering
if !chain
.chars()
.all(|c| c.is_alphanumeric() || c == '-' || c == '_')
|| chain.is_empty()
{
error!("Invalid chain name: {chain}");
return Ok(());
}
//let sql = "drop table if exists tbl_stats;";
let sql = format!("DELETE FROM tbl_stats WHERE chain = '{chain}';");
if let Err(err) = db.execute(&sql) {
error!("error deleting from tbl_stats where chain:{chain} error: {err}");
}
let sql = format!(
"INSERT INTO tbl_stats (
report_date, chain, totals, waiting, sent, failed,
waiting_profit, sent_profit, missed_profit, unique_inputs
)
VALUES (
CURRENT_TIMESTAMP,
'{chain}',
(SELECT COUNT(*) FROM tbl_tx WHERE network = '{chain}'),
(SELECT COUNT(*) FROM tbl_tx WHERE status = 0 AND network = '{chain}'),
(SELECT COUNT(*) FROM tbl_tx WHERE status = 1 AND network = '{chain}'),
(SELECT COUNT(*) FROM tbl_tx WHERE status = 2 AND network = '{chain}'),
(SELECT IFNULL(SUM(our_fees),0) FROM tbl_tx WHERE status = 0 AND network = '{chain}'),
(SELECT IFNULL(SUM(our_fees),0) FROM tbl_tx WHERE status = 1 AND network = '{chain}'),
(SELECT IFNULL(SUM(our_fees),0) FROM tbl_tx WHERE status = 2 AND network = '{chain}'),
(SELECT COUNT(DISTINCT tbl_inp.in_txid)
FROM tbl_inp
JOIN tbl_tx ON tbl_inp.txid = tbl_tx.txid
WHERE tbl_tx.status = 0 AND tbl_tx.network = '{chain}')
)
ON CONFLICT(chain) DO UPDATE SET
report_date = excluded.report_date,
totals = excluded.totals,
waiting = excluded.waiting,
sent = excluded.sent,
failed = excluded.failed,
waiting_profit = excluded.waiting_profit,
sent_profit = excluded.sent_profit,
missed_profit = excluded.missed_profit,
unique_inputs = excluded.unique_inputs;
"
);
if let Err(err) = db.execute(&sql) {
error!("error inserting creating stats table {err}");
} else {
info!("tbl_stats creation success");
}
Ok(())
}
/// Parse the `(host, port)` pair from a base URL like `https://host[:port]`. /// Parse the `(host, port)` pair from a base URL like `https://host[:port]`.
///
/// Falls back to the scheme's well-known default port (443 for `https`,
/// 80 for plain `http`), or to 443 when the scheme is unknown.
fn parse_host_port(base_url: &str) -> Option<(String, u16)> { fn parse_host_port(base_url: &str) -> Option<(String, u16)> {
let url = Url::parse(base_url).ok()?; let url = Url::parse(base_url).ok()?;
let host = url let host = url
@@ -396,8 +308,6 @@ fn parse_host_port(base_url: &str) -> Option<(String, u16)> {
} }
/// Resolve `host:port` and return the first IPv6 (AAAA) address, if any. /// Resolve `host:port` and return the first IPv6 (AAAA) address, if any.
///
/// Returns `None` when the host has no IPv6 address.
async fn resolve_first_ipv6(host: &str, port: u16) -> Option<SocketAddr> { async fn resolve_first_ipv6(host: &str, port: u16) -> Option<SocketAddr> {
use std::net::ToSocketAddrs; use std::net::ToSocketAddrs;
let host = host.to_string(); let host = host.to_string();
@@ -412,14 +322,6 @@ async fn resolve_first_ipv6(host: &str, port: u16) -> Option<SocketAddr> {
.flatten() .flatten()
} }
/// Build the HTTP client used for welist reports.
///
/// When `BAL_PUSHER_PREFER_IPV6` is truthy, the welist host is resolved and
/// the client is pinned to its first IPv6 address (the original hostname is
/// still used for the `Host` header and TLS SNI). This works around networks
/// where the IPv4 route to the welist host is broken while IPv6 works: the
/// default connector may otherwise pick the broken family and the request
/// stalls. When the variable is unset (the default), behavior is unchanged.
async fn welist_http_client(welist_url: &str) -> rClient { async fn welist_http_client(welist_url: &str) -> rClient {
let prefer_ipv6 = env::var("BAL_PUSHER_PREFER_IPV6") let prefer_ipv6 = env::var("BAL_PUSHER_PREFER_IPV6")
.unwrap_or("false".to_string()) .unwrap_or("false".to_string())
@@ -521,6 +423,15 @@ fn sign_message(private_key_path: &str, message: &str) -> String {
} }
fn parse_env(cfg: &mut MyConfig) { fn parse_env(cfg: &mut MyConfig) {
if let Ok(value) = env::var("BAL_PUSHER_DB_BACKEND") {
cfg.db_backend = value;
}
if let Ok(value) = env::var("BAL_PUSHER_DB_FILE") {
cfg.db_file = value;
}
if let Ok(value) = env::var("BAL_PUSHER_PG_DSN") {
cfg.pg_dsn = value;
}
cfg.regtest = parse_env_netconfig(cfg, "regtest"); cfg.regtest = parse_env_netconfig(cfg, "regtest");
cfg.signet = parse_env_netconfig(cfg, "signet"); cfg.signet = parse_env_netconfig(cfg, "signet");
cfg.testnet = parse_env_netconfig(cfg, "testnet"); cfg.testnet = parse_env_netconfig(cfg, "testnet");
@@ -528,7 +439,6 @@ fn parse_env(cfg: &mut MyConfig) {
drop(parse_env_netconfig(cfg, "bitcoin")); drop(parse_env_netconfig(cfg, "bitcoin"));
} }
fn parse_env_netconfig(cfg_lock: &mut MyConfig, chain: &str) -> NetworkParams { fn parse_env_netconfig(cfg_lock: &mut MyConfig, chain: &str) -> NetworkParams {
//fn parse_env_netconfig(cfg_lock: &MutexGuard<MyConfig>, chain: &str) -> &NetworkParams{
let cfg = match chain { let cfg = match chain {
"regtest" => &mut cfg_lock.regtest, "regtest" => &mut cfg_lock.regtest,
"signet" => &mut cfg_lock.signet, "signet" => &mut cfg_lock.signet,

View File

@@ -1,4 +1,5 @@
use actix_governor::{Governor, GovernorConfigBuilder}; use actix_governor::{Governor, GovernorConfigBuilder, KeyExtractor, SimpleKeyExtractionError};
use actix_web::dev::ServiceRequest;
use actix_web::middleware; use actix_web::middleware;
use actix_web::web::Bytes; use actix_web::web::Bytes;
use actix_web::{App, HttpResponse, HttpServer, Responder, web}; use actix_web::{App, HttpResponse, HttpServer, Responder, web};
@@ -7,16 +8,16 @@ use chrono::Utc;
use hex_conservative::FromHex; use hex_conservative::FromHex;
use log::{debug, error, info, trace}; use log::{debug, error, info, trace};
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
use sqlite::State;
use sqlite::{Connection, Value};
use std::collections::{HashMap, HashSet}; use std::collections::{HashMap, HashSet};
use std::env; use std::env;
use std::fs; use std::fs;
use std::sync::Mutex; use std::net::IpAddr;
use std::str::FromStr;
use bal_server::db::{ use bal_server::db::{
check_duplicate_txids, create_database, execute_insert, get_all_addresses_by_xpub, DatabasePool, InsertInpData, InsertOutData, InsertTxData, check_duplicate_txids,
get_last_used_address_by_ip, get_next_address_index, insert_xpub, open_db, save_new_address, create_database, get_all_addresses_by_xpub, get_last_used_address_by_ip,
get_next_address_index, get_stats, insert_xpub, open_database, save_new_address, search_tx,
}; };
use bal_server::xpub::new_address_from_xpub; use bal_server::xpub::new_address_from_xpub;
@@ -56,7 +57,9 @@ struct MyConfig {
info: String, info: String,
bind_address: String, bind_address: String,
bind_port: u16, bind_port: u16,
db_backend: String,
db_file: String, db_file: String,
pg_dsn: String,
pub_key_path: String, pub_key_path: String,
expose_stats: bool, expose_stats: bool,
} }
@@ -71,7 +74,9 @@ impl Default for MyConfig {
mainnet: NetConfig::default_network("bitcoin".to_string(), Network::Bitcoin), mainnet: NetConfig::default_network("bitcoin".to_string(), Network::Bitcoin),
bind_address: "127.0.0.1".to_string(), bind_address: "127.0.0.1".to_string(),
bind_port: 9137, bind_port: 9137,
db_backend: "sqlite".to_string(),
db_file: "bal.db".to_string(), db_file: "bal.db".to_string(),
pg_dsn: String::new(),
info: "Will Executor Server".to_string(), info: "Will Executor Server".to_string(),
pub_key_path: "public_key.pem".to_string(), pub_key_path: "public_key.pem".to_string(),
expose_stats: env::var("BAL_SERVER_EXPOSE_STATS") expose_stats: env::var("BAL_SERVER_EXPOSE_STATS")
@@ -128,6 +133,7 @@ struct ActixConfig {
rate_limit_default: (u64, u32), rate_limit_default: (u64, u32),
workers: usize, workers: usize,
max_connections: usize, max_connections: usize,
trusted_proxy: IpAddr,
} }
fn parse_actix_config() -> ActixConfig { fn parse_actix_config() -> ActixConfig {
@@ -188,11 +194,15 @@ fn parse_actix_config() -> ActixConfig {
.unwrap_or("100".to_string()) .unwrap_or("100".to_string())
.parse::<usize>() .parse::<usize>()
.unwrap_or(100), .unwrap_or(100),
trusted_proxy: env::var("BAL_SERVER_TRUSTED_PROXY")
.unwrap_or_else(|_| "127.0.0.1".to_string())
.parse::<IpAddr>()
.unwrap_or(IpAddr::from_str("127.0.0.1").unwrap()),
} }
} }
struct AppState { struct AppState {
db: Mutex<Connection>, db: DatabasePool,
cfg: MyConfig, cfg: MyConfig,
} }
@@ -217,41 +227,70 @@ async fn echo_version() -> impl Responder {
HttpResponse::Ok().body(VERSION) HttpResponse::Ok().body(VERSION)
} }
fn is_valid_ip(ip: &str) -> bool { fn extract_real_ip(req: &actix_web::HttpRequest, trusted_proxy: IpAddr) -> String {
ip.parse::<std::net::IpAddr>().is_ok() let peer_ip = req.peer_addr().map(|socket| socket.ip());
let connection_info = req.connection_info();
let ip = match peer_ip {
Some(peer) if peer == trusted_proxy => {
connection_info.realip_remote_addr().unwrap_or("unknown")
}
_ => connection_info.peer_addr().unwrap_or("unknown"),
};
debug!("client IP: {}", ip);
ip.to_string()
} }
fn extract_client_ip(req: &actix_web::HttpRequest) -> String { #[derive(Debug, Clone, Copy, PartialEq, Eq)]
if let Some(val) = req.headers().get("X-Real-IP") struct RealIpKeyExtractor;
&& let Ok(s) = val.to_str()
{ impl KeyExtractor for RealIpKeyExtractor {
let ip = s.split(',').next().unwrap_or(s).trim(); type Key = IpAddr;
if is_valid_ip(ip) { type KeyExtractionError = SimpleKeyExtractionError<&'static str>;
debug!("client IP from X-Real-IP: {}", ip);
return ip.to_string(); fn extract(&self, req: &ServiceRequest) -> Result<Self::Key, Self::KeyExtractionError> {
} let proxy_ip = req
} .app_data::<web::Data<IpAddr>>()
if let Some(val) = req.headers().get("X-Forwarded-For") .map(|ip| *ip.get_ref())
&& let Ok(s) = val.to_str() .unwrap_or_else(|| IpAddr::from_str("0.0.0.0").unwrap());
{
let ip = s.split(',').next().unwrap_or(s).trim(); let peer_ip = req.peer_addr().map(|socket| socket.ip());
if is_valid_ip(ip) { let connection_info = req.connection_info();
debug!("client IP from X-Forwarded-For: {}", ip);
return ip.to_string(); match peer_ip {
} Some(peer) if peer == proxy_ip => connection_info
} .realip_remote_addr()
let fallback = req .ok_or_else(|| {
.connection_info() SimpleKeyExtractionError::new("Could not extract real IP address from request")
})
.and_then(|str| {
str.parse::<IpAddr>().map_err(|_| {
SimpleKeyExtractionError::new(
"Could not extract real IP address from request",
)
})
}),
_ => connection_info
.peer_addr() .peer_addr()
.unwrap_or("unknown") .ok_or_else(|| {
.to_string(); SimpleKeyExtractionError::new("Could not extract peer IP address from request")
debug!("client IP from peer_addr fallback: {}", fallback); })
fallback .and_then(|str| {
str.parse::<IpAddr>().map_err(|_| {
SimpleKeyExtractionError::new(
"Could not extract peer IP address from request",
)
})
}),
}
}
} }
async fn echo_info( async fn echo_info(
path: web::Path<String>, path: web::Path<String>,
data: web::Data<AppState>, data: web::Data<AppState>,
proxy: web::Data<IpAddr>,
req: actix_web::HttpRequest, req: actix_web::HttpRequest,
) -> impl Responder { ) -> impl Responder {
let param = path.into_inner(); let param = path.into_inner();
@@ -264,7 +303,7 @@ async fn echo_info(
debug!("network disabled {}", param); debug!("network disabled {}", param);
return HttpResponse::BadRequest().body("error"); return HttpResponse::BadRequest().body("error");
} }
let remote_addr = extract_client_ip(&req); let remote_addr = extract_real_ip(&req, **proxy);
let address = match netconfig.xpub { let address = match netconfig.xpub {
false => { false => {
let address = netconfig.address.to_string(); let address = netconfig.address.to_string();
@@ -272,22 +311,14 @@ async fn echo_info(
address address
} }
true => { true => {
// Lock #1: fetch existing address OR atomically claim next index if let Some(address) = get_last_used_address_by_ip(
let next_idx = { &data.db,
let db = match data.db.lock() {
Ok(g) => g,
Err(_p) => {
error!("DB mutex poisoned in echo_info (lookup phase)");
return HttpResponse::InternalServerError().body("error");
}
};
match get_last_used_address_by_ip(
&db,
&netconfig.name, &netconfig.name,
&netconfig.address, &netconfig.address,
&remote_addr, &remote_addr,
) { )
Some(address) => { .await
{
return HttpResponse::Ok().json(InfoResponse { return HttpResponse::Ok().json(InfoResponse {
address, address,
base_fee: netconfig.fixed_fee, base_fee: netconfig.fixed_fee,
@@ -296,11 +327,10 @@ async fn echo_info(
version: VERSION.to_string(), version: VERSION.to_string(),
}); });
} }
None => get_next_address_index(&db, &netconfig.name, &netconfig.address),
}
}; // lock released
// Derive address (CPU-bound, no lock held) let next_idx =
get_next_address_index(&data.db, &netconfig.name, &netconfig.address).await;
let derived = let derived =
match new_address_from_xpub(&netconfig.address, next_idx.1, netconfig.network) { match new_address_from_xpub(&netconfig.address, next_idx.1, netconfig.network) {
Ok(address) => address, Ok(address) => address,
@@ -310,20 +340,10 @@ async fn echo_info(
} }
}; };
// Lock #2: save the newly derived address save_new_address(&data.db, next_idx.0, &derived.0, &derived.1, &remote_addr).await;
{
let db = match data.db.lock() {
Ok(g) => g,
Err(_p) => {
error!("DB mutex poisoned in echo_info (save phase)");
return HttpResponse::InternalServerError().body("error");
}
};
save_new_address(&db, next_idx.0, &derived.0, &derived.1, &remote_addr);
debug!("save new address {} {}", derived.0, derived.1); debug!("save new address {} {}", derived.0, derived.1);
trace!("next {} {}", next_idx.0, next_idx.1); trace!("next {} {}", next_idx.0, next_idx.1);
derived.0 derived.0
} // lock released
} }
}; };
let info = InfoResponse { let info = InfoResponse {
@@ -357,83 +377,31 @@ async fn echo_stats(path: web::Path<String>, data: web::Data<AppState>) -> impl
if !data.cfg.expose_stats { if !data.cfg.expose_stats {
return HttpResponse::Forbidden().body("error"); return HttpResponse::Forbidden().body("error");
} }
let mut stats: Vec<StatsResponse> = vec![];
let db = match data.db.lock() { let stats_rows = match get_stats(&data.db, &netconfig.name).await {
Ok(g) => g, Ok(rows) => rows,
Err(_p) => {
error!("DB mutex poisoned in echo_stats");
return HttpResponse::InternalServerError().body("error");
}
};
let mut stmt = match db.prepare(
"SELECT report_date, chain, totals, waiting, sent, failed, waiting_profit, sent_profit, missed_profit, unique_inputs FROM tbl_stats WHERE chain = ?"
) {
Ok(s) => s,
Err(e) => { Err(e) => {
error!("Failed to prepare stats query: {}", e); error!("Failed to query stats: {}", e);
return HttpResponse::InternalServerError().body("error"); return HttpResponse::InternalServerError().body("error");
} }
}; };
if let Err(e) = stmt.bind((1, Value::String(netconfig.name.clone()))) {
error!("Failed to bind chain in stats query: {}", e); let stats: Vec<StatsResponse> = stats_rows
return HttpResponse::InternalServerError().body("error"); .into_iter()
} .map(|row| StatsResponse {
while let Ok(State::Row) = stmt.next() { report_date: row.report_date,
let report_date = stmt.read("report_date").unwrap_or("0".to_string()); chain: row.chain,
let chain = stmt.read("chain").unwrap_or("?".to_string()); totals: row.totals,
let totals = stmt waiting: row.waiting,
.read("totals") sent: row.sent,
.unwrap_or("0".to_string()) failed: row.failed,
.parse::<i64>() waiting_profit: row.waiting_profit,
.unwrap_or(0); sent_profit: row.sent_profit,
let waiting = stmt missed_profit: row.missed_profit,
.read("waiting") unique_inputs: row.unique_inputs,
.unwrap_or("0".to_string()) })
.parse::<i64>() .collect();
.unwrap_or(0);
let sent = stmt
.read("sent")
.unwrap_or("0".to_string())
.parse::<i64>()
.unwrap_or(0);
let failed = stmt
.read("failed")
.unwrap_or("0".to_string())
.parse::<i64>()
.unwrap_or(0);
let waiting_profit = stmt
.read("waiting_profit")
.unwrap_or("0".to_string())
.parse::<i64>()
.unwrap_or(0);
let sent_profit = stmt
.read("sent_profit")
.unwrap_or("0".to_string())
.parse::<i64>()
.unwrap_or(0);
let missed_profit = stmt
.read("missed_profit")
.unwrap_or("0".to_string())
.parse::<i64>()
.unwrap_or(0);
let unique_inputs = stmt
.read("unique_inputs")
.unwrap_or("0".to_string())
.parse::<i64>()
.unwrap_or(0);
stats.push(StatsResponse {
report_date,
chain,
totals,
waiting,
sent,
failed,
waiting_profit,
sent_profit,
missed_profit,
unique_inputs,
});
}
debug!("echo stats reply for chain: {}", netconfig.name); debug!("echo stats reply for chain: {}", netconfig.name);
HttpResponse::Ok().json(stats) HttpResponse::Ok().json(stats)
} }
@@ -453,67 +421,15 @@ async fn echo_search(body: Bytes, data: web::Data<AppState>) -> impl Responder {
return HttpResponse::BadRequest().body("error"); return HttpResponse::BadRequest().body("error");
} }
let db = match data.db.lock() { match search_tx(&data.db, strbody).await {
Ok(g) => g, Ok(Some(row)) => {
Err(_p) => {
error!("DB mutex poisoned in echo_search");
return HttpResponse::InternalServerError().body("error");
}
};
let mut statement = match db.prepare("SELECT * FROM tbl_tx WHERE txid = ? LIMIT 1") {
Ok(s) => s,
Err(e) => {
error!("Failed to prepare statement: {}", e);
return HttpResponse::InternalServerError().body("error");
}
};
if let Err(e) = statement.bind((1, strbody)) {
error!("Failed to bind parameter: {}", e);
return HttpResponse::InternalServerError().body("error");
}
if let Ok(State::Row) = statement.next() {
let mut response_data = HashMap::new(); let mut response_data = HashMap::new();
match statement.read::<String, _>("status") { response_data.insert("status", row.status);
Ok(value) => { response_data.insert("tx", row.tx);
response_data.insert("status", value); response_data.insert("our_address", row.our_address);
} response_data.insert("our_fees", row.our_fees);
Err(e) => { response_data.insert("time", row.reqid);
error!("Error reading status: {}", e);
}
}
match statement.read::<String, _>("tx") {
Ok(value) => {
response_data.insert("tx", value);
}
Err(e) => {
error!("Error reading tx: {}", e);
}
}
match statement.read::<String, _>("our_address") {
Ok(value) => {
response_data.insert("our_address", value);
}
Err(e) => {
error!("Error reading address: {}", e);
}
}
match statement.read::<String, _>("our_fees") {
Ok(value) => {
response_data.insert("our_fees", value);
}
Err(e) => {
error!("Error reading fees: {}", e);
}
}
match statement.read::<String, _>("reqid") {
Ok(value) => {
response_data.insert("time", value);
}
Err(e) => {
error!("Error reading reqid: {}", e);
}
}
match serde_json::to_string(&response_data) { match serde_json::to_string(&response_data) {
Ok(json_data) => { Ok(json_data) => {
debug!("echo search reply: {}", json_data); debug!("echo search reply: {}", json_data);
@@ -521,25 +437,26 @@ async fn echo_search(body: Bytes, data: web::Data<AppState>) -> impl Responder {
} }
Err(_) => HttpResponse::BadRequest().body("error"), Err(_) => HttpResponse::BadRequest().body("error"),
} }
} else { }
HttpResponse::BadRequest().body("error") Ok(None) => HttpResponse::BadRequest().body("error"),
Err(e) => {
error!("Failed to search tx: {}", e);
HttpResponse::InternalServerError().body("error")
}
} }
} }
/// Holds a transaction that has already been parsed and validated outside the DB lock.
#[derive(Clone)] #[derive(Clone)]
struct ParsedTx { struct ParsedTx {
txid: String, txid: String,
wtxid: String, wtxid: String,
ntxid: String, ntxid: String,
raw_hex: String, // the original line raw_hex: String,
locktime: String, locktime: String,
inputs: Vec<(String, String)>, // (in_txid, in_vout) inputs: Vec<(String, String)>,
outputs: Vec<(usize, String, u64)>, // (idx, script_pubkey, amount_sat) outputs: Vec<(usize, String, u64)>,
} }
/// Parse all transactions from the request body **without** needing the DB lock.
/// Skips transactions that don't have a valid willexecutor output.
fn parse_request_transactions( fn parse_request_transactions(
strbody: &str, strbody: &str,
_req_time: i64, _req_time: i64,
@@ -576,7 +493,6 @@ fn parse_request_transactions(
let wtxid = tx.compute_wtxid(); let wtxid = tx.compute_wtxid();
let locktime = tx.lock_time.to_string(); let locktime = tx.lock_time.to_string();
// Collect inputs
let mut inputs: Vec<(String, String)> = Vec::with_capacity(tx.input.len()); let mut inputs: Vec<(String, String)> = Vec::with_capacity(tx.input.len());
for input in tx.input { for input in tx.input {
inputs.push(( inputs.push((
@@ -585,7 +501,6 @@ fn parse_request_transactions(
)); ));
} }
// Collect outputs and find which one is ours + its amount
let mut outputs: Vec<(usize, String, u64)> = Vec::with_capacity(tx.output.len()); let mut outputs: Vec<(usize, String, u64)> = Vec::with_capacity(tx.output.len());
let mut found = false; let mut found = false;
let mut our_address = String::new(); let mut our_address = String::new();
@@ -601,13 +516,25 @@ fn parse_request_transactions(
netconfig.network, netconfig.network,
) { ) {
Ok(addr) => addr.to_string(), Ok(addr) => addr.to_string(),
Err(_) => continue, // skip un-decodable outputs Err(_) => continue,
}; };
let expected_ours = if netconfig.xpub { let expected_ours = if netconfig.xpub {
if known_addresses.contains(&address) { if known_addresses.contains(&address) {
trace!(
"output {} address {} found in known_addresses (total: {})",
idx,
&address,
known_addresses.len()
);
address.clone() address.clone()
} else { } else {
trace!(
"output {} address {} NOT in known_addresses (total: {}), skipping",
idx,
&address,
known_addresses.len()
);
continue; continue;
} }
} else { } else {
@@ -619,6 +546,11 @@ fn parse_request_transactions(
our_fees = amount; our_fees = amount;
found = true; found = true;
trace!("address and fees are correct {}: {}", our_address, our_fees); trace!("address and fees are correct {}: {}", our_address, our_fees);
} else if address == expected_ours {
trace!(
"output {} address matches but amount {} < fixed_fee {}, skipping",
idx, amount, netconfig.fixed_fee
);
} }
} }
@@ -679,16 +611,8 @@ async fn echo_push(
}; };
// ===== PHASE 1: parse all transactions WITHOUT the DB lock ===== // ===== PHASE 1: parse all transactions WITHOUT the DB lock =====
let known_addresses: HashSet<String> = { let known_addresses: HashSet<String> = if netconfig.xpub {
let db = match data.db.lock() { match get_all_addresses_by_xpub(&data.db, &netconfig.address).await {
Ok(g) => g,
Err(_p) => {
error!("DB mutex poisoned acquiring addresses in echo_push");
return HttpResponse::InternalServerError().body("error");
}
};
if netconfig.xpub {
match get_all_addresses_by_xpub(&db, &netconfig.address) {
Ok(addrs) => addrs, Ok(addrs) => addrs,
Err(e) => { Err(e) => {
error!("Failed to load addresses from xpub: {}", e); error!("Failed to load addresses from xpub: {}", e);
@@ -697,8 +621,7 @@ async fn echo_push(
} }
} else { } else {
HashSet::new() HashSet::new()
} };
}; // lock released here
// Parse all transactions (CPU-bound, no DB needed) // Parse all transactions (CPU-bound, no DB needed)
let parsed = parse_request_transactions(strbody, req_time, netconfig, &known_addresses); let parsed = parse_request_transactions(strbody, req_time, netconfig, &known_addresses);
@@ -709,134 +632,86 @@ async fn echo_push(
let all_txids: Vec<String> = parsed.iter().map(|(p, _, _)| p.txid.clone()).collect(); let all_txids: Vec<String> = parsed.iter().map(|(p, _, _)| p.txid.clone()).collect();
// ===== PHASE 2: check duplicates in a single batch query ===== // ===== PHASE 2: check duplicates in a single batch query =====
let duplicates = { let duplicates = match check_duplicate_txids(&data.db, &all_txids).await {
let db = match data.db.lock() {
Ok(g) => g,
Err(_p) => {
error!("DB mutex poisoned in echo_push duplicate check");
return HttpResponse::InternalServerError().body("error");
}
};
match check_duplicate_txids(&db, &all_txids) {
Ok(dups) => dups, Ok(dups) => dups,
Err(e) => { Err(e) => {
error!("Duplicate check failed: {}", e); error!("Duplicate check failed: {}", e);
return HttpResponse::InternalServerError().body("error"); return HttpResponse::InternalServerError().body("error");
} }
} };
}; // lock released here
let all_present = all_txids.iter().all(|t| duplicates.contains(t)); let all_present = all_txids.iter().all(|t| duplicates.contains(t));
if all_present { if all_present {
return HttpResponse::Ok().body("already present"); return HttpResponse::Ok().body("already present");
} }
// ===== PHASE 3: build insert statements and execute (single DB lock, minimal time) ===== // ===== PHASE 3: build insert data and execute (single transaction, minimal time) =====
{ let mut tx_data = Vec::new();
let db = match data.db.lock() { let mut inp_data = Vec::new();
Ok(g) => g, let mut out_data = Vec::new();
Err(_p) => {
error!("DB mutex poisoned in echo_push insert phase");
return HttpResponse::InternalServerError().body("error");
}
};
let sqltxshead = "INSERT INTO tbl_tx (txid, wtxid, ntxid, tx, locktime, reqid, network, our_address, our_fees)".to_string();
let mut sqltxs = String::new();
let sqlinpshead = "INSERT INTO tbl_inp (txid, in_txid, in_vout )".to_string();
let mut sqlinps = String::new();
let sqloutshead = "INSERT INTO tbl_out (txid, vout, script_pubkey, amount )".to_string();
let mut sqlouts = String::new();
let mut union_tx = true;
let mut union_inps = true;
let mut union_outs = true;
let mut ptx: Vec<(usize, Value)> = vec![];
let mut pinps: Vec<(usize, Value)> = vec![];
let mut pouts: Vec<(usize, Value)> = vec![];
let mut linenum = 1usize;
let mut lineinp = 1usize;
let mut lineout = 1usize;
for (parsed, our_address, our_fees) in &parsed { for (parsed, our_address, our_fees) in &parsed {
if duplicates.contains(&parsed.txid) { if duplicates.contains(&parsed.txid) {
continue; continue;
} }
if !union_tx { tx_data.push(InsertTxData {
sqltxs.push_str(" UNION ALL"); txid: parsed.txid.clone(),
} else { wtxid: parsed.wtxid.clone(),
union_tx = false; ntxid: parsed.ntxid.clone(),
} raw_hex: parsed.raw_hex.clone(),
sqltxs.push_str(" SELECT ?, ?, ?, ?, ?, ?, ?, ?, ?"); locktime: parsed.locktime.clone(),
ptx.push((linenum, Value::String(parsed.txid.clone()))); reqid: req_time.to_string(),
ptx.push((linenum + 1, Value::String(parsed.wtxid.clone()))); network: netconfig.name.clone(),
ptx.push((linenum + 2, Value::String(parsed.ntxid.clone()))); our_address: our_address.clone(),
ptx.push((linenum + 3, Value::String(parsed.raw_hex.clone()))); our_fees: our_fees.to_string(),
ptx.push((linenum + 4, Value::String(parsed.locktime.clone()))); });
ptx.push((linenum + 5, Value::String(req_time.to_string())));
ptx.push((linenum + 6, Value::String(netconfig.name.clone())));
ptx.push((linenum + 7, Value::String(our_address.clone())));
ptx.push((linenum + 8, Value::String(our_fees.to_string())));
linenum += 9;
for (in_txid, in_vout) in &parsed.inputs { for (in_txid, in_vout) in &parsed.inputs {
if !union_inps { inp_data.push(InsertInpData {
sqlinps.push_str(" UNION ALL"); txid: parsed.txid.clone(),
} else { in_txid: in_txid.clone(),
union_inps = false; in_vout: in_vout.clone(),
} });
sqlinps.push_str(" SELECT ?, ?, ?");
pinps.push((lineinp, Value::String(parsed.txid.clone())));
pinps.push((lineinp + 1, Value::String(in_txid.clone())));
pinps.push((lineinp + 2, Value::String(in_vout.clone())));
lineinp += 3;
} }
for (idx, script, amount) in &parsed.outputs { for (idx, script, amount) in &parsed.outputs {
if !union_outs { out_data.push(InsertOutData {
sqlouts.push_str(" UNION ALL"); txid: parsed.txid.clone(),
} else { vout: i64::try_from(*idx).unwrap_or(-1),
union_outs = false; script_pubkey: script.clone(),
} amount: i64::try_from(*amount).unwrap_or(0),
sqlouts.push_str(" SELECT ?, ?, ?, ?"); });
pouts.push((lineout, Value::String(parsed.txid.clone())));
pouts.push((
lineout + 1,
Value::Integer(i64::try_from(*idx).unwrap_or(-1)),
));
pouts.push((lineout + 2, Value::String(script.clone())));
pouts.push((
lineout + 3,
Value::Integer(i64::try_from(*amount).unwrap_or(0)),
));
lineout += 4;
} }
} }
if sqltxs.is_empty() { if tx_data.is_empty() {
return HttpResponse::Ok().body("already present"); return HttpResponse::Ok().body("already present");
} }
let sqltxs = format!("{}{};", sqltxshead, sqltxs); if let Err(err) = bal_server::db::execute_insert(&data.db, &tx_data, &inp_data, &out_data).await
let sqlinps = format!("{}{};", sqlinpshead, sqlinps); {
let sqlouts = format!("{}{};", sqloutshead, sqlouts);
if let Err(err) = execute_insert(&db, sqltxs, ptx, sqlinps, pinps, sqlouts, pouts) {
error!("execute_insert failed: {}", err); error!("execute_insert failed: {}", err);
return HttpResponse::BadRequest().body("error"); return HttpResponse::BadRequest().body("error");
} }
} // lock released
HttpResponse::Ok().body("thx") HttpResponse::Ok().body("thx")
} }
fn parse_env(data: &MyConfig) -> MyConfig { fn parse_env(data: &MyConfig) -> MyConfig {
let mut cfg = data.clone(); let mut cfg = data.clone();
if let Ok(value) = env::var("BAL_SERVER_DB_BACKEND") {
debug!("BAL_SERVER_DB_BACKEND: {}", value);
cfg.db_backend = value;
}
if let Ok(value) = env::var("BAL_SERVER_DB_FILE") { if let Ok(value) = env::var("BAL_SERVER_DB_FILE") {
debug!("BAL_SERVER_DB_FILE: {}", value); debug!("BAL_SERVER_DB_FILE: {}", value);
cfg.db_file = value; cfg.db_file = value;
} }
if let Ok(value) = env::var("BAL_SERVER_PG_DSN") {
debug!("BAL_SERVER_PG_DSN: {}", value);
cfg.pg_dsn = value;
}
if let Ok(value) = env::var("BAL_SERVER_BIND_ADDRESS") { if let Ok(value) = env::var("BAL_SERVER_BIND_ADDRESS") {
debug!("BAL_SERVER_BIND_ADDRESS: {}", value); debug!("BAL_SERVER_BIND_ADDRESS: {}", value);
cfg.bind_address = value; cfg.bind_address = value;
@@ -889,10 +764,10 @@ fn parse_env_netconfig(cfg: &mut MyConfig, chain: &str) {
} }
} }
fn init_network(db: &Connection, cfg: &MyConfig) { async fn init_network(pool: &DatabasePool, cfg: &MyConfig) {
for network in NETWORKS { for network in NETWORKS {
let netconfig = cfg.get_net_config(network); let netconfig = cfg.get_net_config(network);
insert_xpub(db, &netconfig.name.to_string(), &netconfig.address); insert_xpub(pool, &netconfig.name.to_string(), &netconfig.address).await;
} }
} }
@@ -903,32 +778,45 @@ async fn main() -> std::io::Result<()> {
let actix_cfg = parse_actix_config(); let actix_cfg = parse_actix_config();
let cfg = parse_env(&cfg); let cfg = parse_env(&cfg);
let db = match open_db(&cfg.db_file) {
Ok(c) => c, let connection_string = match cfg.db_backend.as_str() {
"sqlite" => cfg.db_file.clone(),
"postgresql" => cfg.pg_dsn.clone(),
other => {
return Err(std::io::Error::other(format!(
"Unknown DB backend: {}",
other
)));
}
};
let db = match open_database(&cfg.db_backend, &connection_string).await {
Ok(pool) => pool,
Err(e) => { Err(e) => {
return Err(std::io::Error::other(e)); return Err(std::io::Error::other(e));
} }
}; };
// Create database tables // Create database tables
create_database(&db); create_database(&db)
.await
.map_err(|e| std::io::Error::other(format!("Failed to create database: {}", e)))?;
// Initialize networks // Initialize networks
init_network(&db, &cfg); init_network(&db, &cfg).await;
let data = web::Data::new(AppState { let data = web::Data::new(AppState {
db: Mutex::new(db), db,
cfg: cfg.clone(), cfg: cfg.clone(),
}); });
let bind_address = data.cfg.bind_address.clone(); let bind_address = data.cfg.bind_address.clone();
let bind_port = data.cfg.bind_port; let bind_port = data.cfg.bind_port;
// Use a single global rate limiter with the most conservative settings (1 req/sec) let governor_conf = GovernorConfigBuilder::default()
// Per-endpoint rate limiting requires advanced configuration with explicit types .seconds_per_request(actix_cfg.rate_limit_pushtxs.0)
let governor_conf = GovernorConfigBuilder::const_default() .burst_size(actix_cfg.rate_limit_pushtxs.1)
.seconds_per_request(actix_cfg.rate_limit_pushtxs.0) // Most restrictive: 1 req/sec .key_extractor(RealIpKeyExtractor)
.burst_size(actix_cfg.rate_limit_pushtxs.1) // Burst: 3
.finish() .finish()
.unwrap(); .unwrap();
@@ -938,6 +826,7 @@ async fn main() -> std::io::Result<()> {
App::new() App::new()
.app_data(web::PayloadConfig::default().limit(actix_cfg.max_body_size)) .app_data(web::PayloadConfig::default().limit(actix_cfg.max_body_size))
.app_data(data.clone()) .app_data(data.clone())
.app_data(web::Data::new(actix_cfg.trusted_proxy))
.wrap(middleware::Logger::default()) .wrap(middleware::Logger::default())
.wrap(middleware::Compress::default()) .wrap(middleware::Compress::default())
.wrap(Governor::new(&governor_conf)) .wrap(Governor::new(&governor_conf))

412
src/db.rs
View File

@@ -1,412 +0,0 @@
use log::{error, info, trace, warn};
use sqlite::{Connection, Error, State, Value};
use std::collections::HashSet;
use std::path::Path;
use std::thread;
use std::time::Duration;
/// Check which txids are already present in the database in a single batch query.
/// Returns a HashSet of txids that already exist (duplicates).
/// This is O(1) per query regardless of the number of txids, replacing the N+1 pattern.
pub fn check_duplicate_txids(db: &Connection, txids: &[String]) -> Result<HashSet<String>, Error> {
if txids.is_empty() {
return Ok(HashSet::new());
}
// Build a single query with all txids using IN clause placeholders
// SQLite supports up to 1000 parameters per statement, so we chunk for safety
let mut duplicates = HashSet::new();
let chunk_size = 500; // Safe chunk size for SQLite parameters
for chunk in txids.chunks(chunk_size) {
let placeholders = chunk.iter().map(|_| "?").collect::<Vec<_>>().join(",");
let sql = format!("SELECT txid FROM tbl_tx WHERE txid IN ({})", placeholders);
let mut stmt = db.prepare(sql)?;
for (i, txid) in chunk.iter().enumerate() {
stmt.bind((i + 1, Value::String(txid.clone())))?;
}
while let Ok(State::Row) = stmt.next() {
if let Ok(txid) = stmt.read::<String, _>("txid") {
duplicates.insert(txid);
}
}
}
Ok(duplicates)
}
/// Validates and opens the SQLite database, enforcing security best practices:
/// - Path must not contain `..` (directory traversal).
/// - Absolute paths must not target known system directories.
/// - If the file exists, it must be a regular file (not a symlink or device).
/// - WAL journal mode is enabled for safe concurrent access.
/// - Synchronous is set to NORMAL for performance with safety.
///
/// Returns `Err` on validation failure or open error to prevent panics.
pub fn open_db(path: &str) -> Result<Connection, String> {
let p = Path::new(path);
// Prevent directory traversal
for component in p.components() {
if component == std::path::Component::ParentDir {
return Err("Database path may not contain '..'".to_string());
}
}
// If absolute, block known sensitive system directories
if p.is_absolute() {
let path_str = p.to_str().unwrap_or("");
let forbidden = [
"/etc", "/proc", "/sys", "/dev", "/usr", "/bin", "/sbin", "/lib", "/opt",
];
for prefix in &forbidden {
if path_str.starts_with(prefix) {
return Err(format!(
"Absolute database path under {} is forbidden",
prefix
));
}
}
}
// If file exists, must be a regular file (not a symlink, device, etc.)
if p.exists() {
if p.is_symlink() {
return Err("Database path must not be a symlink".to_string());
}
let metadata = std::fs::metadata(p)
.map_err(|e| format!("Cannot access database file metadata: {}", e))?;
if !metadata.is_file() {
return Err(
"Database path must point to a regular file, not a directory or device".to_string(),
);
}
}
let conn = sqlite::open(path).map_err(|e| format!("Failed to open SQLite database: {}", e))?;
// Set busy timeout BEFORE WAL mode so SQLite waits instead of failing immediately.
// This handles the race where two processes (server + pusher) open the same DB
// and both try to enable WAL mode concurrently.
conn.execute("PRAGMA busy_timeout = 5000;")
.map_err(|e| format!("Failed to set busy_timeout: {}", e))?;
// Retry WAL mode up to 5 times (handles concurrent open from bal-pusher).
let mut wal_ok = false;
for attempt in 0..5 {
match conn.execute("PRAGMA journal_mode = WAL;") {
Ok(_) => {
wal_ok = true;
break;
}
Err(e) => {
warn!(
"WAL mode attempt {}/5 failed: {}, retrying in 100ms...",
attempt + 1,
e
);
thread::sleep(Duration::from_millis(100));
}
}
}
if !wal_ok {
// WAL might already be enabled by another process; this is not fatal.
warn!("Could not set WAL mode after retries — may already be enabled by another process");
}
conn.execute("PRAGMA synchronous = NORMAL;")
.map_err(|e| format!("Failed to set synchronous NORMAL: {}", e))?;
Ok(conn)
}
/// Loads all known addresses for a given xpub into a HashSet for fast
/// in-memory lookup during transaction validation (replaces N+1 query).
pub fn get_all_addresses_by_xpub(db: &Connection, xpub: &str) -> Result<HashSet<String>, Error> {
let mut stmt = db.prepare(
"SELECT a.address FROM tbl_address a JOIN tbl_xpub x ON a.xpub = x.id WHERE x.xpub = ?",
)?;
stmt.bind((1, Value::String(xpub.to_string())))?;
let mut addresses = HashSet::new();
while let Ok(State::Row) = stmt.next() {
match stmt.read::<String, _>("address") {
Ok(addr) => {
addresses.insert(addr);
}
Err(e) => {
error!("Failed to read address column: {}", e);
}
}
}
Ok(addresses)
}
pub fn create_database(db: &Connection) {
info!("database sanity check");
let _ = db.execute("CREATE TABLE IF NOT EXISTS tbl_tx (txid PRIMARY KEY, date_creation TIMESTAMP DEFAULT CURRENT_TIMESTAMP, date_update TIMESTAMP DEFAULT CURRENT_TIMESTAMP, wtxid, ntxid, tx, locktime integer, network, network_fees, reqid, our_fees, our_address, status integer DEFAULT 0);");
let _ = db.execute("ALTER TABLE tbl_tx ADD COLUMN push_err TEXT");
let _ = db.execute("CREATE TABLE IF NOT EXISTS tbl_inp(id, txid, in_txid, in_vout);");
let _ = db.execute("CREATE UNIQUE INDEX ON tbl_inp(txid,in_txid,in_vout);");
let _ =
db.execute("CREATE TABLE IF NOT EXISTS tbl_out(id, txid, script_pubkey, amount, vout);");
let _ = db.execute("CREATE UNIQUE INDEX ON tbl_out(txid, script_pubkey, amount, vout);");
let _ = db.execute("CREATE TABLE IF NOT EXISTS tbl_xpub (id INTEGER PRIMARY KEY , network TEXT, xpub TEXT, date_create TIMESTAMP DEFAULT CURRENT_TIMESTAMP,path_idx INTEGER DEFAULT -1);");
let _ = db.execute("CREATE UNIQUE INDEX idx_xpub ON tbl_xpub (network, xpub)");
let _ = db.execute("CREATE TABLE IF NOT EXISTS tbl_address (address TEXT PRIMARY_KEY, path TEXT NOT NULL, date_create TIMESTAMP DEFAULT CURRENT_TIMESTAMP, xpub INTEGER,remote_address TEXT);");
let _ = db.execute("CREATE TABLE IF NOT EXISTS tbl_stats (report_date TEXT, chain TEXT, totals INTEGER, waiting INTEGER, sent INTEGER, failed INTEGER, waiting_profit INTEGER, sent_profit INTEGER, missed_profit INTEGER, unique_inputs INTEGER);");
// UNIQUE index required for ON CONFLICT(chain) DO UPDATE in calculate_stats
let _ = db.execute("DROP INDEX IF EXISTS idx_stats_chain;");
let _ = db.execute("CREATE UNIQUE INDEX IF NOT EXISTS idx_stats_chain ON tbl_stats(chain);");
let _ = db.execute("UPDATE tbl_tx set network='bitcoin' where network='mainnet';");
}
/*
pub fn get_xpub_id(db: &Connection, network: &String, xpub: &String) -> Option<i64>{
let mut stmt = db.prepare("SELECT * FROM tbl_xpub where network = ? and xpub = ?;").unwrap();
let _ = stmt.bind((1,Value::String(network.to_string()))).unwrap();
let _ = stmt.bind((2,Value::String(xpub.to_string()))).unwrap();
if let Ok(State::Row) = stmt.next(){
return Some(stmt.read::<i64, _>("id").unwrap());
} else {
return None;
}
}
*/
pub fn insert_xpub(db: &Connection, network: &str, xpub: &str) {
if !xpub.is_empty() {
trace!("going to insert: {} xpub:{}", network, xpub);
let mut stmt =
match db.prepare("INSERT OR IGNORE INTO tbl_xpub(network,xpub) VALUES(?, ?);") {
Ok(s) => s,
Err(e) => {
error!("Failed to prepare xpub insert statement: {}", e);
return;
}
};
if let Err(e) = stmt.bind((1, Value::String(network.to_string()))) {
error!("Failed to bind network parameter for xpub insert: {}", e);
return;
}
if let Err(e) = stmt.bind((2, Value::String(xpub.to_string()))) {
error!("Failed to bind xpub parameter: {}", e);
return;
}
if let Err(e) = stmt.next() {
error!("Failed to insert xpub: {}", e);
}
}
}
pub fn get_last_used_address_by_ip(
db: &Connection,
network: &String,
xpub: &String,
address: &String,
) -> Option<String> {
let mut stmt = match db.prepare("SELECT tbl_address.address FROM tbl_xpub join tbl_address on(tbl_xpub.id = tbl_address.xpub) where tbl_xpub.network = ? and tbl_address.remote_address = ? and tbl_xpub.xpub = ? ORDER BY tbl_address.date_create DESC LIMIT 1;") {
Ok(s) => s,
Err(e) => {
error!("Failed to prepare address query: {}", e);
return None;
}
};
if let Err(e) = stmt.bind((1, Value::String(network.to_string()))) {
error!("Failed to bind network parameter: {}", e);
return None;
}
if let Err(e) = stmt.bind((2, Value::String(address.to_string()))) {
error!("Failed to bind address parameter: {}", e);
return None;
}
if let Err(e) = stmt.bind((3, Value::String(xpub.to_string()))) {
error!("Failed to bind xpub parameter: {}", e);
return None;
}
if let Ok(State::Row) = stmt.next() {
match stmt.read::<String, _>("address") {
Ok(addr) => Some(addr),
Err(e) => {
error!("Failed to read address column: {}", e);
None
}
}
} else {
None
}
}
pub fn get_next_address_index(db: &Connection, network: &String, xpub: &String) -> (i64, i64) {
let mut stmt = match db.prepare("UPDATE tbl_xpub SET path_idx = path_idx + 1 WHERE network = ? and xpub= ? RETURNING path_idx,id;") {
Ok(s) => s,
Err(e) => {
error!("Failed to prepare xpub index update: {}", e);
return (0, 0);
}
};
if let Err(e) = stmt.bind((1, Value::String(network.to_string()))) {
error!("Failed to bind network parameter: {}", e);
return (0, 0);
}
if let Err(e) = stmt.bind((2, Value::String(xpub.to_string()))) {
error!("Failed to bind xpub parameter: {}", e);
return (0, 0);
}
match stmt.next() {
Ok(State::Row) => match stmt.read::<i64, _>("path_idx") {
Ok(next) => match stmt.read::<i64, _>("id") {
Ok(id) => (id, next),
Err(e) => {
error!("Failed to read id column: {}", e);
(0, 0)
}
},
Err(e) => {
error!("Failed to read path_idx column: {}", e);
(0, 0)
}
},
Err(e) => {
error!("Failed to execute xpub index update: {}", e);
(0, 0)
}
Ok(State::Done) => (0, 0),
}
}
pub fn save_new_address(
db: &Connection,
xpub: i64,
address: &String,
path: &String,
remote_addr: &String,
) {
let mut stmt = match db.prepare(
"INSERT INTO tbl_address(address,path,xpub,remote_address) VALUES(?,?,?,?);
",
) {
Ok(s) => s,
Err(e) => {
error!("Failed to prepare address insert statement: {}", e);
return;
}
};
if let Err(e) = stmt.bind((1, Value::String(address.to_string()))) {
error!("Failed to bind address parameter: {}", e);
return;
}
if let Err(e) = stmt.bind((2, Value::String(path.to_string()))) {
error!("Failed to bind path parameter: {}", e);
return;
}
if let Err(e) = stmt.bind((3, Value::Integer(xpub))) {
error!("Failed to bind xpub parameter: {}", e);
return;
}
if let Err(e) = stmt.bind((4, Value::String(remote_addr.to_string()))) {
error!("Failed to bind remote_addr parameter: {}", e);
return;
}
if let Err(e) = stmt.next() {
error!("Failed to insert address: {}", e);
}
}
pub fn execute_insert(
db: &Connection,
sqltxs: String,
ptx: Vec<(usize, Value)>,
sqlinp: String,
pinp: Vec<(usize, Value)>,
sqlout: String,
pout: Vec<(usize, Value)>,
) -> Result<(), Error> {
let _ = db.execute("BEGIN TRANSACTION");
let mut stmt = match db.prepare(sqltxs.as_str()) {
Ok(s) => s,
Err(err) => {
error!("error preparing sqltxs: {}", err);
let _ = db.execute("ROLLBACK");
return Err(err);
}
};
if let Err(err) = stmt.bind::<&[(_, Value)]>(&ptx[..]) {
error!("error binding transaction parameters: {}", err);
let _ = db.execute("ROLLBACK");
return Err(err);
}
if let Err(err) = stmt.next() {
error!("error inserting transactions {}", err);
let _ = db.execute("ROLLBACK");
} else {
let mut stmt = match db.prepare(sqlinp.as_str()) {
Ok(s) => s,
Err(err) => {
error!("error preparing sqlinp: {}", err);
let _ = db.execute("ROLLBACK");
return Err(err);
}
};
if let Err(err) = stmt.bind::<&[(_, Value)]>(&pinp[..]) {
error!("error binding inputs parameters {}", err);
let _ = db.execute("ROLLBACK");
return Err(err);
}
if let Err(err) = stmt.next() {
error!("error inserting inputs {}", err);
let _ = db.execute("ROLLBACK");
return Err(err);
} else {
let mut stmt = match db.prepare(sqlout.as_str()) {
Ok(s) => s,
Err(err) => {
error!("error preparing sqlout: {}", err);
let _ = db.execute("ROLLBACK");
return Err(err);
}
};
if let Err(err) = stmt.bind::<&[(_, Value)]>(&pout[..]) {
error!("error binding outs parameters {}", err);
let _ = db.execute("ROLLBACK");
return Err(err);
}
if let Err(err) = stmt.next() {
error!("error inserting outs {}", err);
let _ = db.execute("ROLLBACK");
return Err(err);
}
}
}
let _ = db.execute("COMMIT");
Ok(())
}
pub fn get_total_transaction_number(db: Connection, network: &String) -> Result<i64, Error> {
let mut stmt = db
.prepare("SELECT COUNT(*) as total_number FROM tbl_tx where network = ?;")
.map_err(|e| {
error!("Failed to prepare statement: {}", e);
e
})?;
if let Err(e) = stmt.bind((1, Value::String(network.to_string()))) {
error!("Failed to bind network parameter: {}", e);
return Err(e);
}
match stmt.next() {
Ok(State::Row) => match stmt.read::<i64, _>("total_number") {
Ok(val) => Ok(val),
Err(e) => {
error!("Failed to read total_number column: {}", e);
Err(e)
}
},
Ok(sqlite::State::Done) => Ok(0),
Err(err) => {
error!("Failed to execute query: {}", err);
Err(err)
}
}
}

868
src/db/mod.rs Normal file
View File

@@ -0,0 +1,868 @@
pub mod schema;
use log::{error, info, trace};
use sqlx::Row;
use std::collections::HashSet;
use std::path::Path;
#[derive(Clone)]
pub enum DatabasePool {
SQLite(sqlx::SqlitePool),
PostgreSQL(sqlx::PgPool),
}
fn validate_sqlite_path(dsn: &str) -> Result<(), String> {
// Extract file path from DSN formats like "sqlite:path" or "file:path?mode=rwc"
let path_str = if let Some(rest) = dsn.strip_prefix("sqlite:") {
rest.split('?').next().unwrap_or(rest)
} else if let Some(rest) = dsn.strip_prefix("file:") {
rest.split('?').next().unwrap_or(rest)
} else {
dsn
};
// Skip validation for in-memory databases
if path_str == ":memory:" || path_str.is_empty() {
return Ok(());
}
let p = Path::new(path_str);
// Prevent directory traversal
for component in p.components() {
if component == std::path::Component::ParentDir {
return Err("Database path may not contain '..'".to_string());
}
}
// If absolute, block known sensitive system directories
if p.is_absolute() {
let forbidden = [
"/etc", "/proc", "/sys", "/dev", "/usr", "/bin", "/sbin", "/lib", "/opt",
];
for prefix in &forbidden {
if path_str.starts_with(prefix) {
return Err(format!(
"Absolute database path under {} is forbidden",
prefix
));
}
}
}
// If file exists, must be a regular file (not a symlink, device, etc.)
if p.exists() {
if p.is_symlink() {
return Err("Database path must not be a symlink".to_string());
}
let metadata = std::fs::metadata(p)
.map_err(|e| format!("Cannot access database file metadata: {}", e))?;
if !metadata.is_file() {
return Err(
"Database path must point to a regular file, not a directory or device".to_string(),
);
}
}
Ok(())
}
pub async fn open_database(backend: &str, connection_string: &str) -> Result<DatabasePool, String> {
match backend {
"sqlite" => {
validate_sqlite_path(connection_string)?;
let pool = sqlx::sqlite::SqlitePoolOptions::new()
.max_connections(1)
.connect(connection_string)
.await
.map_err(|e| format!("Failed to open SQLite database: {}", e))?;
sqlx::query("PRAGMA journal_mode=WAL")
.execute(&pool)
.await
.map_err(|e| format!("Failed to set WAL mode: {}", e))?;
sqlx::query("PRAGMA busy_timeout=5000")
.execute(&pool)
.await
.map_err(|e| format!("Failed to set busy_timeout: {}", e))?;
Ok(DatabasePool::SQLite(pool))
}
"postgresql" => {
let pool = sqlx::postgres::PgPoolOptions::new()
.max_connections(10)
.connect(connection_string)
.await
.map_err(|e| format!("Failed to open PostgreSQL database: {}", e))?;
Ok(DatabasePool::PostgreSQL(pool))
}
other => Err(format!("Unknown database backend: {}", other)),
}
}
pub async fn create_database(pool: &DatabasePool) -> Result<(), sqlx::Error> {
info!("database sanity check");
match pool {
DatabasePool::SQLite(p) => schema::create_sqlite_schema(p).await,
DatabasePool::PostgreSQL(p) => schema::create_pg_schema(p).await,
}
}
pub async fn check_duplicate_txids(
pool: &DatabasePool,
txids: &[String],
) -> Result<HashSet<String>, sqlx::Error> {
if txids.is_empty() {
return Ok(HashSet::new());
}
let mut duplicates = HashSet::new();
let chunk_size = 500;
for chunk in txids.chunks(chunk_size) {
match pool {
DatabasePool::SQLite(p) => {
let placeholders: Vec<String> = chunk.iter().map(|_| "?".to_string()).collect();
let sql = format!(
"SELECT txid FROM tbl_tx WHERE txid IN ({})",
placeholders.join(",")
);
let mut query = sqlx::query(&sql);
for txid in chunk {
query = query.bind(txid);
}
let rows = query.fetch_all(p).await?;
for row in rows {
if let Ok(txid) = row.try_get::<String, _>("txid") {
duplicates.insert(txid);
}
}
}
DatabasePool::PostgreSQL(p) => {
let placeholders: Vec<String> = chunk
.iter()
.enumerate()
.map(|(i, _)| format!("${}", i + 1))
.collect();
let sql = format!(
"SELECT txid FROM tbl_tx WHERE txid IN ({})",
placeholders.join(",")
);
let mut query = sqlx::query(&sql);
for txid in chunk {
query = query.bind(txid);
}
let rows = query.fetch_all(p).await?;
for row in rows {
if let Ok(txid) = row.try_get::<String, _>("txid") {
duplicates.insert(txid);
}
}
}
}
}
Ok(duplicates)
}
pub async fn get_all_addresses_by_xpub(
pool: &DatabasePool,
xpub: &str,
) -> Result<HashSet<String>, sqlx::Error> {
let mut addresses = HashSet::new();
trace!("get_all_addresses_by_xpub: querying for xpub={}", xpub);
match pool {
DatabasePool::SQLite(p) => {
let rows = sqlx::query(
"SELECT a.address FROM tbl_address a JOIN tbl_xpub x ON a.xpub = x.id WHERE x.xpub = ?",
)
.bind(xpub)
.fetch_all(p)
.await?;
trace!(
"get_all_addresses_by_xpub: SQLite returned {} rows",
rows.len()
);
for row in rows {
if let Ok(addr) = row.try_get::<String, _>("address") {
trace!("get_all_addresses_by_xpub: address={}", &addr);
addresses.insert(addr);
}
}
}
DatabasePool::PostgreSQL(p) => {
let rows = sqlx::query(
"SELECT a.address FROM tbl_address a JOIN tbl_xpub x ON a.xpub = x.id WHERE x.xpub = $1",
)
.bind(xpub)
.fetch_all(p)
.await?;
trace!(
"get_all_addresses_by_xpub: PostgreSQL returned {} rows",
rows.len()
);
for row in rows {
if let Ok(addr) = row.try_get::<String, _>("address") {
trace!("get_all_addresses_by_xpub: address={}", &addr);
addresses.insert(addr);
}
}
}
}
trace!(
"get_all_addresses_by_xpub: returning {} addresses",
addresses.len()
);
Ok(addresses)
}
pub async fn insert_xpub(pool: &DatabasePool, network: &str, xpub: &str) {
if xpub.is_empty() {
return;
}
trace!("going to insert: {} xpub:{}", network, xpub);
match pool {
DatabasePool::SQLite(p) => {
if let Err(e) =
sqlx::query("INSERT OR IGNORE INTO tbl_xpub(network, xpub) VALUES(?, ?)")
.bind(network)
.bind(xpub)
.execute(p)
.await
{
error!("Failed to insert xpub: {}", e);
}
}
DatabasePool::PostgreSQL(p) => {
if let Err(e) = sqlx::query(
"INSERT INTO tbl_xpub(network, xpub) VALUES($1, $2) ON CONFLICT DO NOTHING",
)
.bind(network)
.bind(xpub)
.execute(p)
.await
{
error!("Failed to insert xpub: {}", e);
}
}
}
}
pub async fn get_last_used_address_by_ip(
pool: &DatabasePool,
network: &str,
xpub: &str,
address: &str,
) -> Option<String> {
match pool {
DatabasePool::SQLite(p) => {
let result = sqlx::query(
"SELECT tbl_address.address FROM tbl_xpub JOIN tbl_address ON(tbl_xpub.id = tbl_address.xpub) WHERE tbl_xpub.network = ? AND tbl_address.remote_address = ? AND tbl_xpub.xpub = ? ORDER BY tbl_address.date_create DESC LIMIT 1",
)
.bind(network)
.bind(address)
.bind(xpub)
.fetch_optional(p)
.await;
match result {
Ok(Some(row)) => row.try_get::<String, _>("address").ok(),
Ok(None) => None,
Err(e) => {
error!("Failed to query last used address: {}", e);
None
}
}
}
DatabasePool::PostgreSQL(p) => {
let result = sqlx::query(
"SELECT tbl_address.address FROM tbl_xpub JOIN tbl_address ON(tbl_xpub.id = tbl_address.xpub) WHERE tbl_xpub.network = $1 AND tbl_address.remote_address = $2 AND tbl_xpub.xpub = $3 ORDER BY tbl_address.date_create DESC LIMIT 1",
)
.bind(network)
.bind(address)
.bind(xpub)
.fetch_optional(p)
.await;
match result {
Ok(Some(row)) => row.try_get::<String, _>("address").ok(),
Ok(None) => None,
Err(e) => {
error!("Failed to query last used address: {}", e);
None
}
}
}
}
}
pub async fn get_next_address_index(pool: &DatabasePool, network: &str, xpub: &str) -> (i64, i64) {
match pool {
DatabasePool::SQLite(p) => {
let result = sqlx::query(
"UPDATE tbl_xpub SET path_idx = path_idx + 1 WHERE network = ? AND xpub = ? RETURNING path_idx, id",
)
.bind(network)
.bind(xpub)
.fetch_optional(p)
.await;
match result {
Ok(Some(row)) => {
let idx = row.try_get::<i64, _>("path_idx").unwrap_or(0);
let id = row.try_get::<i64, _>("id").unwrap_or(0);
(id, idx)
}
Ok(None) => (0, 0),
Err(e) => {
error!("Failed to get next address index: {}", e);
(0, 0)
}
}
}
DatabasePool::PostgreSQL(p) => {
let result = sqlx::query(
"UPDATE tbl_xpub SET path_idx = path_idx + 1 WHERE network = $1 AND xpub = $2 RETURNING path_idx, id",
)
.bind(network)
.bind(xpub)
.fetch_optional(p)
.await;
match result {
Ok(Some(row)) => {
let idx = row.try_get::<i32, _>("path_idx").unwrap_or(0) as i64;
let id = row.try_get::<i32, _>("id").unwrap_or(0) as i64;
(id, idx)
}
Ok(None) => (0, 0),
Err(e) => {
error!("Failed to get next address index: {}", e);
(0, 0)
}
}
}
}
}
pub async fn save_new_address(
pool: &DatabasePool,
xpub: i64,
address: &str,
path: &str,
remote_addr: &str,
) {
match pool {
DatabasePool::SQLite(p) => {
if let Err(e) = sqlx::query(
"INSERT INTO tbl_address(address, path, xpub, remote_address) VALUES(?, ?, ?, ?)",
)
.bind(address)
.bind(path)
.bind(xpub)
.bind(remote_addr)
.execute(p)
.await
{
error!("Failed to save address: {}", e);
}
}
DatabasePool::PostgreSQL(p) => {
if let Err(e) = sqlx::query(
"INSERT INTO tbl_address(address, path, xpub, remote_address) VALUES($1, $2, $3, $4)",
)
.bind(address)
.bind(path)
.bind(xpub as i32)
.bind(remote_addr)
.execute(p)
.await
{
error!("Failed to save address: {}", e);
}
}
}
}
#[derive(Debug, Clone)]
pub struct InsertTxData {
pub txid: String,
pub wtxid: String,
pub ntxid: String,
pub raw_hex: String,
pub locktime: String,
pub reqid: String,
pub network: String,
pub our_address: String,
pub our_fees: String,
}
#[derive(Debug, Clone)]
pub struct InsertInpData {
pub txid: String,
pub in_txid: String,
pub in_vout: String,
}
#[derive(Debug, Clone)]
pub struct InsertOutData {
pub txid: String,
pub vout: i64,
pub script_pubkey: String,
pub amount: i64,
}
pub async fn execute_insert(
pool: &DatabasePool,
txs: &[InsertTxData],
inps: &[InsertInpData],
outs: &[InsertOutData],
) -> Result<(), sqlx::Error> {
match pool {
DatabasePool::SQLite(p) => {
let mut tx = p.begin().await?;
for item in txs {
sqlx::query(
"INSERT INTO tbl_tx (txid, wtxid, ntxid, tx, locktime, reqid, network, our_address, our_fees) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)"
)
.bind(&item.txid)
.bind(&item.wtxid)
.bind(&item.ntxid)
.bind(&item.raw_hex)
.bind(&item.locktime)
.bind(&item.reqid)
.bind(&item.network)
.bind(&item.our_address)
.bind(&item.our_fees)
.execute(&mut *tx)
.await?;
}
for item in inps {
sqlx::query("INSERT INTO tbl_inp (txid, in_txid, in_vout) VALUES (?, ?, ?)")
.bind(&item.txid)
.bind(&item.in_txid)
.bind(&item.in_vout)
.execute(&mut *tx)
.await?;
}
for item in outs {
sqlx::query(
"INSERT INTO tbl_out (txid, vout, script_pubkey, amount) VALUES (?, ?, ?, ?)",
)
.bind(&item.txid)
.bind(item.vout)
.bind(&item.script_pubkey)
.bind(item.amount)
.execute(&mut *tx)
.await?;
}
tx.commit().await?;
}
DatabasePool::PostgreSQL(p) => {
let mut tx = p.begin().await?;
for item in txs {
let locktime_i64: i64 = item.locktime.parse().unwrap_or(0);
sqlx::query(
"INSERT INTO tbl_tx (txid, wtxid, ntxid, tx, locktime, reqid, network, our_address, our_fees) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9)"
)
.bind(&item.txid)
.bind(&item.wtxid)
.bind(&item.ntxid)
.bind(&item.raw_hex)
.bind(locktime_i64)
.bind(&item.reqid)
.bind(&item.network)
.bind(&item.our_address)
.bind(&item.our_fees)
.execute(&mut *tx)
.await?;
}
for item in inps {
let in_vout_i32: i32 = item.in_vout.parse().unwrap_or(0);
sqlx::query("INSERT INTO tbl_inp (txid, in_txid, in_vout) VALUES ($1, $2, $3)")
.bind(&item.txid)
.bind(&item.in_txid)
.bind(in_vout_i32)
.execute(&mut *tx)
.await?;
}
for item in outs {
sqlx::query(
"INSERT INTO tbl_out (txid, vout, script_pubkey, amount) VALUES ($1, $2, $3, $4)",
)
.bind(&item.txid)
.bind(item.vout as i32)
.bind(&item.script_pubkey)
.bind(item.amount.to_string())
.execute(&mut *tx)
.await?;
}
tx.commit().await?;
}
}
Ok(())
}
pub async fn get_total_transaction_number(
pool: &DatabasePool,
network: &str,
) -> Result<i64, sqlx::Error> {
match pool {
DatabasePool::SQLite(p) => {
let row = sqlx::query("SELECT COUNT(*) as total_number FROM tbl_tx WHERE network = ?")
.bind(network)
.fetch_one(p)
.await?;
Ok(row.try_get::<i64, _>("total_number").unwrap_or(0))
}
DatabasePool::PostgreSQL(p) => {
let row = sqlx::query("SELECT COUNT(*) as total_number FROM tbl_tx WHERE network = $1")
.bind(network)
.fetch_one(p)
.await?;
Ok(row.try_get::<i64, _>("total_number").unwrap_or(0))
}
}
}
pub async fn update_tx_status(
pool: &DatabasePool,
txid: &str,
status: i32,
push_err: Option<&str>,
) -> Result<(), sqlx::Error> {
match pool {
DatabasePool::SQLite(p) => {
if let Some(err) = push_err {
sqlx::query("UPDATE tbl_tx SET status = ?, push_err = ? WHERE txid = ?")
.bind(status)
.bind(err)
.bind(txid)
.execute(p)
.await?;
} else {
sqlx::query("UPDATE tbl_tx SET status = ? WHERE txid = ?")
.bind(status)
.bind(txid)
.execute(p)
.await?;
}
}
DatabasePool::PostgreSQL(p) => {
if let Some(err) = push_err {
sqlx::query("UPDATE tbl_tx SET status = $1, push_err = $2 WHERE txid = $3")
.bind(status)
.bind(err)
.bind(txid)
.execute(p)
.await?;
} else {
sqlx::query("UPDATE tbl_tx SET status = $1 WHERE txid = $2")
.bind(status)
.bind(txid)
.execute(p)
.await?;
}
}
}
Ok(())
}
#[derive(Debug, Clone)]
pub struct TxRow {
pub txid: String,
pub tx: String,
pub locktime: i64,
pub network: String,
pub status: i64,
}
pub async fn get_pending_txs(
pool: &DatabasePool,
network: &str,
locktime_threshold: i64,
bestblock_height: i64,
bestblock_time: i64,
) -> Result<Vec<TxRow>, sqlx::Error> {
let mut results = Vec::new();
match pool {
DatabasePool::SQLite(p) => {
let rows = sqlx::query(
"SELECT txid, tx, locktime, network, status FROM tbl_tx WHERE network = ? AND status = 0 AND (locktime < ? OR (locktime > ? AND locktime < ?))",
)
.bind(network)
.bind(bestblock_height)
.bind(locktime_threshold)
.bind(bestblock_time)
.fetch_all(p)
.await?;
for row in rows {
results.push(TxRow {
txid: row.try_get("txid")?,
tx: row.try_get("tx")?,
locktime: row.try_get("locktime")?,
network: row.try_get("network")?,
status: row.try_get("status")?,
});
}
}
DatabasePool::PostgreSQL(p) => {
let rows = sqlx::query(
"SELECT txid, tx, locktime, network, status FROM tbl_tx WHERE network = $1 AND status = 0 AND (locktime < $2 OR (locktime > $3 AND locktime < $4))",
)
.bind(network)
.bind(bestblock_height)
.bind(locktime_threshold)
.bind(bestblock_time)
.fetch_all(p)
.await?;
for row in rows {
results.push(TxRow {
txid: row.try_get("txid")?,
tx: row.try_get("tx")?,
locktime: row.try_get::<i64, _>("locktime").unwrap_or(0),
network: row.try_get("network")?,
status: row.try_get::<i32, _>("status").unwrap_or(0) as i64,
});
}
}
}
Ok(results)
}
#[derive(Debug, Clone)]
pub struct StatsRow {
pub report_date: String,
pub chain: String,
pub totals: i64,
pub waiting: i64,
pub sent: i64,
pub failed: i64,
pub waiting_profit: i64,
pub sent_profit: i64,
pub missed_profit: i64,
pub unique_inputs: i64,
}
pub async fn get_stats(pool: &DatabasePool, chain: &str) -> Result<Vec<StatsRow>, sqlx::Error> {
let mut results = Vec::new();
match pool {
DatabasePool::SQLite(p) => {
let rows = sqlx::query(
"SELECT report_date, chain, totals, waiting, sent, failed, waiting_profit, sent_profit, missed_profit, unique_inputs FROM tbl_stats WHERE chain = ?",
)
.bind(chain)
.fetch_all(p)
.await?;
for row in rows {
results.push(StatsRow {
report_date: row.try_get("report_date").unwrap_or_default(),
chain: row.try_get("chain").unwrap_or_default(),
totals: row.try_get("totals").unwrap_or(0),
waiting: row.try_get("waiting").unwrap_or(0),
sent: row.try_get("sent").unwrap_or(0),
failed: row.try_get("failed").unwrap_or(0),
waiting_profit: row.try_get("waiting_profit").unwrap_or(0),
sent_profit: row.try_get("sent_profit").unwrap_or(0),
missed_profit: row.try_get("missed_profit").unwrap_or(0),
unique_inputs: row.try_get("unique_inputs").unwrap_or(0),
});
}
}
DatabasePool::PostgreSQL(p) => {
let rows = sqlx::query(
"SELECT report_date, chain, totals, waiting, sent, failed, waiting_profit, sent_profit, missed_profit, unique_inputs FROM tbl_stats WHERE chain = $1",
)
.bind(chain)
.fetch_all(p)
.await?;
for row in rows {
results.push(StatsRow {
report_date: row.try_get("report_date").unwrap_or_default(),
chain: row.try_get("chain").unwrap_or_default(),
totals: row.try_get::<i32, _>("totals").unwrap_or(0) as i64,
waiting: row.try_get::<i32, _>("waiting").unwrap_or(0) as i64,
sent: row.try_get::<i32, _>("sent").unwrap_or(0) as i64,
failed: row.try_get::<i32, _>("failed").unwrap_or(0) as i64,
waiting_profit: row.try_get::<i32, _>("waiting_profit").unwrap_or(0) as i64,
sent_profit: row.try_get::<i32, _>("sent_profit").unwrap_or(0) as i64,
missed_profit: row.try_get::<i32, _>("missed_profit").unwrap_or(0) as i64,
unique_inputs: row.try_get::<i32, _>("unique_inputs").unwrap_or(0) as i64,
});
}
}
}
Ok(results)
}
#[derive(Debug, Clone)]
pub struct SearchTxRow {
pub status: String,
pub tx: String,
pub our_address: String,
pub our_fees: String,
pub reqid: String,
}
pub async fn search_tx(
pool: &DatabasePool,
txid: &str,
) -> Result<Option<SearchTxRow>, sqlx::Error> {
match pool {
DatabasePool::SQLite(p) => {
let result = sqlx::query("SELECT * FROM tbl_tx WHERE txid = ? LIMIT 1")
.bind(txid)
.fetch_optional(p)
.await?;
Ok(result.map(|row| SearchTxRow {
status: row.try_get::<i64, _>("status").unwrap_or(0).to_string(),
tx: row.try_get::<String, _>("tx").unwrap_or_default(),
our_address: row.try_get::<String, _>("our_address").unwrap_or_default(),
our_fees: row.try_get::<String, _>("our_fees").unwrap_or_default(),
reqid: row.try_get::<String, _>("reqid").unwrap_or_default(),
}))
}
DatabasePool::PostgreSQL(p) => {
let result = sqlx::query("SELECT * FROM tbl_tx WHERE txid = $1 LIMIT 1")
.bind(txid)
.fetch_optional(p)
.await?;
Ok(result.map(|row| SearchTxRow {
status: row.try_get::<i32, _>("status").unwrap_or(0).to_string(),
tx: row.try_get::<String, _>("tx").unwrap_or_default(),
our_address: row.try_get::<String, _>("our_address").unwrap_or_default(),
our_fees: row.try_get::<String, _>("our_fees").unwrap_or_default(),
reqid: row.try_get::<String, _>("reqid").unwrap_or_default(),
}))
}
}
}
pub async fn calculate_and_upsert_stats(
pool: &DatabasePool,
chain: &str,
) -> Result<(), sqlx::Error> {
if !chain
.chars()
.all(|c| c.is_alphanumeric() || c == '-' || c == '_')
|| chain.is_empty()
{
error!("Invalid chain name: {chain}");
return Ok(());
}
match pool {
DatabasePool::SQLite(p) => {
sqlx::query("DELETE FROM tbl_stats WHERE chain = ?")
.bind(chain)
.execute(p)
.await?;
sqlx::query(
"INSERT INTO tbl_stats (
report_date, chain, totals, waiting, sent, failed,
waiting_profit, sent_profit, missed_profit, unique_inputs
)
SELECT
CURRENT_TIMESTAMP,
?,
(SELECT COUNT(*) FROM tbl_tx WHERE network = ?),
(SELECT COUNT(*) FROM tbl_tx WHERE status = 0 AND network = ?),
(SELECT COUNT(*) FROM tbl_tx WHERE status = 1 AND network = ?),
(SELECT COUNT(*) FROM tbl_tx WHERE status = 2 AND network = ?),
(SELECT IFNULL(SUM(our_fees),0) FROM tbl_tx WHERE status = 0 AND network = ?),
(SELECT IFNULL(SUM(our_fees),0) FROM tbl_tx WHERE status = 1 AND network = ?),
(SELECT IFNULL(SUM(our_fees),0) FROM tbl_tx WHERE status = 2 AND network = ?),
(SELECT COUNT(DISTINCT tbl_inp.in_txid)
FROM tbl_inp
JOIN tbl_tx ON tbl_inp.txid = tbl_tx.txid
WHERE tbl_tx.status = 0 AND tbl_tx.network = ?)
ON CONFLICT(chain) DO UPDATE SET
report_date = excluded.report_date,
totals = excluded.totals,
waiting = excluded.waiting,
sent = excluded.sent,
failed = excluded.failed,
waiting_profit = excluded.waiting_profit,
sent_profit = excluded.sent_profit,
missed_profit = excluded.missed_profit,
unique_inputs = excluded.unique_inputs",
)
.bind(chain)
.bind(chain)
.bind(chain)
.bind(chain)
.bind(chain)
.bind(chain)
.bind(chain)
.bind(chain)
.bind(chain)
.execute(p)
.await?;
}
DatabasePool::PostgreSQL(p) => {
sqlx::query("DELETE FROM tbl_stats WHERE chain = $1")
.bind(chain)
.execute(p)
.await?;
sqlx::query(
"INSERT INTO tbl_stats (
report_date, chain, totals, waiting, sent, failed,
waiting_profit, sent_profit, missed_profit, unique_inputs
)
SELECT
CURRENT_TIMESTAMP,
$1,
(SELECT COUNT(*) FROM tbl_tx WHERE network = $1),
(SELECT COUNT(*) FROM tbl_tx WHERE status = 0 AND network = $1),
(SELECT COUNT(*) FROM tbl_tx WHERE status = 1 AND network = $1),
(SELECT COUNT(*) FROM tbl_tx WHERE status = 2 AND network = $1),
(SELECT COALESCE(SUM(CAST(our_fees AS BIGINT)),0) FROM tbl_tx WHERE status = 0 AND network = $1),
(SELECT COALESCE(SUM(CAST(our_fees AS BIGINT)),0) FROM tbl_tx WHERE status = 1 AND network = $1),
(SELECT COALESCE(SUM(CAST(our_fees AS BIGINT)),0) FROM tbl_tx WHERE status = 2 AND network = $1),
(SELECT COUNT(DISTINCT tbl_inp.in_txid)
FROM tbl_inp
JOIN tbl_tx ON tbl_inp.txid = tbl_tx.txid
WHERE tbl_tx.status = 0 AND tbl_tx.network = $1)
ON CONFLICT(chain) DO UPDATE SET
report_date = excluded.report_date,
totals = excluded.totals,
waiting = excluded.waiting,
sent = excluded.sent,
failed = excluded.failed,
waiting_profit = excluded.waiting_profit,
sent_profit = excluded.sent_profit,
missed_profit = excluded.missed_profit,
unique_inputs = excluded.unique_inputs",
)
.bind(chain)
.execute(p)
.await?;
}
}
info!("tbl_stats creation success");
Ok(())
}

240
src/db/schema.rs Normal file
View File

@@ -0,0 +1,240 @@
use sqlx::{PgPool, SqlitePool};
pub async fn create_sqlite_schema(pool: &SqlitePool) -> Result<(), sqlx::Error> {
sqlx::query(
"CREATE TABLE IF NOT EXISTS tbl_tx (
txid TEXT PRIMARY KEY,
date_creation TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
date_update TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
wtxid TEXT,
ntxid TEXT,
tx TEXT,
locktime INTEGER,
network TEXT,
network_fees TEXT,
reqid TEXT,
our_fees TEXT,
our_address TEXT,
status INTEGER DEFAULT 0
)",
)
.execute(pool)
.await?;
let _ = sqlx::query("ALTER TABLE tbl_tx ADD COLUMN push_err TEXT")
.execute(pool)
.await;
sqlx::query(
"CREATE TABLE IF NOT EXISTS tbl_inp (
id INTEGER,
txid TEXT,
in_txid TEXT,
in_vout INTEGER
)",
)
.execute(pool)
.await?;
sqlx::query(
"CREATE UNIQUE INDEX IF NOT EXISTS idx_inp_unique ON tbl_inp(txid, in_txid, in_vout)",
)
.execute(pool)
.await?;
sqlx::query(
"CREATE TABLE IF NOT EXISTS tbl_out (
id INTEGER,
txid TEXT,
script_pubkey TEXT,
amount TEXT,
vout INTEGER
)",
)
.execute(pool)
.await?;
sqlx::query(
"CREATE UNIQUE INDEX IF NOT EXISTS idx_out_unique ON tbl_out(txid, script_pubkey, amount, vout)",
)
.execute(pool)
.await?;
sqlx::query(
"CREATE TABLE IF NOT EXISTS tbl_xpub (
id INTEGER PRIMARY KEY,
network TEXT,
xpub TEXT,
date_create TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
path_idx INTEGER DEFAULT -1
)",
)
.execute(pool)
.await?;
sqlx::query("CREATE UNIQUE INDEX IF NOT EXISTS idx_xpub ON tbl_xpub(network, xpub)")
.execute(pool)
.await?;
sqlx::query(
"CREATE TABLE IF NOT EXISTS tbl_address (
address TEXT PRIMARY KEY,
path TEXT NOT NULL,
date_create TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
xpub INTEGER,
remote_address TEXT
)",
)
.execute(pool)
.await?;
sqlx::query(
"CREATE TABLE IF NOT EXISTS tbl_stats (
report_date TEXT,
chain TEXT,
totals INTEGER,
waiting INTEGER,
sent INTEGER,
failed INTEGER,
waiting_profit INTEGER,
sent_profit INTEGER,
missed_profit INTEGER,
unique_inputs INTEGER
)",
)
.execute(pool)
.await?;
sqlx::query("DROP INDEX IF EXISTS idx_stats_chain")
.execute(pool)
.await?;
sqlx::query("CREATE UNIQUE INDEX IF NOT EXISTS idx_stats_chain ON tbl_stats(chain)")
.execute(pool)
.await?;
sqlx::query("UPDATE tbl_tx SET network='bitcoin' WHERE network='mainnet'")
.execute(pool)
.await?;
Ok(())
}
pub async fn create_pg_schema(pool: &PgPool) -> Result<(), sqlx::Error> {
sqlx::query(
"CREATE TABLE IF NOT EXISTS tbl_tx (
txid TEXT PRIMARY KEY,
date_creation TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
date_update TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
wtxid TEXT,
ntxid TEXT,
tx TEXT,
locktime BIGINT,
network TEXT,
network_fees TEXT,
reqid TEXT,
our_fees TEXT,
our_address TEXT,
status INTEGER DEFAULT 0,
push_err TEXT
)",
)
.execute(pool)
.await?;
// Migrate pre-existing deployments where locktime was created as INTEGER.
// nLockTime is a u32 (up to 4_294_967_295); INTEGER (i32) cannot hold
// timestamp-based locktimes after 2038-01-19.
let _ = sqlx::query("ALTER TABLE tbl_tx ALTER COLUMN locktime TYPE BIGINT")
.execute(pool)
.await;
sqlx::query(
"CREATE TABLE IF NOT EXISTS tbl_inp (
id SERIAL PRIMARY KEY,
txid TEXT,
in_txid TEXT,
in_vout INTEGER
)",
)
.execute(pool)
.await?;
sqlx::query(
"CREATE UNIQUE INDEX IF NOT EXISTS idx_inp_unique ON tbl_inp(txid, in_txid, in_vout)",
)
.execute(pool)
.await?;
sqlx::query(
"CREATE TABLE IF NOT EXISTS tbl_out (
id SERIAL PRIMARY KEY,
txid TEXT,
script_pubkey TEXT,
amount TEXT,
vout INTEGER
)",
)
.execute(pool)
.await?;
sqlx::query(
"CREATE UNIQUE INDEX IF NOT EXISTS idx_out_unique ON tbl_out(txid, script_pubkey, amount, vout)",
)
.execute(pool)
.await?;
sqlx::query(
"CREATE TABLE IF NOT EXISTS tbl_xpub (
id SERIAL PRIMARY KEY,
network TEXT,
xpub TEXT,
date_create TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
path_idx INTEGER DEFAULT -1
)",
)
.execute(pool)
.await?;
sqlx::query("CREATE UNIQUE INDEX IF NOT EXISTS idx_xpub ON tbl_xpub(network, xpub)")
.execute(pool)
.await?;
sqlx::query(
"CREATE TABLE IF NOT EXISTS tbl_address (
address TEXT PRIMARY KEY,
path TEXT NOT NULL,
date_create TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
xpub INTEGER,
remote_address TEXT
)",
)
.execute(pool)
.await?;
sqlx::query(
"CREATE TABLE IF NOT EXISTS tbl_stats (
report_date TEXT,
chain TEXT,
totals INTEGER,
waiting INTEGER,
sent INTEGER,
failed INTEGER,
waiting_profit INTEGER,
sent_profit INTEGER,
missed_profit INTEGER,
unique_inputs INTEGER
)",
)
.execute(pool)
.await?;
sqlx::query("CREATE UNIQUE INDEX IF NOT EXISTS idx_stats_chain ON tbl_stats(chain)")
.execute(pool)
.await?;
sqlx::query("UPDATE tbl_tx SET network='bitcoin' WHERE network='mainnet'")
.execute(pool)
.await?;
Ok(())
}

View File

@@ -148,12 +148,12 @@ pub fn calculate_fingerprint(tpub: &str) -> Result<String, String> {
fn base58check_decode(s: &str) -> Result<Vec<u8>, String> { fn base58check_decode(s: &str) -> Result<Vec<u8>, String> {
let data = bs58::decode(s).into_vec().map_err(|e| e.to_string())?; let data = bs58::decode(s).into_vec().map_err(|e| e.to_string())?;
if data.len() < 4 { if data.len() < 4 {
return Err("Data troppo corta".to_string()); return Err("Data too short".to_string());
} }
let (payload, checksum) = data.split_at(data.len() - 4); let (payload, checksum) = data.split_at(data.len() - 4);
let hash = Sha256::digest(Sha256::digest(payload)); let hash = Sha256::digest(Sha256::digest(payload));
if hash[0..4] != checksum[..] { if hash[0..4] != checksum[..] {
return Err("Checksum invalido".to_string()); return Err("Invalid checksum".to_string());
} }
Ok(payload.to_vec()) Ok(payload.to_vec())
} }
@@ -168,7 +168,7 @@ fn convert_to(zpub: &str, prefix: BS58Prefix) -> Result<String, String> {
let mut data = base58check_decode(zpub)?; let mut data = base58check_decode(zpub)?;
if data.len() < 4 { if data.len() < 4 {
return Err("Non è una zpub valida.".to_string()); return Err("Not a valid zpub".to_string());
} }
data.splice( data.splice(
0..4, 0..4,
@@ -207,7 +207,7 @@ pub fn new_address_from_xpub(
fn main() -> Result<(), Box<dyn std::error::Error>>{ fn main() -> Result<(), Box<dyn std::error::Error>>{
match convert_to(zpub,BS58Prefix::Tpub) { match convert_to(zpub,BS58Prefix::Tpub) {
Ok(tpub) => println!("XPUB: {}", tpub), Ok(tpub) => println!("XPUB: {}", tpub),
Err(e) => eprintln!("Errore: {}", e), Err(e) => eprintln!("Error: {}", e),
} }
let fingerprint = base58check_encode(&calculate_fingerprint(zpub)); let fingerprint = base58check_encode(&calculate_fingerprint(zpub));
println!("ZPUB: {}, FINGERPRINT: {}",zpub,fingerprint); println!("ZPUB: {}, FINGERPRINT: {}",zpub,fingerprint);

View File

@@ -1,15 +1,62 @@
use bal_server::db::open_db; use bal_server::db::{DatabasePool, create_database, open_database};
use sqlite::State; use sqlx::Row;
use std::fs;
#[test] #[tokio::test]
fn test_open_db_blocks_traversal() { async fn test_open_database_sqlite() {
let res = open_db("../etc/passwd"); let pool = open_database("sqlite", "sqlite::memory:").await;
assert!(pool.is_ok(), "Opening SQLite in-memory should succeed");
}
#[tokio::test]
async fn test_open_database_invalid_backend() {
let pool = open_database("oracle", "connection_string").await;
assert!(pool.is_err(), "Unknown backend should fail");
}
#[tokio::test]
async fn test_create_database_sqlite() {
let pool = open_database("sqlite", "sqlite::memory:").await.unwrap();
let result = create_database(&pool).await;
assert!(result.is_ok(), "Creating SQLite schema should succeed");
}
#[tokio::test]
async fn test_create_database_is_idempotent() {
let pool = open_database("sqlite", "sqlite::memory:").await.unwrap();
create_database(&pool).await.unwrap();
let result = create_database(&pool).await;
assert!(
result.is_ok(),
"Creating schema twice should succeed (idempotent)"
);
}
#[tokio::test]
async fn test_sqlite_wal_mode() {
let tmp_path = std::env::temp_dir().join("tmp_test_wal_mode.db");
let path_str = tmp_path.to_str().unwrap();
let _ = std::fs::remove_file(&tmp_path);
let dsn = format!("sqlite:{}?mode=rwc", path_str);
let pool = open_database("sqlite", &dsn).await.unwrap();
if let DatabasePool::SQLite(p) = &pool {
let row = sqlx::query("PRAGMA journal_mode")
.fetch_one(p)
.await
.unwrap();
let mode: String = row.try_get(0).unwrap();
assert_eq!(mode, "wal", "SQLite journal mode should be WAL");
}
drop(pool);
let _ = std::fs::remove_file(&tmp_path);
let _ = std::fs::remove_file(format!("{}-shm", path_str));
let _ = std::fs::remove_file(format!("{}-wal", path_str));
}
#[tokio::test]
async fn test_open_database_blocks_traversal() {
let res = open_database("sqlite", "sqlite:../etc/passwd").await;
assert!(res.is_err(), "Path with '..' should be rejected"); assert!(res.is_err(), "Path with '..' should be rejected");
let err = match res { let err = res.err().unwrap();
Err(e) => e,
Ok(_) => panic!("Expected error for traversal path"),
};
assert!( assert!(
err.contains("'..'"), err.contains("'..'"),
"Error should mention directory traversal: {}", "Error should mention directory traversal: {}",
@@ -17,15 +64,17 @@ fn test_open_db_blocks_traversal() {
); );
} }
#[test] #[tokio::test]
fn test_open_db_blocks_forbidden_absolute() { async fn test_open_database_blocks_forbidden_absolute() {
for path in ["/etc/passwd", "/proc/self/mem", "/dev/null", "/usr/bin/ls"] { for path in [
let res = open_db(path); "sqlite:/etc/passwd",
"sqlite:/proc/self/mem",
"sqlite:/dev/null",
"sqlite:/usr/bin/ls",
] {
let res = open_database("sqlite", path).await;
assert!(res.is_err(), "Absolute path {} should be rejected", path); assert!(res.is_err(), "Absolute path {} should be rejected", path);
let err = match res { let err = res.err().unwrap();
Err(e) => e,
Ok(_) => panic!("Expected error for forbidden path {}", path),
};
assert!( assert!(
err.contains("forbidden"), err.contains("forbidden"),
"Error should mention forbidden prefix: {}", "Error should mention forbidden prefix: {}",
@@ -34,59 +83,26 @@ fn test_open_db_blocks_forbidden_absolute() {
} }
} }
#[test] #[tokio::test]
fn test_open_db_allows_relative() { async fn test_open_database_rejects_symlink() {
let test_path = "tmp_test_bal.db"; let tmp_dir = std::env::temp_dir();
let _ = fs::remove_file(test_path); let real = tmp_dir.join("tmp_test_real_symlink.db");
let res = open_db(test_path); let link = tmp_dir.join("tmp_test_link_symlink.db");
assert!(res.is_ok(), "Valid relative path should be allowed"); let _ = std::fs::remove_file(&real);
let db = res.unwrap(); let _ = std::fs::remove_file(&link);
drop(db); std::fs::File::create(&real).unwrap();
let _ = fs::remove_file(test_path); std::os::unix::fs::symlink(&real, &link).unwrap();
}
#[test] let dsn = format!("sqlite:{}?mode=rwc", link.to_str().unwrap());
fn test_open_db_wal_pragmas_set() { let res = open_database("sqlite", &dsn).await;
let test_path = "tmp_test_wal.db";
let _ = fs::remove_file(test_path);
let _ = fs::remove_file(format!("{}-shm", test_path));
let _ = fs::remove_file(format!("{}-wal", test_path));
let db = open_db(test_path).expect("Should open DB");
let mut stmt = db.prepare("PRAGMA journal_mode;").unwrap();
if let Ok(State::Row) = stmt.next() {
let mode: String = stmt.read(0).unwrap();
assert_eq!(mode, "wal", "SQLite journal mode should be WAL");
} else {
panic!("Could not read journal_mode pragma");
}
let _ = fs::remove_file(test_path);
let _ = fs::remove_file(format!("{}-shm", test_path));
let _ = fs::remove_file(format!("{}-wal", test_path));
}
#[test]
fn test_open_db_rejects_symlink() {
let real = "tmp_test_real.db";
let link = "tmp_test_link.db";
let _ = fs::remove_file(real);
let _ = fs::remove_file(link);
fs::File::create(real).unwrap();
std::os::unix::fs::symlink(real, link).unwrap();
let res = open_db(link);
assert!(res.is_err(), "Symlink DB path should be rejected"); assert!(res.is_err(), "Symlink DB path should be rejected");
let err = match res { let err = res.err().unwrap();
Err(e) => e,
Ok(_) => panic!("Expected error for symlink"),
};
assert!( assert!(
err.contains("symlink"), err.contains("symlink"),
"Error should mention symlink: {}", "Error should mention symlink: {}",
err err
); );
let _ = fs::remove_file(real); let _ = std::fs::remove_file(&real);
let _ = fs::remove_file(link); let _ = std::fs::remove_file(&link);
} }

View File

@@ -1,60 +1,48 @@
use bal_server::db::{get_all_addresses_by_xpub, open_db}; use bal_server::db::{DatabasePool, create_database, get_all_addresses_by_xpub, open_database};
use sqlite::Value;
fn setup_db_with_xpub() -> sqlite::Connection { async fn setup_db_with_xpub() -> DatabasePool {
let db = open_db(":memory:").unwrap(); let pool = open_database("sqlite", "sqlite::memory:").await.unwrap();
let _ = db.execute( create_database(&pool).await.unwrap();
"CREATE TABLE tbl_xpub (id INTEGER PRIMARY KEY, network TEXT, xpub TEXT, path_idx INTEGER DEFAULT -1);"
); if let DatabasePool::SQLite(p) = &pool {
let _ = db.execute( sqlx::query("INSERT INTO tbl_xpub(id, network, xpub) VALUES(1, 'testnet', 'tpub_test')")
"CREATE TABLE tbl_address (address TEXT PRIMARY KEY, path TEXT, xpub INTEGER, remote_address TEXT);" .execute(p)
); .await
// Insert test xpub
let mut stmt = db
.prepare("INSERT INTO tbl_xpub(id, network, xpub) VALUES(?, ?, ?);")
.unwrap(); .unwrap();
stmt.bind((1, Value::Integer(1))).unwrap();
stmt.bind((2, Value::String("testnet".to_string())))
.unwrap();
stmt.bind((3, Value::String("tpub_test".to_string())))
.unwrap();
let _ = stmt.next();
drop(stmt);
// Insert test addresses
for addr in ["addr1", "addr2", "addr3"] { for addr in ["addr1", "addr2", "addr3"] {
let mut stmt = db sqlx::query("INSERT INTO tbl_address(address, path, xpub) VALUES(?, 'm/0/1', 1)")
.prepare("INSERT INTO tbl_address(address, path, xpub) VALUES(?, ?, ?);") .bind(addr)
.execute(p)
.await
.unwrap(); .unwrap();
stmt.bind((1, Value::String(addr.to_string()))).unwrap();
stmt.bind((2, Value::String("m/0/1".to_string()))).unwrap();
stmt.bind((3, Value::Integer(1))).unwrap();
let _ = stmt.next();
drop(stmt);
} }
db }
pool
} }
#[test] #[tokio::test]
fn test_get_all_addresses_by_xpub_returns_known() { async fn test_get_all_addresses_by_xpub_returns_known() {
let db = setup_db_with_xpub(); let pool = setup_db_with_xpub().await;
let addresses = get_all_addresses_by_xpub(&db, "tpub_test").unwrap(); let addresses = get_all_addresses_by_xpub(&pool, "tpub_test").await.unwrap();
assert!(addresses.contains("addr1")); assert!(addresses.contains("addr1"));
assert!(addresses.contains("addr2")); assert!(addresses.contains("addr2"));
assert!(addresses.contains("addr3")); assert!(addresses.contains("addr3"));
assert_eq!(addresses.len(), 3); assert_eq!(addresses.len(), 3);
} }
#[test] #[tokio::test]
fn test_get_all_addresses_by_xpub_empty_for_missing() { async fn test_get_all_addresses_by_xpub_empty_for_missing() {
let db = setup_db_with_xpub(); let pool = setup_db_with_xpub().await;
let addresses = get_all_addresses_by_xpub(&db, "tpub_nonexistent").unwrap(); let addresses = get_all_addresses_by_xpub(&pool, "tpub_nonexistent")
.await
.unwrap();
assert!(addresses.is_empty()); assert!(addresses.is_empty());
} }
#[test] #[test]
fn test_network_unknown_returns_404() { fn test_network_unknown_returns_404() {
// This is a code-level check; the actual HTTP test requires actix-web setup.
// Verify that the NETWORKS constant includes the expected set.
let networks = ["bitcoin", "testnet", "testnet4", "signet", "regtest"]; let networks = ["bitcoin", "testnet", "testnet4", "signet", "regtest"];
for n in networks { for n in networks {
assert!(networks.contains(&n), "{} should be a valid network", n); assert!(networks.contains(&n), "{} should be a valid network", n);

View File

@@ -1,5 +1,4 @@
use sqlite::Connection; use sqlx::Row;
use std::collections::HashMap;
use std::sync::{Arc, Mutex}; use std::sync::{Arc, Mutex};
use std::thread; use std::thread;
@@ -8,39 +7,44 @@ fn test_mutex_poisoning_recovery() {
let data = Arc::new(Mutex::new(0)); let data = Arc::new(Mutex::new(0));
let c = data.clone(); let c = data.clone();
let handle = thread::spawn(move || { let handle = thread::spawn(move || {
let _guard = c.lock(); // Acquire lock let _guard = c.lock();
panic!("test panic"); // Panic while holding the lock panic!("test panic");
// _guard is dropped during panic unwinding, poisoning the mutex
}); });
let result = handle.join(); let result = handle.join();
assert!(result.is_err()); // Thread panicked assert!(result.is_err());
// Recovery: the same pattern used in bal-server.rs
let guard = match data.lock() { let guard = match data.lock() {
Ok(g) => g, Ok(g) => g,
Err(p) => { Err(p) => p.into_inner(),
p.into_inner() // Should not panic
}
}; };
assert_eq!(*guard, 0); assert_eq!(*guard, 0);
} }
#[test] #[tokio::test]
fn test_db_null_unwrap_or() { async fn test_db_null_unwrap_or() {
let db = Connection::open(":memory:").unwrap(); let pool = bal_server::db::open_database("sqlite", "sqlite::memory:")
let _ = db.execute( .await
.unwrap();
if let bal_server::db::DatabasePool::SQLite(p) = &pool {
sqlx::query(
"CREATE TABLE test_stats (report_date TEXT, chain TEXT, totals TEXT, waiting TEXT);", "CREATE TABLE test_stats (report_date TEXT, chain TEXT, totals TEXT, waiting TEXT);",
); )
let _ = .execute(p)
db.execute("INSERT INTO test_stats (report_date, chain) VALUES ('2024-01-01', 'testnet');"); .await
.unwrap();
let mut found_value = None; sqlx::query("INSERT INTO test_stats (report_date, chain) VALUES ('2024-01-01', 'testnet')")
let _ = db.iterate("SELECT * FROM test_stats;", |pairs| { .execute(p)
let row: HashMap<_, _> = pairs.iter().map(|(k, v)| (k.to_string(), *v)).collect(); .await
let totals = row["totals"].unwrap_or("0").to_string(); .unwrap();
found_value = Some(totals);
true
});
assert_eq!(found_value.unwrap(), "0"); let row = sqlx::query("SELECT * FROM test_stats")
.fetch_one(p)
.await
.unwrap();
let totals: Option<String> = row.try_get("totals").unwrap_or(None);
let totals_value = totals.unwrap_or_else(|| "0".to_string());
assert_eq!(totals_value, "0");
}
} }

View File

@@ -0,0 +1,335 @@
use bal_server::db::{
DatabasePool, calculate_and_upsert_stats, check_duplicate_txids, create_database,
get_all_addresses_by_xpub, get_next_address_index, get_pending_txs, get_stats, insert_xpub,
open_database, save_new_address, search_tx, update_tx_status,
};
use sqlx::Row;
fn pg_dsn() -> Option<String> {
std::env::var("BAL_TEST_PG_DSN").ok()
}
async fn setup_pg() -> Option<DatabasePool> {
let dsn = pg_dsn()?;
let pool = open_database("postgresql", &dsn).await.ok()?;
// Drop and recreate schema for clean test
if let DatabasePool::PostgreSQL(p) = &pool {
sqlx::query("DROP SCHEMA public CASCADE; CREATE SCHEMA public")
.execute(p)
.await
.ok()?;
}
create_database(&pool).await.ok()?;
Some(pool)
}
#[tokio::test]
async fn test_pg_open_database() {
let Some(dsn) = pg_dsn() else {
eprintln!("skipped: BAL_TEST_PG_DSN not set");
return;
};
let pool = open_database("postgresql", &dsn).await;
assert!(
pool.is_ok(),
"Opening PostgreSQL should succeed: {:?}",
pool.err()
);
}
#[tokio::test]
async fn test_pg_create_schema() {
let Some(pool) = setup_pg().await else {
eprintln!("skipped: PostgreSQL not available");
return;
};
// Second call should also succeed (idempotent)
let result = create_database(&pool).await;
assert!(
result.is_ok(),
"Creating PG schema twice should be idempotent: {:?}",
result.err()
);
}
#[tokio::test]
async fn test_pg_insert_xpub() {
let Some(pool) = setup_pg().await else {
eprintln!("skipped: PostgreSQL not available");
return;
};
insert_xpub(&pool, "testnet", "tpub_test123").await;
insert_xpub(&pool, "testnet", "tpub_test123").await; // duplicate should be ignored
if let DatabasePool::PostgreSQL(p) = &pool {
let row = sqlx::query("SELECT COUNT(*) as cnt FROM tbl_xpub WHERE xpub = 'tpub_test123'")
.fetch_one(p)
.await
.unwrap();
let count: i64 = row.try_get("cnt").unwrap();
assert_eq!(count, 1, "INSERT OR IGNORE should prevent duplicates");
}
}
#[tokio::test]
async fn test_pg_get_next_address_index() {
let Some(pool) = setup_pg().await else {
eprintln!("skipped: PostgreSQL not available");
return;
};
insert_xpub(&pool, "testnet", "tpub_addr_test").await;
let (id, idx) = get_next_address_index(&pool, "testnet", "tpub_addr_test").await;
assert!(id > 0, "Should return valid xpub id, got {}", id);
assert_eq!(idx, 0, "First index should be 0, got {}", idx);
let (id2, idx2) = get_next_address_index(&pool, "testnet", "tpub_addr_test").await;
assert_eq!(id, id2, "xpub id should be stable");
assert_eq!(idx2, 1, "Second index should be 1, got {}", idx2);
}
#[tokio::test]
async fn test_pg_save_and_get_address() {
let Some(pool) = setup_pg().await else {
eprintln!("skipped: PostgreSQL not available");
return;
};
insert_xpub(&pool, "testnet", "tpub_addr_test2").await;
let (xpub_id, _idx) = get_next_address_index(&pool, "testnet", "tpub_addr_test2").await;
save_new_address(&pool, xpub_id, "tb1qtestaddr", "m/0/0", "1.2.3.4").await;
let addrs = get_all_addresses_by_xpub(&pool, "tpub_addr_test2")
.await
.unwrap();
assert!(
addrs.contains("tb1qtestaddr"),
"Should find saved address: {:?}",
addrs
);
}
#[tokio::test]
async fn test_pg_check_duplicate_txids() {
let Some(pool) = setup_pg().await else {
eprintln!("skipped: PostgreSQL not available");
return;
};
// Insert a transaction first via raw SQL (to have a txid to check)
if let DatabasePool::PostgreSQL(p) = &pool {
sqlx::query(
"INSERT INTO tbl_tx (txid, wtxid, ntxid, tx, locktime, network, status)
VALUES ('txid_dup_test', 'wtx1', 'ntx1', 'rawtx', 100, 'testnet', 0)",
)
.execute(p)
.await
.unwrap();
}
let dups = check_duplicate_txids(
&pool,
&["txid_dup_test".to_string(), "txid_new".to_string()],
)
.await
.unwrap();
assert!(
dups.contains("txid_dup_test"),
"Should detect existing txid"
);
assert!(
!dups.contains("txid_new"),
"Should not report non-existing txid"
);
}
#[tokio::test]
async fn test_pg_search_tx() {
let Some(pool) = setup_pg().await else {
eprintln!("skipped: PostgreSQL not available");
return;
};
if let DatabasePool::PostgreSQL(p) = &pool {
sqlx::query(
"INSERT INTO tbl_tx (txid, wtxid, ntxid, tx, locktime, network, status, our_address, our_fees, reqid)
VALUES ('txid_search', 'wtx', 'ntx', 'rawhex', 500, 'testnet', 1, 'tb1ouraddr', '0.0001', 'req123')"
)
.execute(p)
.await
.unwrap();
}
let result = search_tx(&pool, "txid_search").await.unwrap();
assert!(result.is_some(), "Should find the transaction");
let row = result.unwrap();
assert_eq!(row.status, "1", "Status should be read as string '1'");
assert_eq!(row.tx, "rawhex");
assert_eq!(row.our_address, "tb1ouraddr");
assert_eq!(row.our_fees, "0.0001");
assert_eq!(row.reqid, "req123");
let not_found = search_tx(&pool, "txid_nonexistent").await.unwrap();
assert!(not_found.is_none(), "Should return None for missing txid");
}
#[tokio::test]
async fn test_pg_update_tx_status() {
let Some(pool) = setup_pg().await else {
eprintln!("skipped: PostgreSQL not available");
return;
};
if let DatabasePool::PostgreSQL(p) = &pool {
sqlx::query(
"INSERT INTO tbl_tx (txid, wtxid, ntxid, tx, locktime, network, status)
VALUES ('txid_status', 'wtx', 'ntx', 'raw', 100, 'testnet', 0)",
)
.execute(p)
.await
.unwrap();
}
update_tx_status(&pool, "txid_status", 1, None)
.await
.unwrap();
if let DatabasePool::PostgreSQL(p) = &pool {
let row = sqlx::query("SELECT status FROM tbl_tx WHERE txid = 'txid_status'")
.fetch_one(p)
.await
.unwrap();
let status: i64 = row.try_get("status").unwrap();
assert_eq!(status, 1, "Status should be updated to 1");
}
update_tx_status(&pool, "txid_status", 2, Some("test error"))
.await
.unwrap();
if let DatabasePool::PostgreSQL(p) = &pool {
let row = sqlx::query("SELECT status, push_err FROM tbl_tx WHERE txid = 'txid_status'")
.fetch_one(p)
.await
.unwrap();
let status: i64 = row.try_get("status").unwrap();
let push_err: Option<String> = row.try_get("push_err").unwrap();
assert_eq!(status, 2);
assert_eq!(push_err.as_deref(), Some("test error"));
}
}
#[tokio::test]
async fn test_pg_get_pending_txs() {
let Some(pool) = setup_pg().await else {
eprintln!("skipped: PostgreSQL not available");
return;
};
if let DatabasePool::PostgreSQL(p) = &pool {
// Insert pending tx (status=0, locktime < height)
sqlx::query(
"INSERT INTO tbl_tx (txid, wtxid, ntxid, tx, locktime, network, status)
VALUES ('pending1', 'w', 'n', 'rawtx1', 100, 'testnet', 0)",
)
.execute(p)
.await
.unwrap();
// Insert already pushed tx (status=1)
sqlx::query(
"INSERT INTO tbl_tx (txid, wtxid, ntxid, tx, locktime, network, status)
VALUES ('pushed1', 'w', 'n', 'rawtx2', 100, 'testnet', 1)",
)
.execute(p)
.await
.unwrap();
}
let txs = get_pending_txs(&pool, "testnet", 5000000, 200, 1000)
.await
.unwrap();
assert_eq!(txs.len(), 1, "Should only return pending txs");
assert_eq!(txs[0].txid, "pending1");
assert_eq!(txs[0].tx, "rawtx1");
}
#[tokio::test]
async fn test_pg_stats() {
let Some(pool) = setup_pg().await else {
eprintln!("skipped: PostgreSQL not available");
return;
};
// Insert test data
if let DatabasePool::PostgreSQL(p) = &pool {
sqlx::query(
"INSERT INTO tbl_tx (txid, wtxid, ntxid, tx, locktime, network, status, our_fees)
VALUES
('stx1', 'w', 'n', 'r', 100, 'testnet', 0, '0.0001'),
('stx2', 'w', 'n', 'r', 100, 'testnet', 1, '0.0002'),
('stx3', 'w', 'n', 'r', 100, 'testnet', 2, '0.0003')",
)
.execute(p)
.await
.unwrap();
}
calculate_and_upsert_stats(&pool, "testnet").await.unwrap();
let stats = get_stats(&pool, "testnet").await.unwrap();
assert_eq!(stats.len(), 1, "Should have one stats row");
assert_eq!(stats[0].totals, 3);
assert_eq!(stats[0].waiting, 1);
assert_eq!(stats[0].sent, 1);
assert_eq!(stats[0].failed, 1);
}
#[tokio::test]
async fn test_pg_sql_injection_via_push_err() {
let Some(pool) = setup_pg().await else {
eprintln!("skipped: PostgreSQL not available");
return;
};
if let DatabasePool::PostgreSQL(p) = &pool {
sqlx::query(
"CREATE TABLE test_inject (txid TEXT PRIMARY KEY, status INTEGER, push_err TEXT)",
)
.execute(p)
.await
.unwrap();
sqlx::query("INSERT INTO test_inject (txid, status, push_err) VALUES ($1, $2, $3)")
.bind("dummy")
.bind(0_i64)
.bind("")
.execute(p)
.await
.unwrap();
let malicious = "'; DROP TABLE test_inject; --";
sqlx::query("UPDATE test_inject SET status = 2, push_err = $1 WHERE txid = $2")
.bind(malicious)
.bind("dummy")
.execute(p)
.await
.unwrap();
let row = sqlx::query("SELECT status, push_err FROM test_inject WHERE txid = 'dummy'")
.fetch_one(p)
.await
.unwrap();
let status: i64 = row.try_get("status").unwrap();
let push_err: String = row.try_get("push_err").unwrap();
assert_eq!(status, 2);
assert_eq!(push_err, malicious);
// Table should still exist
let cnt = sqlx::query("SELECT COUNT(*) as cnt FROM test_inject")
.fetch_one(p)
.await
.unwrap();
let count: i64 = cnt.try_get("cnt").unwrap();
assert_eq!(count, 1);
}
}

View File

@@ -1,137 +1,136 @@
use sqlite::{Connection, Value}; use bal_server::db::{DatabasePool, open_database};
use sqlx::Row;
#[test] async fn setup_db() -> DatabasePool {
fn test_sql_injection_via_push_err_update() { open_database("sqlite", "sqlite::memory:").await.unwrap()
// Create an in-memory database and the required table }
let db = Connection::open(":memory:").unwrap();
let _ =
db.execute("CREATE TABLE tbl_tx (txid TEXT PRIMARY KEY, status INTEGER, push_err TEXT);");
// Insert a dummy transaction #[tokio::test]
let mut stmt = db async fn test_sql_injection_via_push_err_update() {
.prepare("INSERT INTO tbl_tx (txid, status, push_err) VALUES (?, ?, ?);") let pool = setup_db().await;
if let DatabasePool::SQLite(p) = &pool {
sqlx::query("CREATE TABLE tbl_tx (txid TEXT PRIMARY KEY, status INTEGER, push_err TEXT);")
.execute(p)
.await
.unwrap();
sqlx::query("INSERT INTO tbl_tx (txid, status, push_err) VALUES (?, ?, ?)")
.bind("dummy_txid")
.bind(0_i64)
.bind("")
.execute(p)
.await
.unwrap(); .unwrap();
stmt.bind((1, Value::String("dummy_txid".to_string())))
.unwrap();
stmt.bind((2, Value::Integer(0))).unwrap();
stmt.bind((3, Value::String("".to_string()))).unwrap();
let _ = stmt.next();
drop(stmt);
// Malicious error payload containing a single quote (SQL injection attempt)
let malicious_error = "'; DROP TABLE tbl_tx; --"; let malicious_error = "'; DROP TABLE tbl_tx; --";
let txid = "dummy_txid"; let txid = "dummy_txid";
// Execute the fixed query using parameter binding (safe) sqlx::query("UPDATE tbl_tx SET status = 2, push_err = ? WHERE txid = ?")
let sql = "UPDATE tbl_tx SET status = 2, push_err = ? WHERE txid = ?"; .bind(malicious_error)
let mut stmt = db.prepare(sql).unwrap(); .bind(txid)
stmt.bind((1, Value::String(malicious_error.to_string()))) .execute(p)
.await
.unwrap(); .unwrap();
stmt.bind((2, Value::String(txid.to_string()))).unwrap();
let _ = stmt.next();
// Verify the table still exists and the row was updated correctly let row = sqlx::query("SELECT status, push_err FROM tbl_tx WHERE txid = ?")
let mut check = db .bind("dummy_txid")
.prepare("SELECT status, push_err FROM tbl_tx WHERE txid = ?;") .fetch_one(p)
.await
.unwrap(); .unwrap();
check
.bind((1, Value::String("dummy_txid".to_string()))) let status: i64 = row.try_get("status").unwrap();
.unwrap(); let push_err: String = row.try_get("push_err").unwrap();
assert!(check.next().unwrap() == sqlite::State::Row);
let status: i64 = check.read("status").unwrap();
let push_err: String = check.read("push_err").unwrap();
assert_eq!(status, 2); assert_eq!(status, 2);
assert_eq!(push_err, malicious_error); assert_eq!(push_err, malicious_error);
// Ensure no second row was created (injection would have failed or produced extra rows) let row = sqlx::query("SELECT COUNT(*) as cnt FROM tbl_tx")
let mut count_stmt = db.prepare("SELECT COUNT(*) FROM tbl_tx;").unwrap(); .fetch_one(p)
assert!(count_stmt.next().unwrap() == sqlite::State::Row); .await
let count: i64 = count_stmt.read(0).unwrap(); .unwrap();
let count: i64 = row.try_get("cnt").unwrap();
assert_eq!(count, 1); assert_eq!(count, 1);
}
} }
#[test] #[tokio::test]
fn test_sql_injection_via_txid_update() { async fn test_sql_injection_via_txid_update() {
let db = Connection::open(":memory:").unwrap(); let pool = setup_db().await;
let _ = db.execute("CREATE TABLE tbl_tx (txid TEXT PRIMARY KEY, status INTEGER);"); if let DatabasePool::SQLite(p) = &pool {
sqlx::query("CREATE TABLE tbl_tx (txid TEXT PRIMARY KEY, status INTEGER);")
.execute(p)
.await
.unwrap();
// Insert multiple dummy transactions
for i in 0..3 { for i in 0..3 {
let mut stmt = db sqlx::query("INSERT INTO tbl_tx (txid, status) VALUES (?, ?)")
.prepare("INSERT INTO tbl_tx (txid, status) VALUES (?, ?);") .bind(format!("txid_{}", i))
.bind(0_i64)
.execute(p)
.await
.unwrap(); .unwrap();
stmt.bind((1, Value::String(format!("txid_{}", i))))
.unwrap();
stmt.bind((2, Value::Integer(0))).unwrap();
let _ = stmt.next();
} }
// Malicious txid payload
let malicious_txid = "' OR '1'='1"; let malicious_txid = "' OR '1'='1";
// The fixed query parameterizes the txid, so this should only update zero rows sqlx::query("UPDATE tbl_tx SET status = 1 WHERE txid = ?")
let sql = "UPDATE tbl_tx SET status = 1 WHERE txid = ?"; .bind(malicious_txid)
let mut stmt = db.prepare(sql).unwrap(); .execute(p)
stmt.bind((1, Value::String(malicious_txid.to_string()))) .await
.unwrap(); .unwrap();
let _ = stmt.next();
// Verify no rows were updated (status should still be 0 for all)
for i in 0..3 { for i in 0..3 {
let mut check = db let row = sqlx::query("SELECT status FROM tbl_tx WHERE txid = ?")
.prepare("SELECT status FROM tbl_tx WHERE txid = ?;") .bind(format!("txid_{}", i))
.fetch_one(p)
.await
.unwrap(); .unwrap();
check let status: i64 = row.try_get("status").unwrap();
.bind((1, Value::String(format!("txid_{}", i))))
.unwrap();
assert!(check.next().unwrap() == sqlite::State::Row);
let status: i64 = check.read("status").unwrap();
assert_eq!( assert_eq!(
status, 0, status, 0,
"Row txid_{} should not be updated by malicious txid", "Row txid_{} should not be updated by malicious txid",
i i
); );
} }
}
} }
#[test] #[tokio::test]
fn test_sql_injection_via_txid_with_comment() { async fn test_sql_injection_via_txid_with_comment() {
let db = Connection::open(":memory:").unwrap(); let pool = setup_db().await;
let _ = db.execute("CREATE TABLE tbl_tx (txid TEXT PRIMARY KEY, status INTEGER);"); if let DatabasePool::SQLite(p) = &pool {
sqlx::query("CREATE TABLE tbl_tx (txid TEXT PRIMARY KEY, status INTEGER);")
let mut stmt = db .execute(p)
.prepare("INSERT INTO tbl_tx (txid, status) VALUES (?, ?);") .await
.unwrap();
sqlx::query("INSERT INTO tbl_tx (txid, status) VALUES (?, ?)")
.bind("safe_txid")
.bind(0_i64)
.execute(p)
.await
.unwrap(); .unwrap();
stmt.bind((1, Value::String("safe_txid".to_string())))
.unwrap();
stmt.bind((2, Value::Integer(0))).unwrap();
let _ = stmt.next();
// Another common injection pattern
let malicious_txid = "safe_txid'; UPDATE tbl_tx SET status = 99; --"; let malicious_txid = "safe_txid'; UPDATE tbl_tx SET status = 99; --";
let sql = "UPDATE tbl_tx SET status = 1 WHERE txid = ?"; sqlx::query("UPDATE tbl_tx SET status = 1 WHERE txid = ?")
let mut stmt = db.prepare(sql).unwrap(); .bind(malicious_txid)
stmt.bind((1, Value::String(malicious_txid.to_string()))) .execute(p)
.await
.unwrap(); .unwrap();
let _ = stmt.next();
// Verify the original row was NOT updated (because it was looking for the full malicious string) let row = sqlx::query("SELECT status FROM tbl_tx WHERE txid = ?")
// and no rows have status 99 (the injected update did not execute) .bind("safe_txid")
let mut check = db .fetch_one(p)
.prepare("SELECT status FROM tbl_tx WHERE txid = ?;") .await
.unwrap(); .unwrap();
check let status: i64 = row.try_get("status").unwrap();
.bind((1, Value::String("safe_txid".to_string())))
.unwrap();
assert!(check.next().unwrap() == sqlite::State::Row);
let status: i64 = check.read("status").unwrap();
assert_eq!(status, 0, "Original row should not be updated"); assert_eq!(status, 0, "Original row should not be updated");
let mut count_stmt = db let row = sqlx::query("SELECT COUNT(*) as cnt FROM tbl_tx WHERE status = 99")
.prepare("SELECT COUNT(*) FROM tbl_tx WHERE status = 99;") .fetch_one(p)
.await
.unwrap(); .unwrap();
assert!(count_stmt.next().unwrap() == sqlite::State::Row); let count: i64 = row.try_get("cnt").unwrap();
let count: i64 = count_stmt.read(0).unwrap();
assert_eq!(count, 0, "No rows should have status 99"); assert_eq!(count, 0, "No rows should have status 99");
}
} }

287
tests/test_endpoints.sh Executable file
View File

@@ -0,0 +1,287 @@
#!/bin/bash
# tests/test_endpoints.sh — Integration tests for bal-server endpoints
# Usage: ./tests/test_endpoints.sh <base_url>
# Example: ./tests/test_endpoints.sh http://127.0.0.1:9133
# Returns 0 if all tests pass, 1 otherwise
set -uo pipefail
BASE_URL="${1:?Usage: $0 <base_url>}"
# Delay between requests to avoid rate limiter (actix-governor)
DELAY=1.0
PASS=0
FAIL=0
TOTAL=0
# Colors
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
NC='\033[0m'
# Known network for testing (must be enabled in server config)
TEST_NETWORK="regtest"
# A valid 64-char hex txid that does NOT exist in the database
NONEXISTENT_TXID="551dc4841830e457b0932b81eb458a00f87e5342b70333bb92df7475d6ca90f4"
# A valid raw transaction hex (minimal valid tx for testing)
VALID_TX_HEX="020000000100000000000000000000000000000000000000000000000000000000000000000000000000ffffffff0100f2052a0100000043410496b538e853519c726a2c91e61ec112f826d36b0c8080a01e2894d24b051f05e0f03ed7790b09fd327518756ff2a55ecee44b5e08d76f994a7c5f3ffcac88bac"
# ─────────────────────────────────────────────
# Helpers
# ─────────────────────────────────────────────
log_pass() {
PASS=$((PASS + 1))
TOTAL=$((TOTAL + 1))
echo -e " ${GREEN}PASS${NC} $1"
}
log_fail() {
FAIL=$((FAIL + 1))
TOTAL=$((TOTAL + 1))
echo -e " ${RED}FAIL${NC} $1"
if [ -n "${2:-}" ]; then
echo -e " Expected: ${YELLOW}$2${NC}"
echo -e " Got: ${YELLOW}$3${NC}"
fi
}
# Perform a GET request with rate-limit delay, split into BODY and STATUS
curl_get() {
sleep "$DELAY"
local resp
resp=$(curl -s -w "\n%{http_code}" "$@")
BODY=$(echo "$resp" | sed '$d')
STATUS=$(echo "$resp" | tail -1)
}
# Perform a POST request with rate-limit delay, split into BODY and STATUS
curl_post() {
sleep "$DELAY"
local resp
resp=$(curl -s -w "\n%{http_code}" "$@")
BODY=$(echo "$resp" | sed '$d')
STATUS=$(echo "$resp" | tail -1)
}
assert_status() {
local expected="$1" actual="$2" name="$3"
if [ "$actual" = "$expected" ]; then
log_pass "$name (HTTP $actual)"
else
log_fail "$name" "HTTP $expected" "HTTP $actual"
fi
}
assert_body_contains() {
local pattern="$1" body="$2" name="$3"
if echo "$body" | grep -q "$pattern"; then
log_pass "$name (contains '$pattern')"
else
log_fail "$name" "body contains '$pattern'" "body: '$(echo "$body" | head -c 80)'"
fi
}
assert_body_not_empty() {
local body="$1" name="$2"
if [ -n "$body" ]; then
log_pass "$name (non-empty response)"
else
log_fail "$name" "non-empty body" "empty body"
fi
}
assert_json_valid() {
local body="$1" name="$2"
if echo "$body" | jq . >/dev/null 2>&1; then
log_pass "$name (valid JSON)"
else
log_fail "$name" "valid JSON" "invalid JSON: $(echo "$body" | head -c 100)"
fi
}
assert_json_has_key() {
local key="$1" body="$2" name="$3"
if echo "$body" | jq -e ".$key" >/dev/null 2>&1; then
log_pass "$name (has key '$key')"
else
log_fail "$name" "JSON has key '$key'" "key not found"
fi
}
# ─────────────────────────────────────────────
# 1. GET /
# ─────────────────────────────────────────────
echo ""
echo -e "${YELLOW}[1] GET /${NC}"
curl_get "$BASE_URL/"
assert_status "200" "$STATUS" "GET / returns 200"
assert_body_not_empty "$BODY" "GET / returns non-empty body"
# ─────────────────────────────────────────────
# 2. GET /.pub_key.pem
# ─────────────────────────────────────────────
echo ""
echo -e "${YELLOW}[2] GET /.pub_key.pem${NC}"
curl_get "$BASE_URL/.pub_key.pem"
assert_status "200" "$STATUS" "GET /.pub_key.pem returns 200"
assert_body_contains "BEGIN PUBLIC KEY" "$BODY" "GET /.pub_key.pem contains PEM header"
# ─────────────────────────────────────────────
# 3. GET /version
# ─────────────────────────────────────────────
echo ""
echo -e "${YELLOW}[3] GET /version${NC}"
curl_get "$BASE_URL/version"
assert_status "200" "$STATUS" "GET /version returns 200"
assert_body_not_empty "$BODY" "GET /version returns non-empty body"
# ─────────────────────────────────────────────
# 4. GET /{network}/info
# ─────────────────────────────────────────────
echo ""
echo -e "${YELLOW}[4] GET /$TEST_NETWORK/info${NC}"
curl_get "$BASE_URL/$TEST_NETWORK/info"
assert_status "200" "$STATUS" "GET /$TEST_NETWORK/info returns 200"
assert_json_valid "$BODY" "GET /$TEST_NETWORK/info returns valid JSON"
assert_json_has_key "chain" "$BODY" "GET /$TEST_NETWORK/info has 'chain' key"
assert_json_has_key "address" "$BODY" "GET /$TEST_NETWORK/info has 'address' key"
assert_json_has_key "base_fee" "$BODY" "GET /$TEST_NETWORK/info has 'base_fee' key"
assert_json_has_key "info" "$BODY" "GET /$TEST_NETWORK/info has 'info' key"
assert_json_has_key "version" "$BODY" "GET /$TEST_NETWORK/info has 'version' key"
echo ""
echo -e "${YELLOW}[4b] GET /invalidnet/info${NC}"
curl_get "$BASE_URL/invalidnet/info"
assert_status "404" "$STATUS" "GET /invalidnet/info returns 404"
assert_body_contains "Unknown network" "$BODY" "GET /invalidnet/info body says 'Unknown network'"
# ─────────────────────────────────────────────
# 5. GET /{network}/stats
# ─────────────────────────────────────────────
echo ""
echo -e "${YELLOW}[5] GET /$TEST_NETWORK/stats${NC}"
curl_get "$BASE_URL/$TEST_NETWORK/stats"
assert_status "200" "$STATUS" "GET /$TEST_NETWORK/stats returns 200"
assert_json_valid "$BODY" "GET /$TEST_NETWORK/stats returns valid JSON"
echo ""
echo -e "${YELLOW}[5b] GET /invalidnet/stats${NC}"
curl_get "$BASE_URL/invalidnet/stats"
assert_status "404" "$STATUS" "GET /invalidnet/stats returns 404"
assert_body_contains "Unknown network" "$BODY" "GET /invalidnet/stats body says 'Unknown network'"
# ─────────────────────────────────────────────
# 6. POST /searchtx
# ─────────────────────────────────────────────
echo ""
echo -e "${YELLOW}[6] POST /searchtx — empty body${NC}"
curl_post -X POST -d "" "$BASE_URL/searchtx"
assert_status "400" "$STATUS" "POST /searchtx empty body returns 400"
assert_body_contains "Invalid txid" "$BODY" "POST /searchtx empty body says 'Invalid txid'"
echo ""
echo -e "${YELLOW}[6b] POST /searchtx — short txid${NC}"
curl_post -X POST -d "abc123" "$BASE_URL/searchtx"
assert_status "400" "$STATUS" "POST /searchtx short txid returns 400"
assert_body_contains "Invalid txid" "$BODY" "POST /searchtx short txid says 'Invalid txid'"
echo ""
echo -e "${YELLOW}[6c] POST /searchtx — non-hex txid${NC}"
curl_post -X POST -d "zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz" "$BASE_URL/searchtx"
assert_status "400" "$STATUS" "POST /searchtx non-hex txid returns 400"
assert_body_contains "Invalid txid" "$BODY" "POST /searchtx non-hex txid says 'Invalid txid'"
echo ""
echo -e "${YELLOW}[6d] POST /searchtx — nonexistent valid hex txid${NC}"
curl_post -X POST -d "$NONEXISTENT_TXID" "$BASE_URL/searchtx"
# When txid is valid hex but not in DB: either 200 with empty JSON or 404
if [ "$STATUS" = "200" ] || [ "$STATUS" = "404" ]; then
log_pass "POST /searchtx nonexistent txid returns HTTP $STATUS (expected)"
else
log_fail "POST /searchtx nonexistent txid" "HTTP 200 or 404" "HTTP $STATUS"
fi
# Verify Content-Type is application/json for successful searchtx responses
if [ "$STATUS" = "200" ]; then
CONTENT_TYPE=$(curl -s -D - --max-time 3 -X POST -d "$NONEXISTENT_TXID" "$BASE_URL/searchtx" 2>/dev/null | grep -i "^content-type:" | tr -d '\r')
if echo "$CONTENT_TYPE" | grep -q "application/json"; then
log_pass "POST /searchtx Content-Type is application/json"
else
log_fail "POST /searchtx Content-Type" "application/json" "$CONTENT_TYPE"
fi
fi
echo ""
echo -e "${YELLOW}[6e] POST /searchtx — binary non-UTF8 body${NC}"
curl_post -X POST --data-binary $'\xff\xfe\xfd' "$BASE_URL/searchtx"
assert_status "400" "$STATUS" "POST /searchtx binary body returns 400"
assert_body_contains "Invalid UTF-8 body" "$BODY" "POST /searchtx binary body says 'Invalid UTF-8 body'"
# ─────────────────────────────────────────────
# 7. POST /{network}/pushtxs
# ─────────────────────────────────────────────
echo ""
echo -e "${YELLOW}[7] POST /invalidnet/pushtxs${NC}"
curl_post -X POST -d "" "$BASE_URL/invalidnet/pushtxs"
assert_status "404" "$STATUS" "POST /invalidnet/pushtxs returns 404"
assert_body_contains "Unknown network" "$BODY" "POST /invalidnet/pushtxs body says 'Unknown network'"
echo ""
echo -e "${YELLOW}[7b] POST /$TEST_NETWORK/pushtxs — empty body${NC}"
curl_post -X POST -d "" "$BASE_URL/$TEST_NETWORK/pushtxs"
assert_status "200" "$STATUS" "POST /$TEST_NETWORK/pushtxs empty body returns 200"
echo ""
echo -e "${YELLOW}[7c] POST /$TEST_NETWORK/pushtxs — valid hex tx${NC}"
curl_post -X POST -d "$VALID_TX_HEX" "$BASE_URL/$TEST_NETWORK/pushtxs"
# Server returns 200 with "thx" or "already present" or "Bad data received"
if [ "$STATUS" = "200" ]; then
if echo "$BODY" | grep -qE "^(thx|already present|Bad data received)$"; then
log_pass "POST /$TEST_NETWORK/pushtxs valid hex returns HTTP 200 with '$BODY'"
else
log_pass "POST /$TEST_NETWORK/pushtxs valid hex returns HTTP 200 (body: $(echo "$BODY" | head -c 50))"
fi
else
log_fail "POST /$TEST_NETWORK/pushtxs valid hex" "HTTP 200" "HTTP $STATUS"
fi
echo ""
echo -e "${YELLOW}[7d] POST /$TEST_NETWORK/pushtxs — non-hex garbage${NC}"
curl_post -X POST -d "not-a-transaction" "$BASE_URL/$TEST_NETWORK/pushtxs"
assert_status "200" "$STATUS" "POST /$TEST_NETWORK/pushtxs garbage returns 200"
echo ""
echo -e "${YELLOW}[7e] POST /$TEST_NETWORK/pushtxs — binary non-UTF8 body${NC}"
curl_post -X POST --data-binary $'\xff\xfe\xfd' "$BASE_URL/$TEST_NETWORK/pushtxs"
# Invalid UTF-8 should be rejected
if [ "$STATUS" = "400" ]; then
assert_body_contains "Invalid UTF-8 body" "$BODY" "POST /$TEST_NETWORK/pushtxs binary body says 'Invalid UTF-8 body'"
else
# Server may accept binary as latin-1 and skip invalid lines gracefully
log_pass "POST /$TEST_NETWORK/pushtxs binary body returns HTTP $STATUS (skipped gracefully)"
fi
# ─────────────────────────────────────────────
# 8. GET on unknown routes
# ─────────────────────────────────────────────
echo ""
echo -e "${YELLOW}[8] GET /nonexistent${NC}"
curl_get "$BASE_URL/nonexistent"
assert_status "404" "$STATUS" "GET /nonexistent returns 404"
# ─────────────────────────────────────────────
# Summary
# ─────────────────────────────────────────────
echo ""
echo "========================================="
echo -e "Results: ${GREEN}$PASS passed${NC}, ${RED}$FAIL failed${NC} ($TOTAL total)"
echo "========================================="
if [ "$FAIL" -gt 0 ]; then
exit 1
fi
exit 0