bitcoinafterlife/bal-server
1
0
Fork 1
Code Issues Pull Requests Actions Packages Projects Releases 7 Wiki Activity

fix: resolve PostgreSQL type mismatches and add willexecutor debug logging

Browse Source 
- Fix get_next_address_index: use try_get::<i32> for PG SERIAL/INTEGER columns
- Fix search_tx: use try_get::<i32> for PG status column
- Fix execute_insert: parse locktime (String→i64) and in_vout (String→i32)
  before binding to PG INTEGER columns
- Fix execute_insert: bind tbl_out vout as i32, amount as String for PG
- Fix save_new_address: cast xpub i64 to i32 for PG INTEGER column
- Fix get_pending_txs: cast i64 bind params to i32, use try_get::<i32> for reads
- Fix get_stats: use try_get::<i32> for all numeric PG INTEGER columns
- Add trace logging in parse_request_transactions for xpub address matching
- Add trace logging in get_all_addresses_by_xpub for query debugging
This commit is contained in:
svatantrya
2026-08-18 03:14:03 -04:00
parent 8dc344cbd1
commit 8f764f06b2
65 changed files with 4879 additions and 1188 deletions
Download Patch File Download Diff File Expand all files Collapse all files

0
.gitsecret/paths/mapping.cfg Normal file
View File

715
Cargo.lock generated
View File

File diff suppressed because it is too large Load Diff

2
Cargo.toml
View File

@@ -20,7 +20,7 @@ hex = { version = "0.4.3" }
log = { version = "0.4.21" }
serde = { version = "1.0.152", features = ["derive"] }
serde_json = { version = "1.0.116" }
sqlite = { version = "0.34.0" }
sqlx = { version = "0.8", features = ["runtime-tokio", "sqlite", "postgres", "chrono"] }
regex = { version = "1.10.4" }
tokio = { version = "1", features = ["rt", "net","macros","rt-multi-thread"] }
url = { version = "2" }

BIN
bal-server-0.2.2-1_x86_64_linux-gnu.tar.gz Normal file
View File

Binary file not shown.

85
bal-server-0.2.2-1_x86_64_linux-gnu/README.md Normal file
View File

@@ -0,0 +1,85 @@
# bal-server
## Installation
```bash
$ git clone ....
$ cd bal-server
$ openssl genpkey -algorithm ED25519 -out private_key.pem
$ openssl pkey -in private_key.pem -pubout -out public_key.pem
$ cargo build --release
$ sudo cp target/release/bal-server /usr/local/bin
$ bal-server
```
## Configuration
The `bal-server` application can be configured using environment variables. The following variables are available:
| Variable | Description | Default |
| --- | --- | --- |
| `BAL_SERVER_CONFIG_FILE` | Path to the configuration file. If the file does not exist, a new one will be created. | `$HOME/.config/bal-server/default-config.toml` |
| `BAL_SERVER_DB_FILE` | Path to the SQLite3 database file. If the file does not exist, a new one will be created. | `bal.db` |
| `BAL_SERVER_BIND_ADDRESS` | Public address for listening to requests. | `127.0.0.1` |
| `BAL_SERVER_BIND_PORT` | Default port for listening to requests. | `9137` |
| `BAL_SERVER_PUB_KEY_PATH` | WillExecutor Ed25519 public key | `public_key.pem` |
| `BAL_SERVER_REGTEST_ADDRESS` | Bitcoin address for the regtest environment. | - |
| `BAL_SERVER_REGTEST_FIXED_FEE` | Fixed fee for the regtest environment. | 50000 |
| `BAL_SERVER_SIGNET_ADDRESS` | Bitcoin address for the signet environment. | - |
| `BAL_SERVER_SIGNET_FIXED_FEE` | Fixed fee for the signet environment. | 50000 |
| `BAL_SERVER_TESTNET_ADDRESS` | Bitcoin address for the testnet environment. | - |
| `BAL_SERVER_TESTNET_FIXED_FEE` | Fixed fee for the testnet environment. | 50000 |
| `BAL_SERVER_BITCOIN_ADDRESS` | Bitcoin address for the mainnet environment. | - |
| `BAL_SERVER_BITCOIN_FIXED_FEE` | Fixed fee for the mainnet environment. | 50000 |
# bal-pusher
`bal-pusher` is a tool that retrieves Bitcoin transactions from a database and pushes them to the Bitcoin network when their **locktime** exceeds the **median time past** (MTP). It listens for Bitcoin block updates via ZMQ.
## Installation
To use `bal-pusher`, you need to compile and install Bitcoin with ZMQ (ZeroMQ) support enabled. Then, configure your Bitcoin node and `bal-pusher` to push the transactions.
### Prerequisites
1. **Bitcoin with ZMQ Support**:
Ensure that Bitcoin is compiled with ZMQ support. Add the following line to your `bitcoin.conf` file:
```
zmqpubhashblock=tcp://127.0.0.1:28332
```
2. **Install Rust and Cargo**:
If you haven't already installed Rust and Cargo, you can follow the official instructions to do so: [Rust Installation](https://www.rust-lang.org/tools/install).
## Configuration
`bal-pusher` can be configured using environment variables. If no configuration file is provided, a default configuration file will be created.
### Available Configuration Variables
| Variable | Description | Default |
|---------------------------------------|------------------------------------------|----------------------------------------------|
| `BAL_PUSHER_CONFIG_FILE` | Path to the configuration file. If the file does not exist, it will be created. | `$HOME/.config/bal-pusher/default-config.toml` |
| `BAL_PUSHER_DB_FILE` | Path to the SQLite3 database file. If the file does not exist, it will be created. | `bal.db` |
| `BAL_PUSHER_ZMQ_LISTENER` | ZMQ listener for Bitcoin updates. | `tcp://127.0.0.1:28332` |
| `BAL_PUSHER_BITCOIN_HOST` | Bitcoin server host for RPC connections. | `http://127.0.0.1` |
| `BAL_PUSHER_BITCOIN_PORT` | Bitcoin RPC server port. | `8332` |
| `BAL_PUSHER_BITCOIN_COOKIE_FILE` | Path to Bitcoin RPC cookie file. | `$HOME/.bitcoin/.cookie` |
| `BAL_PUSHER_BITCOIN_RPC_USER` | Bitcoin RPC username. | - |
| `BAL_PUSHER_BITCOIN_RPC_PASSWORD` | Bitcoin RPC password. | - |
| `BAL_PUSHER_SEND_STATS` | Contact welist to provide times | false |
| `WELIST_SERVER_URL` | welist server url to provide times | https://welist.bitcoin-afer.life |
| `BAL_SERVER_URL` | WillExecutor server url | - |
| `SSL_KEY_PATH` | Ed25519 private key pem file | `private_key.pem` |
## Running `bal-pusher`
Once the application is installed and configured, you can start `bal-pusher` by running the following command:
```bash
$ bal-pusher
```
This will start the service, which will listen for Bitcoin blocks via ZMQ and push transactions from the database when their locktime exceeds the median time past.

BIN
bal-server-0.2.2-1_x86_64_linux-gnu/bal-pusher Executable file
View File

Binary file not shown.

BIN
bal-server-0.2.2-1_x86_64_linux-gnu/bal-server Executable file
View File

Binary file not shown.

25
bal-server-0.2.2-1_x86_64_linux-gnu/bal-server.sh Normal file
View File

@@ -0,0 +1,25 @@
WORKING_DIR=$(pwd)
if [ ! -f "$WORKING_DIR/public_key.pem" ]; then
echo "creating keypairs"
openssl genpkey -algorithm ED25519 -out private_key.pem
openssl pkey -in private_key.pem -pubout -out public_key.pem
fi
export RUST_LOG="trace"
export BAL_SERVER_DB_FILE="$WORKING_DIR/bal.db"
export BAL_SERVER_INFO="BAL devel willexecutor server"
export BAL_SERVER_BIND_ADDRESS="127.0.0.1"
export BAL_SERVER_BIND_PORT=9133
export BAL_SERVER_PUB_KEY_PATH="$WORKING_DIR/public_key.pem"
export BAL_SERVER_EXPOSE_STATS=true;
#export BAL_SERVER_BITCOIN_ADDRESS="your bitcoin address or xpub to recive payments here"
#export BAL_SERVER_BITCOIN_FIXED_FEE=50000
export BAL_SERVER_REGTEST_ADDRESS="vpub5UhLrYG1qQjnJhvJgBdqgpznyH11mxW9hwBYxf3KhfdjiupCFPUVDvgwpeZ9Wj5YUJXjKjXjy7DSbJNBW1sXbKwARiaphm1UjHYy3mKvTG4"
export BAL_SERVER_REGTEST_FIXED_FEE=1000
#export BAL_SERVER_TESTNET_ADDRESS=
#export BAL_SERVER_TESTNET_FEE=100000
#export BAL_SERVER_SIGNET_ADDRESS=
#export BAL_SERVER_SIGNET_FEE=100000
./bal-server

3
bal-server-0.2.2-1_x86_64_linux-gnu/public_key.pem Normal file
View File

@@ -0,0 +1,3 @@
-----BEGIN PUBLIC KEY-----
MCowBQYDK2VwAyEAklkvdmBJSncODyKwkGuEqxqbd3y+gb3X73EPx+0QAZY=
-----END PUBLIC KEY-----

40
bal-server.sh Normal file → Executable file
View File

@@ -1,6 +1,12 @@
#!/usr/bin/env bash
set -e
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
. "$SCRIPT_DIR/lib.sh"
WORKING_DIR=$(pwd)
if [ ! -f "$WORKING_DIR/public_key.pem" ]; then
echo "creating keypairs"
echo_i "creating keypairs"
openssl genpkey -algorithm ED25519 -out private_key.pem
openssl pkey -in private_key.pem -pubout -out public_key.pem
fi
@@ -11,7 +17,7 @@ export BAL_SERVER_INFO="BAL devel willexecutor server"
export BAL_SERVER_BIND_ADDRESS="127.0.0.1"
export BAL_SERVER_BIND_PORT=9133
export BAL_SERVER_PUB_KEY_PATH="$WORKING_DIR/public_key.pem"
export BAL_SERVER_EXPOSE_STATS=true;
export BAL_SERVER_EXPOSE_STATS=true
#export BAL_SERVER_BITCOIN_ADDRESS="your bitcoin address or xpub to recive payments here"
#export BAL_SERVER_BITCOIN_FIXED_FEE=50000
@@ -22,4 +28,34 @@ export BAL_SERVER_REGTEST_FIXED_FEE=1000
#export BAL_SERVER_TESTNET_FEE=100000
#export BAL_SERVER_SIGNET_ADDRESS=
#export BAL_SERVER_SIGNET_FEE=100000
export BAL_SERVER_DB_BACKEND="postgresql"
if [ "$BAL_SERVER_DB_BACKEND" = "postgresql" ]; then
PG_CONTAINER="bal-pg"
PG_DSN="${BAL_SERVER_PG_DSN:-postgres://bal:balpass@localhost:5432/baldb}"
docker rm -f "$PG_CONTAINER" 2>/dev/null || true
echo_i "Starting PostgreSQL container..."
docker run -d --name "$PG_CONTAINER" \
-e POSTGRES_USER=bal \
-e POSTGRES_PASSWORD=balpass \
-e POSTGRES_DB=baldb \
-p 5432:5432 postgres:16-alpine
echo_i "Waiting for PostgreSQL..."
until docker exec "$PG_CONTAINER" pg_isready -U bal >/dev/null 2>&1; do
sleep 1
done
echo_i "PostgreSQL ready at $PG_DSN"
export BAL_SERVER_PG_DSN="$PG_DSN"
cleanup() {
echo_i "Stopping PostgreSQL container..."
docker stop "$PG_CONTAINER" >/dev/null 2>&1
docker rm "$PG_CONTAINER" >/dev/null 2>&1
}
trap cleanup EXIT
fi
cargo run --bin=bal-server

55
bitcoind.service Normal file
View File

@@ -0,0 +1,55 @@
# /etc/systemd/system/bitcoind.service
[Unit]
Description=Bitcoin daemon
After=network.target
[Service]
# Service execution
###################
ExecStart=/usr/local/bin/bitcoind -daemon \
-pid=/run/bitcoind/bitcoind.pid \
-conf=/home/bitcoin/.bitcoin/bitcoin.conf \
-datadir=/home/bitcoin/.bitcoin \
-startupnotify="chmod g+r /home/bitcoin/.bitcoin/.cookie"
# Process management
####################
Type=forking
PIDFile=/run/bitcoind/bitcoind.pid
Restart=on-failure
TimeoutSec=300
RestartSec=30
# Directory creation and permissions
####################################
User=bitcoin
UMask=0027
# /run/bitcoind
RuntimeDirectory=bitcoind
RuntimeDirectoryMode=0710
# Hardening measures
####################
# Provide a private /tmp and /var/tmp.
PrivateTmp=true
# Mount /usr, /boot/ and /etc read-only for the process.
ProtectSystem=full
# Disallow the process and all of its children to gain
# new privileges through execve().
NoNewPrivileges=true
# Use a new /dev namespace only populated with API pseudo devices
# such as /dev/null, /dev/zero and /dev/random.
PrivateDevices=true
# Deny the creation of writable and executable memory mappings.
MemoryDenyWriteExecute=true
[Install]
WantedBy=multi-user.target

BIN
contrib.tar.gz Normal file
View File

Binary file not shown.

0
contrib/.config_init Normal file
View File

325
contrib/deploy.sh Executable file
View File

@@ -0,0 +1,325 @@
#!/bin/bash
set -euo pipefail
# deploy.sh — Deploy the latest bal-server release on a remote server.
#
# Downloads, verifies (SHA-256 + GPG), and installs the release entirely
# on the remote server via SSH. No file transfer (scp/rsync) needed.
#
# Usage:
# ./contrib/deploy.sh # deploy latest
# ./contrib/deploy.sh --dry-run # show plan without executing
# ./contrib/deploy.sh v0.3.1 # deploy a specific tag
# DEPLOY_DRY_RUN=1 ./contrib/deploy.sh # same as --dry-run
#
# Prerequisites:
# - ssh key access to debian@bitcoin-after.life:47081
# - The server must have: curl, jq, sha256sum, gpg, systemctl
#
# Services managed:
# bal-server, bal-pusher, tbal-pusher, t4bal-pusher
REMOTE_USER="debian"
REMOTE_HOST="bitcoin-after.life"
REMOTE_PORT="47081"
SSH_OPTS="-p ${REMOTE_PORT} -o ConnectTimeout=15 -o BatchMode=yes"
GITEA_API="https://bitcoin-after.life/gitea/api/v1/repos/bitcoinafterlife/bal-server"
GPG_SIGNER="svatantrya@bitcoin-after.life"
GPG_KEY_ID="A847D004DB91610711CA6A0DFE756706E833E0D1"
SERVICES=("bal-server" "bal-pusher" "tbal-pusher" "t4bal-pusher")
INSTALL_DIR="/usr/local/bin"
# ── Parse arguments ─────────────────────────────────────────────────
DRY_RUN=0
DEPLOY_TAG=""
for arg in "$@"; do
case "$arg" in
--dry-run) DRY_RUN=1 ;;
--help|-h)
echo "Usage: $0 [--dry-run] [TAG]"
echo " TAG Deploy a specific release tag (e.g. v0.3.1). Default: latest."
exit 0
;;
*) DEPLOY_TAG="$arg" ;;
esac
done
[[ "${DEPLOY_DRY_RUN:-}" == "1" ]] && DRY_RUN=1
# ── Helpers ──────────────────────────────────────────────────────────
info() { echo -e "\033[1m==> $1\033[0m"; }
ok() { echo -e "\033[32;1m ✔ $1\033[0m"; }
warn() { echo -e "\033[33;1m ⚠ $1\033[0m"; }
err() { echo -e "\033[31;1m ✖ $1\033[0m"; }
die() { err "$1"; exit "${2:-1}"; }
remote_exec() {
if [[ $DRY_RUN -eq 1 ]]; then
info "[dry-run] remote: $1"
return 0
fi
ssh $SSH_OPTS "${REMOTE_USER}@${REMOTE_HOST}" "$@"
}
# ── Step 0: Verify SSH connectivity ────────────────────────────────
info "Verifying SSH connection to ${REMOTE_HOST} ..."
if ! remote_exec "echo ok" >/dev/null 2>&1; then
die "Cannot connect via SSH. Check your key and network."
fi
ok "SSH connection OK"
# ── Step 1: Resolve release to deploy ──────────────────────────────
if [[ -n "$DEPLOY_TAG" ]]; then
TAG="$DEPLOY_TAG"
info "Deploying explicit tag: $TAG"
RELEASE_JSON=$(curl -sfL "${GITEA_API}/releases/tags/${TAG}") \
|| die "Release $TAG not found at $GITEA_API/releases/tags/$TAG"
else
info "Fetching latest release metadata ..."
RELEASE_JSON=$(curl -sfL "${GITEA_API}/releases/latest") \
|| die "Failed to fetch latest release from $GITEA_API"
TAG=$(echo "$RELEASE_JSON" | jq -r '.tag_name // empty')
fi
RELEASE_NAME=$(echo "$RELEASE_JSON" | jq -r '.name // empty')
if [[ -z "$TAG" ]]; then
die "Could not determine release tag"
fi
info "Release: $RELEASE_NAME (tag: $TAG)"
TARBALL_URL=$(echo "$RELEASE_JSON" | jq -r \
'.assets[] | select(.name | test("\\.tar\\.gz$")) | .browser_download_url' | head -1)
[[ -z "$TARBALL_URL" ]] && die "No .tar.gz asset found in release $TAG"
ASSET_NAME=$(basename "$TARBALL_URL")
info "Asset: $ASSET_NAME"
# Collect sidecar URLs
SHA256_URL="${TARBALL_URL}.sha256"
SIG_URL="${TARBALL_URL}.sig"
ASC_URL="${TARBALL_URL}.asc"
# ── Step 2: Check current version on server ─────────────────────────
info "Checking current version on server ..."
CURRENT=$(remote_exec "$INSTALL_DIR/bal-server --version" 2>/dev/null \
|| remote_exec "strings $INSTALL_DIR/bal-server 2>/dev/null | head -1" 2>/dev/null \
|| echo "unknown")
info "Current version on server: $CURRENT"
# ── Step 2b: Export GPG key for remote import ────────────────────────
info "Exporting GPG public key for $GPG_SIGNER ..."
GPG_PUBKEY_B64=""
if gpg --list-keys "$GPG_SIGNER" &>/dev/null; then
GPG_PUBKEY_B64=$(gpg --armor --export "$GPG_SIGNER" | base64 -w 0)
ok "GPG public key exported (${#GPG_PUBKEY_B64} chars base64)"
else
warn "GPG key for $GPG_SIGNER not found locally — remote import will try keyservers"
fi
# ── Step 3: Deploy on remote ───────────────────────────────────────
# Build a single shell script that runs entirely on the server.
# This avoids scp/rsync issues and ensures atomic, auditable deployment.
DEPLOY_SCRIPT=$(cat << 'REMOTE_EOF'
#!/bin/bash
set -euo pipefail
TAG="__TAG__"
ASSET_NAME="__ASSET_NAME__"
TARBALL_URL="__TARBALL_URL__"
SHA256_URL="__SHA256_URL__"
SIG_URL="__SIG_URL__"
ASC_URL="__ASC_URL__"
GPG_SIGNER="__GPG_SIGNER__"
GPG_KEY_ID="__GPG_KEY_ID__"
INSTALL_DIR="__INSTALL_DIR__"
SERVICES="__SERVICES__"
DRY_RUN="__DRY_RUN__"
info() { echo -e "\033[1m==> $1\033[0m"; }
ok() { echo -e "\033[32;1m ✔ $1\033[0m"; }
warn() { echo -e "\033[33;1m ⚠ $1\033[0m"; }
err() { echo -e "\033[31;1m ✖ $1\033[0m"; }
die() { err "$1"; exit "${2:-1}"; }
WORKDIR=$(mktemp -d /tmp/bal-deploy.XXXXXX)
trap 'rm -rf "$WORKDIR"' EXIT
# ── Ensure GPG is available ─────────────────────────────────────────
if ! command -v gpg &>/dev/null; then
info "Installing gnupg ..."
sudo apt-get update -qq && sudo apt-get install -y -qq gnupg
fi
# ── Import GPG key ──────────────────────────────────────────────────
info "Importing GPG key $GPG_KEY_ID ..."
if ! gpg --list-keys "$GPG_SIGNER" &>/dev/null; then
imported=0
# Try embedded key first (base64-encoded, pushed from deployer)
if [[ -n "__GPG_PUBKEY_B64__" ]]; then
echo "__GPG_PUBKEY_B64__" | base64 -d | gpg --batch --import 2>/dev/null && imported=1 && ok "Key imported from deployer"
fi
# Fallback to keyservers
if [[ $imported -eq 0 ]]; then
for ks in keyserver.ubuntu.com keys.openpgp.org pgp.mit.edu; do
if gpg --batch --keyserver "$ks" --recv-keys "$GPG_KEY_ID" 2>/dev/null; then
ok "Key imported from $ks"
imported=1
break
fi
done
fi
if [[ $imported -eq 0 ]]; then
die "Cannot import GPG key $GPG_KEY_ID. Import it manually and re-run."
fi
else
ok "GPG key already present"
fi
# ── Download release assets ─────────────────────────────────────────
info "Downloading $ASSET_NAME ..."
curl -sfL -o "$WORKDIR/$ASSET_NAME" "$TARBALL_URL" || die "Download failed"
for sidecar in "sha256:$SHA256_URL:.sha256" "sig:$SIG_URL:.sig" "asc:$ASC_URL:.asc"; do
IFS=: read -r label url suffix <<< "$sidecar"
if curl -sfL -o "$WORKDIR/$(basename "$ASSET_NAME")$suffix" "$url" 2>/dev/null; then
ok "Downloaded $ASSET_NAME$suffix"
else
warn "$label file not available — skipping"
fi
done
# ── Verify SHA-256 ──────────────────────────────────────────────────
SHA_FILE="$WORKDIR/${ASSET_NAME}.sha256"
if [[ -f "$SHA_FILE" ]]; then
info "Verifying SHA-256 checksum ..."
(cd "$WORKDIR" && sha256sum -c "$SHA_FILE") || die "SHA-256 verification FAILED"
ok "SHA-256 OK"
else
warn "No .sha256 file — skipping checksum"
fi
# ── Verify GPG signature ────────────────────────────────────────────
SIG_FILE="$WORKDIR/${ASSET_NAME}.sig"
ASC_FILE="$WORKDIR/${ASSET_NAME}.asc"
verified=0
if [[ -f "$SIG_FILE" ]]; then
info "Verifying GPG signature (binary) ..."
gpg --batch --verify "$SIG_FILE" "$WORKDIR/$ASSET_NAME" 2>&1 && verified=1
fi
if [[ $verified -eq 0 ]] && [[ -f "$ASC_FILE" ]]; then
info "Verifying GPG signature (ASCII-armored) ..."
gpg --batch --verify "$ASC_FILE" "$WORKDIR/$ASSET_NAME" 2>&1 && verified=1
fi
if [[ $verified -eq 0 ]]; then
warn "No GPG signature available — skipping signature verification"
fi
# ── Extract ──────────────────────────────────────────────────────────
info "Extracting tarball ..."
tar -xzf "$WORKDIR/$ASSET_NAME" -C "$WORKDIR"
EXTRACTED="$WORKDIR/$(basename "$ASSET_NAME" .tar.gz)"
for bin in bal-server bal-pusher; do
if [[ ! -f "$EXTRACTED/$bin" ]]; then
die "Binary '$bin' not found in archive"
fi
ok "Found $bin ($(stat -c%s "$EXTRACTED/$bin") bytes)"
done
# ── Stop services ────────────────────────────────────────────────────
info "Stopping services ..."
IFS=',' read -ra SVC_LIST <<< "$SERVICES"
for svc in "${SVC_LIST[@]}"; do
svc=$(echo "$svc" | xargs) # trim whitespace
if systemctl is-active --quiet "$svc" 2>/dev/null; then
sudo systemctl stop "$svc" || warn "Failed to stop $svc"
ok "Stopped $svc"
else
warn "$svc is not running"
fi
done
# ── Backup current binaries ─────────────────────────────────────────
BACKUP_DIR="$WORKDIR/backup"
mkdir -p "$BACKUP_DIR"
for bin in bal-server bal-pusher; do
if [[ -f "$INSTALL_DIR/$bin" ]]; then
cp "$INSTALL_DIR/$bin" "$BACKUP_DIR/$bin" 2>/dev/null || true
fi
done
ok "Current binaries backed up"
# ── Install new binaries ────────────────────────────────────────────
info "Installing binaries to $INSTALL_DIR ..."
sudo install -m 0755 -o root -g root "$EXTRACTED/bal-server" "$EXTRACTED/bal-pusher" "$INSTALL_DIR/"
ok "Installed bal-server and bal-pusher"
# ── Restart services ────────────────────────────────────────────────
info "Restarting services ..."
sudo systemctl daemon-reload
for svc in "${SVC_LIST[@]}"; do
svc=$(echo "$svc" | xargs)
if systemctl is-enabled --quiet "$svc" 2>/dev/null; then
sudo systemctl restart "$svc" || warn "Failed to restart $svc"
ok "Restarted $svc"
else
warn "$svc is not enabled — skipping restart"
fi
done
# ── Verify ──────────────────────────────────────────────────────────
echo ""
info "Checking service status ..."
sleep 2
for svc in "${SVC_LIST[@]}"; do
svc=$(echo "$svc" | xargs)
status=$(systemctl is-active "$svc" 2>/dev/null || echo "inactive")
if [[ "$status" == "active" ]]; then
ok "$svc: active"
else
warn "$svc: $status"
fi
done
echo ""
ok "Deploy of $TAG completed successfully!"
REMOTE_EOF
)
# ── Substitute variables into the remote script ─────────────────────
DEPLOY_SCRIPT="${DEPLOY_SCRIPT//__TAG__/$TAG}"
DEPLOY_SCRIPT="${DEPLOY_SCRIPT//__ASSET_NAME__/$ASSET_NAME}"
DEPLOY_SCRIPT="${DEPLOY_SCRIPT//__TARBALL_URL__/$TARBALL_URL}"
DEPLOY_SCRIPT="${DEPLOY_SCRIPT//__SHA256_URL__/$SHA256_URL}"
DEPLOY_SCRIPT="${DEPLOY_SCRIPT//__SIG_URL__/$SIG_URL}"
DEPLOY_SCRIPT="${DEPLOY_SCRIPT//__ASC_URL__/$ASC_URL}"
DEPLOY_SCRIPT="${DEPLOY_SCRIPT//__GPG_SIGNER__/$GPG_SIGNER}"
DEPLOY_SCRIPT="${DEPLOY_SCRIPT//__GPG_KEY_ID__/$GPG_KEY_ID}"
DEPLOY_SCRIPT="${DEPLOY_SCRIPT//__INSTALL_DIR__/$INSTALL_DIR}"
DEPLOY_SCRIPT="${DEPLOY_SCRIPT//__DRY_RUN__/$DRY_RUN}"
# Join SERVICES array into comma-separated string for the remote script
SERVICES_STR=$(IFS=,; echo "${SERVICES[*]}")
DEPLOY_SCRIPT="${DEPLOY_SCRIPT//__SERVICES__/$SERVICES_STR}"
DEPLOY_SCRIPT="${DEPLOY_SCRIPT//__GPG_PUBKEY_B64__/${GPG_PUBKEY_B64:-}}"
# ── Execute ──────────────────────────────────────────────────────────
if [[ $DRY_RUN -eq 1 ]]; then
info "=== DRY RUN — would execute the following on remote: ==="
echo "$DEPLOY_SCRIPT"
echo "=== end dry run ==="
exit 0
fi
REMOTE_SCRIPT="/tmp/bal-deploy-$(date +%s).sh"
info "Uploading deploy script to remote ..."
ssh $SSH_OPTS "${REMOTE_USER}@${REMOTE_HOST}" "cat > '$REMOTE_SCRIPT'" <<< "$DEPLOY_SCRIPT" \
|| die "Failed to upload deploy script"
info "Executing deploy on remote server ..."
ssh $SSH_OPTS "${REMOTE_USER}@${REMOTE_HOST}" "chmod +x '$REMOTE_SCRIPT' && bash '$REMOTE_SCRIPT'; rc=\$?; rm -f '$REMOTE_SCRIPT'; exit \$rc" \
|| die "Deploy script failed on remote (exit $?)"
echo ""
ok "Deploy finished!"

19
contrib/lib.sh Normal file
View File

@@ -0,0 +1,19 @@
echo_i() {
echo -e "\033[1m==> $1\033[0m"
}
echo_e() {
echo -e "\033[31;1m$1\033[0m"
}
echo_s() {
echo -e "\033[32;1m$1\033[0m"
}
echo_w() {
echo -e "\033[33;1m$1\033[0m"
}
alias echo_i=echo_i
alias echo_e=echo_e
alias echo_s=echo_s
alias echo_w=echo_w

3
docker/s6-rc.d/bal-pusher/run Executable file
View File

@@ -0,0 +1,3 @@
#!/command/with-contenv sh
exec s6-setsid logutil-service bal-pusher \
/usr/local/bin/bal-pusher

1
docker/s6-rc.d/bal-pusher/type Normal file
View File

@@ -0,0 +1 @@
longrun

3
docker/s6-rc.d/bal-server/run Executable file
View File

@@ -0,0 +1,3 @@
#!/command/with-contenv sh
exec s6-setsid logutil-service bal-server \
/usr/local/bin/bal-server

1
docker/s6-rc.d/bal-server/type Normal file
View File

@@ -0,0 +1 @@
longrun

0
docker/s6-rc.d/user/contents.d/bal-pusher Normal file
View File

0
docker/s6-rc.d/user/contents.d/bal-server Normal file
View File

12
generate_random_ascii.sh Normal file
View File

@@ -0,0 +1,12 @@
#!/bin/bash
len=$1
if [ -z $len ]; then
len=32;
fi
# Genera 256 caratteri ASCII stampabili (32-126)
random_ascii_256=$(LC_ALL=C tr -dc 'A-Za-z0-9' < /dev/urandom | head -c $len)
echo "Stringa ASCII random (256 caratteri):"
echo "$random_ascii_256"
echo
echo "Lunghezza: ${#random_ascii_256} caratteri"

2
invalid_txs Normal file
View File

@@ -0,0 +1,2 @@
3508f2970e7d6a597db58e3cc1e72d7f0f81f782fa95b3ea0f9388693b6587b2: JSON-RPC error: RPC error response: RpcError { code: -25, message: "bad-txns-inputs-missingorspent", data: None } : 1752489317 : 1751860800
4df113c0d6c4f31cb01841edb1ce4434ec45bff78eef0b9fcf96129b0b21bb30: JSON-RPC error: RPC error response: RpcError { code: -27, message: "Transaction outputs already in utxo set", data: None } : 1752650972 : 1710573876

19
lib.sh Normal file
View File

@@ -0,0 +1,19 @@
echo_i() {
echo -e "\033[1m==> $1\033[0m"
}
echo_e() {
echo -e "\033[31;1m$1\033[0m"
}
echo_s() {
echo -e "\033[32;1m$1\033[0m"
}
echo_w() {
echo -e "\033[33;1m$1\033[0m"
}
alias echo_i=echo_i
alias echo_e=echo_e
alias echo_s=echo_s
alias echo_w=echo_w

3
public_key.pem Normal file
View File

@@ -0,0 +1,3 @@
-----BEGIN PUBLIC KEY-----
MCowBQYDK2VwAyEAm6ADLznAAoMqr4nDoLF9KdkAd397eA7BcINlvhpxFm8=
-----END PUBLIC KEY-----

159
releases/0.3.0/x86_64/README.md Normal file
View File

@@ -0,0 +1,159 @@
# bal-server
## Installation
```bash
git clone https://bitcoin-after.life/gitea/bitcoinafterlife/bal-server.git
cd bal-server
openssl genpkey -algorithm ED25519 -out private_key.pem
openssl pkey -in private_key.pem -pubout -out public_key.pem
cargo build --release
sudo cp target/release/bal-server target/release/bal-pusher /usr/local/bin
```
## Docker
### Build
```bash
docker build -t bal-server .
```
### Run
```bash
docker run -d \
--name bal-server \
--network host \
--tmpfs /tmp:rw,noexec,nosuid \
-v /path/to/data:/var/bal:rw \
-v /path/to/.bitcoin/regtest/.cookie:/var/bal/.bitcoin/regtest/.cookie:ro \
-e BAL_SERVER_REGTEST_ADDRESS="your_xpub_or_address" \
-e BAL_SERVER_REGTEST_FIXED_FEE=50000 \
-e BAL_SERVER_INFO="BAL server" \
-e BAL_PUSHER_NETWORK=regtest \
-e BAL_PUSHER_REGTEST_ZMQ_HASHBLOCK=tcp://127.0.0.1:28332 \
-e BAL_PUSHER_REGTEST_COOKIE_FILE=/var/bal/.bitcoin/regtest/.cookie \
bal-server
```
### Docker environment variables
| Variable | Description | Default |
| --- | --- | --- |
| `BAL_PUSHER_NETWORK` | Network to run pusher on (`bitcoin`, `testnet`, `testnet4`, `signet`, `regtest`). | `bitcoin` |
| `BAL_PUSHER_REGTEST_ZMQ_HASHBLOCK` | ZMQ endpoint for regtest blocks. | `tcp://127.0.0.1:21332` |
| `BAL_PUSHER_REGTEST_COOKIE_FILE` | Absolute path to Bitcoin Core cookie file inside the container. | - |
> **Note:** The container runs as a non-root `bal` user (uid 1000) with `tini` as PID 1.
> The `/var/bal` volume stores the database. Mount Bitcoin Core's cookie file as read-only.
> When using `--network host`, ensure only `127.0.0.1` is used for internal services.
## Configuration (bal-server)
The `bal-server` application can be configured using environment variables.
### General
| Variable | Description | Default |
| --- | --- | --- |
| `BAL_SERVER_DB_FILE` | Path to the SQLite3 database file. | `bal.db` |
| `BAL_SERVER_BIND_ADDRESS` | Address to listen on. **Never bind to `0.0.0.0` in production without a reverse proxy.** | `127.0.0.1` |
| `BAL_SERVER_BIND_PORT` | Port to listen on. | `9137` |
| `BAL_SERVER_INFO` | Server info string returned by the `/` endpoint. | - |
| `BAL_SERVER_PUB_KEY_PATH` | Ed25519 public key for signature verification. | `public_key.pem` |
| `BAL_SERVER_URL` | Public URL of this server (used for stats reporting). | - |
| `SSL_KEY_PATH` | Ed25519 private key for signing stats reports. | `private_key.pem` |
| `RUST_LOG` | Log level (`error`, `warn`, `info`, `debug`, `trace`). | `info` |
### Per-network addresses and fees
| Variable | Description | Default |
| --- | --- | --- |
| `BAL_SERVER_BITCOIN_ADDRESS` | xpub or address for mainnet. | - |
| `BAL_SERVER_BITCOIN_FIXED_FEE` | Fixed fee (satoshis) for mainnet. | `50000` |
| `BAL_SERVER_REGTEST_ADDRESS` | xpub or address for regtest. | - |
| `BAL_SERVER_REGTEST_FIXED_FEE` | Fixed fee (satoshis) for regtest. | `50000` |
| `BAL_SERVER_SIGNET_ADDRESS` | xpub or address for signet. | - |
| `BAL_SERVER_SIGNET_FIXED_FEE` | Fixed fee (satoshis) for signet. | `50000` |
| `BAL_SERVER_TESTNET_ADDRESS` | xpub or address for testnet. | - |
| `BAL_SERVER_TESTNET_FIXED_FEE` | Fixed fee (satoshis) for testnet. | `50000` |
| `BAL_SERVER_TESTNET4_ADDRESS` | xpub or address for testnet4. | - |
| `BAL_SERVER_TESTNET4_FIXED_FEE` | Fixed fee (satoshis) for testnet4. | `50000` |
### DoS protection (Actix Web)
| Variable | Description | Default |
| --- | --- | --- |
| `BAL_SERVER_ACTIX_MAX_BODY_SIZE` | Maximum request body size in bytes. | `1048576` (1 MB) |
| `BAL_SERVER_ACTIX_TIMEOUT_SECS` | Request timeout in seconds. | `5` |
| `BAL_SERVER_ACTIX_PUSHTXS_PER_SEC` | Rate limit: push txs requests per second. | `1` |
| `BAL_SERVER_ACTIX_PUSHTXS_BURST` | Rate limit: push txs burst size. | `3` |
| `BAL_SERVER_ACTIX_SEARCHTX_PER_SEC` | Rate limit: search tx requests per second. | `5` |
| `BAL_SERVER_ACTIX_SEARCHTX_BURST` | Rate limit: search tx burst size. | `10` |
| `BAL_SERVER_ACTIX_INFO_PER_SEC` | Rate limit: info requests per second. | `20` |
| `BAL_SERVER_ACTIX_INFO_BURST` | Rate limit: info burst size. | `30` |
| `BAL_SERVER_ACTIX_DEFAULT_PER_SEC` | Rate limit: default requests per second. | `50` |
| `BAL_SERVER_ACTIX_DEFAULT_BURST` | Rate limit: default burst size. | `100` |
| `BAL_SERVER_ACTIX_WORKERS` | Number of Actix worker threads. | `4` |
| `BAL_SERVER_ACTIX_MAX_CONNECTIONS` | Maximum concurrent connections. | `100` |
---
# bal-pusher
`bal-pusher` monitors Bitcoin blocks via ZMQ and pushes time-locked transactions from the database to the Bitcoin network when their **locktime** exceeds the **median time past** (MTP).
## Prerequisites
- **Bitcoin Core** with ZMQ support enabled. Add to `bitcoin.conf`:
```
zmqpubhashblock=tcp://127.0.0.1:28332
```
- **Rust and Cargo**: [Rust Installation](https://www.rust-lang.org/tools/install)
- **Libraries**: `libssl-dev`, `libsodium-dev`, `libzmq5-dev`, `libsqlite3-dev`
## Running
```bash
bal-pusher [bitcoin|testnet|testnet4|signet|regtest]
```
If no network is specified, defaults to `bitcoin`.
## Configuration (bal-pusher)
### General
| Variable | Description | Default |
| --- | --- | --- |
| `BAL_PUSHER_DB_FILE` | Path to the SQLite3 database file. | `bal.db` |
| `BAL_PUSHER_SEND_STATS` | Send stats to welist server. | `false` |
| `BAL_SERVER_URL` | URL of bal-server (for stats reporting). | - |
| `SSL_KEY_PATH` | Ed25519 private key for signing stats reports. | `private_key.pem` |
| `WELIST_SERVER_URL` | Welist server URL. | `https://welist.bitcoin-after.life` |
### Per-network configuration
Each network (`bitcoin`, `regtest`, `testnet`, `testnet4`, `signet`) supports the following variables.
Replace `{NETWORK}` with the uppercase network name (e.g., `REGTEST`, `BITCOIN`).
| Variable | Description | Default |
| --- | --- | --- |
| `BAL_PUSHER_{NETWORK}_ZMQ_HASHBLOCK` | ZMQ endpoint for block notifications. | `tcp://127.0.0.1:28332` (mainnet) |
| `BAL_PUSHER_{NETWORK}_COOKIE_FILE` | Absolute path to Bitcoin Core cookie file. | `$HOME/.bitcoin/{dir}/.cookie` |
| `BAL_PUSHER_{NETWORK}_RPC_USER` | Bitcoin Core RPC username (alternative to cookie auth). | - |
| `BAL_PUSHER_{NETWORK}_RPC_PASSWORD` | Bitcoin Core RPC password. | - |
| `BAL_PUSHER_{NETWORK}_HOST` | Bitcoin Core RPC host. | `http://127.0.0.1` |
| `BAL_PUSHER_{NETWORK}_PORT` | Bitcoin Core RPC port. | `8332` (mainnet) |
| `BAL_PUSHER_{NETWORK}_DIR_PATH` | Bitcoin Core data directory subfolder. | `` (mainnet) |
Default ZMQ ports per network:
| Network | ZMQ Port | RPC Port |
| --- | --- | --- |
| `bitcoin` | 28332 | 8332 |
| `regtest` | 21332 | 18443 |
| `testnet` | 23332 | 18332 |
| `testnet4` | 22332 | 48332 |
| `signet` | 24332 | 38332 |

BIN
releases/0.3.0/x86_64/bal-pusher Executable file
View File

Binary file not shown.

BIN
releases/0.3.0/x86_64/bal-server Executable file
View File

Binary file not shown.

BIN
releases/0.3.0/x86_64/bal-server-0.3.0_x86_64_linux-gnu.tar.gz Normal file
View File

Binary file not shown.

12
releases/0.3.0/x86_64/bal-server-0.3.0_x86_64_linux-gnu.tar.gz.asc Normal file
View File

@@ -0,0 +1,12 @@
-----BEGIN PGP SIGNATURE-----
iQFSBAABCgA8FiEEqEfQBNuRYQcRymoN/nVnBugz4NEFAmpdecseHHN2YXRhbnRy
eWFAYml0Y29pbi1hZnRlci5saWZlAAoJEP51ZwboM+DRhBwH/jlj/9Uug1mEbPqe
cKPKC2cgRXfoHsVK2YUOZH1XB4wW4qmjJSDV6TyCMmy5F+z4I5rEhaa/QyLL2YIE
bPaT7xJ5xKp3gzmRpHrwPEyvQEDj6ZxeZuAg/Rzk6FL1O0R8PQhxOqzOrW3uMfN+
5VSrw8yaqNDoI5rpNF2R8D/Gfr2Ya4Xo9I0ScIaXUeH4oq5aHu3dFUI9jldfagbM
6u7U7xPeT4gwNF01r47GrMdIfXpOmC0q8ba0wLPfvkDLz4N907Pp0CpCq0Hg5IlH
F0MEh3ReKQXimhHWZ/gDex5M6gt5nFSZLxA68VvPoi5f3j7rTxrNveRyJFa6EPTH
mzp58Ac=
=tx0k
-----END PGP SIGNATURE-----

1
releases/0.3.0/x86_64/bal-server-0.3.0_x86_64_linux-gnu.tar.gz.sha256 Normal file
View File

@@ -0,0 +1 @@
e8055bf320b527266a071cf515759b4dec5d6a166f5bca6f23fa3d804eb52ffa bal-server-0.3.0_x86_64_linux-gnu.tar.gz

BIN
releases/0.3.0/x86_64/bal-server-0.3.0_x86_64_linux-gnu.tar.gz.sig Normal file
View File

Binary file not shown.

159
releases/0.3.0/x86_64/bal-server-0.3.0_x86_64_linux-gnu/README.md Normal file
View File

@@ -0,0 +1,159 @@
# bal-server
## Installation
```bash
git clone https://bitcoin-after.life/gitea/bitcoinafterlife/bal-server.git
cd bal-server
openssl genpkey -algorithm ED25519 -out private_key.pem
openssl pkey -in private_key.pem -pubout -out public_key.pem
cargo build --release
sudo cp target/release/bal-server target/release/bal-pusher /usr/local/bin
```
## Docker
### Build
```bash
docker build -t bal-server .
```
### Run
```bash
docker run -d \
--name bal-server \
--network host \
--tmpfs /tmp:rw,noexec,nosuid \
-v /path/to/data:/var/bal:rw \
-v /path/to/.bitcoin/regtest/.cookie:/var/bal/.bitcoin/regtest/.cookie:ro \
-e BAL_SERVER_REGTEST_ADDRESS="your_xpub_or_address" \
-e BAL_SERVER_REGTEST_FIXED_FEE=50000 \
-e BAL_SERVER_INFO="BAL server" \
-e BAL_PUSHER_NETWORK=regtest \
-e BAL_PUSHER_REGTEST_ZMQ_HASHBLOCK=tcp://127.0.0.1:28332 \
-e BAL_PUSHER_REGTEST_COOKIE_FILE=/var/bal/.bitcoin/regtest/.cookie \
bal-server
```
### Docker environment variables
| Variable | Description | Default |
| --- | --- | --- |
| `BAL_PUSHER_NETWORK` | Network to run pusher on (`bitcoin`, `testnet`, `testnet4`, `signet`, `regtest`). | `bitcoin` |
| `BAL_PUSHER_REGTEST_ZMQ_HASHBLOCK` | ZMQ endpoint for regtest blocks. | `tcp://127.0.0.1:21332` |
| `BAL_PUSHER_REGTEST_COOKIE_FILE` | Absolute path to Bitcoin Core cookie file inside the container. | - |
> **Note:** The container runs as a non-root `bal` user (uid 1000) with `tini` as PID 1.
> The `/var/bal` volume stores the database. Mount Bitcoin Core's cookie file as read-only.
> When using `--network host`, ensure only `127.0.0.1` is used for internal services.
## Configuration (bal-server)
The `bal-server` application can be configured using environment variables.
### General
| Variable | Description | Default |
| --- | --- | --- |
| `BAL_SERVER_DB_FILE` | Path to the SQLite3 database file. | `bal.db` |
| `BAL_SERVER_BIND_ADDRESS` | Address to listen on. **Never bind to `0.0.0.0` in production without a reverse proxy.** | `127.0.0.1` |
| `BAL_SERVER_BIND_PORT` | Port to listen on. | `9137` |
| `BAL_SERVER_INFO` | Server info string returned by the `/` endpoint. | - |
| `BAL_SERVER_PUB_KEY_PATH` | Ed25519 public key for signature verification. | `public_key.pem` |
| `BAL_SERVER_URL` | Public URL of this server (used for stats reporting). | - |
| `SSL_KEY_PATH` | Ed25519 private key for signing stats reports. | `private_key.pem` |
| `RUST_LOG` | Log level (`error`, `warn`, `info`, `debug`, `trace`). | `info` |
### Per-network addresses and fees
| Variable | Description | Default |
| --- | --- | --- |
| `BAL_SERVER_BITCOIN_ADDRESS` | xpub or address for mainnet. | - |
| `BAL_SERVER_BITCOIN_FIXED_FEE` | Fixed fee (satoshis) for mainnet. | `50000` |
| `BAL_SERVER_REGTEST_ADDRESS` | xpub or address for regtest. | - |
| `BAL_SERVER_REGTEST_FIXED_FEE` | Fixed fee (satoshis) for regtest. | `50000` |
| `BAL_SERVER_SIGNET_ADDRESS` | xpub or address for signet. | - |
| `BAL_SERVER_SIGNET_FIXED_FEE` | Fixed fee (satoshis) for signet. | `50000` |
| `BAL_SERVER_TESTNET_ADDRESS` | xpub or address for testnet. | - |
| `BAL_SERVER_TESTNET_FIXED_FEE` | Fixed fee (satoshis) for testnet. | `50000` |
| `BAL_SERVER_TESTNET4_ADDRESS` | xpub or address for testnet4. | - |
| `BAL_SERVER_TESTNET4_FIXED_FEE` | Fixed fee (satoshis) for testnet4. | `50000` |
### DoS protection (Actix Web)
| Variable | Description | Default |
| --- | --- | --- |
| `BAL_SERVER_ACTIX_MAX_BODY_SIZE` | Maximum request body size in bytes. | `1048576` (1 MB) |
| `BAL_SERVER_ACTIX_TIMEOUT_SECS` | Request timeout in seconds. | `5` |
| `BAL_SERVER_ACTIX_PUSHTXS_PER_SEC` | Rate limit: push txs requests per second. | `1` |
| `BAL_SERVER_ACTIX_PUSHTXS_BURST` | Rate limit: push txs burst size. | `3` |
| `BAL_SERVER_ACTIX_SEARCHTX_PER_SEC` | Rate limit: search tx requests per second. | `5` |
| `BAL_SERVER_ACTIX_SEARCHTX_BURST` | Rate limit: search tx burst size. | `10` |
| `BAL_SERVER_ACTIX_INFO_PER_SEC` | Rate limit: info requests per second. | `20` |
| `BAL_SERVER_ACTIX_INFO_BURST` | Rate limit: info burst size. | `30` |
| `BAL_SERVER_ACTIX_DEFAULT_PER_SEC` | Rate limit: default requests per second. | `50` |
| `BAL_SERVER_ACTIX_DEFAULT_BURST` | Rate limit: default burst size. | `100` |
| `BAL_SERVER_ACTIX_WORKERS` | Number of Actix worker threads. | `4` |
| `BAL_SERVER_ACTIX_MAX_CONNECTIONS` | Maximum concurrent connections. | `100` |
---
# bal-pusher
`bal-pusher` monitors Bitcoin blocks via ZMQ and pushes time-locked transactions from the database to the Bitcoin network when their **locktime** exceeds the **median time past** (MTP).
## Prerequisites
- **Bitcoin Core** with ZMQ support enabled. Add to `bitcoin.conf`:
```
zmqpubhashblock=tcp://127.0.0.1:28332
```
- **Rust and Cargo**: [Rust Installation](https://www.rust-lang.org/tools/install)
- **Libraries**: `libssl-dev`, `libsodium-dev`, `libzmq5-dev`, `libsqlite3-dev`
## Running
```bash
bal-pusher [bitcoin|testnet|testnet4|signet|regtest]
```
If no network is specified, defaults to `bitcoin`.
## Configuration (bal-pusher)
### General
| Variable | Description | Default |
| --- | --- | --- |
| `BAL_PUSHER_DB_FILE` | Path to the SQLite3 database file. | `bal.db` |
| `BAL_PUSHER_SEND_STATS` | Send stats to welist server. | `false` |
| `BAL_SERVER_URL` | URL of bal-server (for stats reporting). | - |
| `SSL_KEY_PATH` | Ed25519 private key for signing stats reports. | `private_key.pem` |
| `WELIST_SERVER_URL` | Welist server URL. | `https://welist.bitcoin-after.life` |
### Per-network configuration
Each network (`bitcoin`, `regtest`, `testnet`, `testnet4`, `signet`) supports the following variables.
Replace `{NETWORK}` with the uppercase network name (e.g., `REGTEST`, `BITCOIN`).
| Variable | Description | Default |
| --- | --- | --- |
| `BAL_PUSHER_{NETWORK}_ZMQ_HASHBLOCK` | ZMQ endpoint for block notifications. | `tcp://127.0.0.1:28332` (mainnet) |
| `BAL_PUSHER_{NETWORK}_COOKIE_FILE` | Absolute path to Bitcoin Core cookie file. | `$HOME/.bitcoin/{dir}/.cookie` |
| `BAL_PUSHER_{NETWORK}_RPC_USER` | Bitcoin Core RPC username (alternative to cookie auth). | - |
| `BAL_PUSHER_{NETWORK}_RPC_PASSWORD` | Bitcoin Core RPC password. | - |
| `BAL_PUSHER_{NETWORK}_HOST` | Bitcoin Core RPC host. | `http://127.0.0.1` |
| `BAL_PUSHER_{NETWORK}_PORT` | Bitcoin Core RPC port. | `8332` (mainnet) |
| `BAL_PUSHER_{NETWORK}_DIR_PATH` | Bitcoin Core data directory subfolder. | `` (mainnet) |
Default ZMQ ports per network:
| Network | ZMQ Port | RPC Port |
| --- | --- | --- |
| `bitcoin` | 28332 | 8332 |
| `regtest` | 21332 | 18443 |
| `testnet` | 23332 | 18332 |
| `testnet4` | 22332 | 48332 |
| `signet` | 24332 | 38332 |

BIN
releases/0.3.0/x86_64/bal-server-0.3.0_x86_64_linux-gnu/bal-pusher Executable file
View File

Binary file not shown.

BIN
releases/0.3.0/x86_64/bal-server-0.3.0_x86_64_linux-gnu/bal-server Executable file
View File

Binary file not shown.

BIN
releases/0.3.1/x86_64/bal-server-0.3.1_x86_64_linux-gnu.tar.gz Normal file
View File

Binary file not shown.

12
releases/0.3.1/x86_64/bal-server-0.3.1_x86_64_linux-gnu.tar.gz.asc Normal file
View File

@@ -0,0 +1,12 @@
-----BEGIN PGP SIGNATURE-----
iQFSBAABCgA8FiEEqEfQBNuRYQcRymoN/nVnBugz4NEFAmpeGfseHHN2YXRhbnRy
eWFAYml0Y29pbi1hZnRlci5saWZlAAoJEP51ZwboM+DRoJUH/A+pp7RizJd0g60u
FULkipJxDZhXQ5TJoWkjnVVRvkNfnoAms86Bik0sLxt+e8wy3w+NziUU+/ZwheCX
hsOn1PANXjvyK8xWVKFjCpGNJY5R4iYe5cdpsY142J96UAHyrz1WJowl6HZ4wNwy
XsNkOX1m5z/twrHosUULKLUVxs6Us4jit1SGT+4CP7us4WDBN8IAYgnMrTv3QjmI
aAGOcOnB5FEfs0Egeh6a63l216boO/M4/eI7fkA1AnoxvVfUVnM7kL5MLwDvO6Gl
ytGJxfq32H2mpWVYBzFaS6aue0xPQKur1G7hopRteq340tC6u44CEcomlj3O9f0p
CR5QErI=
=0KWl
-----END PGP SIGNATURE-----

1
releases/0.3.1/x86_64/bal-server-0.3.1_x86_64_linux-gnu.tar.gz.sha256 Normal file
View File

@@ -0,0 +1 @@
b6cdb29904e51cfbe354c25b346d3573e1c85bfcdc5aa589ebb391a55d8e2aa2 bal-server-0.3.1_x86_64_linux-gnu.tar.gz

BIN
releases/0.3.1/x86_64/bal-server-0.3.1_x86_64_linux-gnu.tar.gz.sig Normal file
View File

Binary file not shown.

BIN
releases/0.3.2/x86_64/bal-server-0.3.2_x86_64_linux-gnu.tar.gz Normal file
View File

Binary file not shown.

12
releases/0.3.2/x86_64/bal-server-0.3.2_x86_64_linux-gnu.tar.gz.asc Normal file
View File

@@ -0,0 +1,12 @@
-----BEGIN PGP SIGNATURE-----
iQFSBAABCgA8FiEEqEfQBNuRYQcRymoN/nVnBugz4NEFAmpeXM8eHHN2YXRhbnRy
eWFAYml0Y29pbi1hZnRlci5saWZlAAoJEP51ZwboM+DR/08H/jY15bL48QVnPy0J
XsgiYXP06jSZNnOoLb9hxyE41T5fZjWp/Cdu9ZyOu/xatPhNXXItYe80CKxHk/py
JV9jdBUACVdjzWgC4xMZASUD0rwiv6UFBbMngchwy+WOubzsZY4Jp3Q/PBJm64rU
OrWlqJ7DzKpeUtBnoS/YL4PHBCV/HXkPsXjc4wvLBOKNP/5enRY/OIVr1ySR4qQs
HGODM3ozQeXipWebBtjs9K+p98baMXxxow7fCIT2W0jblhukRbckPPk5Bg8AklVg
+BrhcS0Gt3bpFylbmB/J1x8gP2/lWQxSBrLxdyQYMcXT+OwbZSLL2MEYinwcwnzC
yXH7OI8=
=QCat
-----END PGP SIGNATURE-----

1
releases/0.3.2/x86_64/bal-server-0.3.2_x86_64_linux-gnu.tar.gz.sha256 Normal file
View File

@@ -0,0 +1 @@
481727931fd73a1959c17095fa900f5871b75cebc0d176576845a4dda51808f3 bal-server-0.3.2_x86_64_linux-gnu.tar.gz

BIN
releases/0.3.2/x86_64/bal-server-0.3.2_x86_64_linux-gnu.tar.gz.sig Normal file
View File

Binary file not shown.

159
releases/0.3.2/x86_64/bal-server-0.3.2_x86_64_linux-gnu/README.md Normal file
View File

@@ -0,0 +1,159 @@
# bal-server
## Installation
```bash
git clone https://bitcoin-after.life/gitea/bitcoinafterlife/bal-server.git
cd bal-server
openssl genpkey -algorithm ED25519 -out private_key.pem
openssl pkey -in private_key.pem -pubout -out public_key.pem
cargo build --release
sudo cp target/release/bal-server target/release/bal-pusher /usr/local/bin
```
## Docker
### Build
```bash
docker build -t bal-server .
```
### Run
```bash
docker run -d \
--name bal-server \
--network host \
--tmpfs /tmp:rw,noexec,nosuid \
-v /path/to/data:/var/bal:rw \
-v /path/to/.bitcoin/regtest/.cookie:/var/bal/.bitcoin/regtest/.cookie:ro \
-e BAL_SERVER_REGTEST_ADDRESS="your_xpub_or_address" \
-e BAL_SERVER_REGTEST_FIXED_FEE=50000 \
-e BAL_SERVER_INFO="BAL server" \
-e BAL_PUSHER_NETWORK=regtest \
-e BAL_PUSHER_REGTEST_ZMQ_HASHBLOCK=tcp://127.0.0.1:28332 \
-e BAL_PUSHER_REGTEST_COOKIE_FILE=/var/bal/.bitcoin/regtest/.cookie \
bal-server
```
### Docker environment variables
| Variable | Description | Default |
| --- | --- | --- |
| `BAL_PUSHER_NETWORK` | Network to run pusher on (`bitcoin`, `testnet`, `testnet4`, `signet`, `regtest`). | `bitcoin` |
| `BAL_PUSHER_REGTEST_ZMQ_HASHBLOCK` | ZMQ endpoint for regtest blocks. | `tcp://127.0.0.1:21332` |
| `BAL_PUSHER_REGTEST_COOKIE_FILE` | Absolute path to Bitcoin Core cookie file inside the container. | - |
> **Note:** The container runs as a non-root `bal` user (uid 1000) with `tini` as PID 1.
> The `/var/bal` volume stores the database. Mount Bitcoin Core's cookie file as read-only.
> When using `--network host`, ensure only `127.0.0.1` is used for internal services.
## Configuration (bal-server)
The `bal-server` application can be configured using environment variables.
### General
| Variable | Description | Default |
| --- | --- | --- |
| `BAL_SERVER_DB_FILE` | Path to the SQLite3 database file. | `bal.db` |
| `BAL_SERVER_BIND_ADDRESS` | Address to listen on. **Never bind to `0.0.0.0` in production without a reverse proxy.** | `127.0.0.1` |
| `BAL_SERVER_BIND_PORT` | Port to listen on. | `9137` |
| `BAL_SERVER_INFO` | Server info string returned by the `/` endpoint. | - |
| `BAL_SERVER_PUB_KEY_PATH` | Ed25519 public key for signature verification. | `public_key.pem` |
| `BAL_SERVER_URL` | Public URL of this server (used for stats reporting). | - |
| `SSL_KEY_PATH` | Ed25519 private key for signing stats reports. | `private_key.pem` |
| `RUST_LOG` | Log level (`error`, `warn`, `info`, `debug`, `trace`). | `info` |
### Per-network addresses and fees
| Variable | Description | Default |
| --- | --- | --- |
| `BAL_SERVER_BITCOIN_ADDRESS` | xpub or address for mainnet. | - |
| `BAL_SERVER_BITCOIN_FIXED_FEE` | Fixed fee (satoshis) for mainnet. | `50000` |
| `BAL_SERVER_REGTEST_ADDRESS` | xpub or address for regtest. | - |
| `BAL_SERVER_REGTEST_FIXED_FEE` | Fixed fee (satoshis) for regtest. | `50000` |
| `BAL_SERVER_SIGNET_ADDRESS` | xpub or address for signet. | - |
| `BAL_SERVER_SIGNET_FIXED_FEE` | Fixed fee (satoshis) for signet. | `50000` |
| `BAL_SERVER_TESTNET_ADDRESS` | xpub or address for testnet. | - |
| `BAL_SERVER_TESTNET_FIXED_FEE` | Fixed fee (satoshis) for testnet. | `50000` |
| `BAL_SERVER_TESTNET4_ADDRESS` | xpub or address for testnet4. | - |
| `BAL_SERVER_TESTNET4_FIXED_FEE` | Fixed fee (satoshis) for testnet4. | `50000` |
### DoS protection (Actix Web)
| Variable | Description | Default |
| --- | --- | --- |
| `BAL_SERVER_ACTIX_MAX_BODY_SIZE` | Maximum request body size in bytes. | `1048576` (1 MB) |
| `BAL_SERVER_ACTIX_TIMEOUT_SECS` | Request timeout in seconds. | `5` |
| `BAL_SERVER_ACTIX_PUSHTXS_PER_SEC` | Rate limit: push txs requests per second. | `1` |
| `BAL_SERVER_ACTIX_PUSHTXS_BURST` | Rate limit: push txs burst size. | `3` |
| `BAL_SERVER_ACTIX_SEARCHTX_PER_SEC` | Rate limit: search tx requests per second. | `5` |
| `BAL_SERVER_ACTIX_SEARCHTX_BURST` | Rate limit: search tx burst size. | `10` |
| `BAL_SERVER_ACTIX_INFO_PER_SEC` | Rate limit: info requests per second. | `20` |
| `BAL_SERVER_ACTIX_INFO_BURST` | Rate limit: info burst size. | `30` |
| `BAL_SERVER_ACTIX_DEFAULT_PER_SEC` | Rate limit: default requests per second. | `50` |
| `BAL_SERVER_ACTIX_DEFAULT_BURST` | Rate limit: default burst size. | `100` |
| `BAL_SERVER_ACTIX_WORKERS` | Number of Actix worker threads. | `4` |
| `BAL_SERVER_ACTIX_MAX_CONNECTIONS` | Maximum concurrent connections. | `100` |
---
# bal-pusher
`bal-pusher` monitors Bitcoin blocks via ZMQ and pushes time-locked transactions from the database to the Bitcoin network when their **locktime** exceeds the **median time past** (MTP).
## Prerequisites
- **Bitcoin Core** with ZMQ support enabled. Add to `bitcoin.conf`:
```
zmqpubhashblock=tcp://127.0.0.1:28332
```
- **Rust and Cargo**: [Rust Installation](https://www.rust-lang.org/tools/install)
- **Libraries**: `libssl-dev`, `libsodium-dev`, `libzmq5-dev`, `libsqlite3-dev`
## Running
```bash
bal-pusher [bitcoin|testnet|testnet4|signet|regtest]
```
If no network is specified, defaults to `bitcoin`.
## Configuration (bal-pusher)
### General
| Variable | Description | Default |
| --- | --- | --- |
| `BAL_PUSHER_DB_FILE` | Path to the SQLite3 database file. | `bal.db` |
| `BAL_PUSHER_SEND_STATS` | Send stats to welist server. | `false` |
| `BAL_SERVER_URL` | URL of bal-server (for stats reporting). | - |
| `SSL_KEY_PATH` | Ed25519 private key for signing stats reports. | `private_key.pem` |
| `WELIST_SERVER_URL` | Welist server URL. | `https://welist.bitcoin-after.life` |
### Per-network configuration
Each network (`bitcoin`, `regtest`, `testnet`, `testnet4`, `signet`) supports the following variables.
Replace `{NETWORK}` with the uppercase network name (e.g., `REGTEST`, `BITCOIN`).
| Variable | Description | Default |
| --- | --- | --- |
| `BAL_PUSHER_{NETWORK}_ZMQ_HASHBLOCK` | ZMQ endpoint for block notifications. | `tcp://127.0.0.1:28332` (mainnet) |
| `BAL_PUSHER_{NETWORK}_COOKIE_FILE` | Absolute path to Bitcoin Core cookie file. | `$HOME/.bitcoin/{dir}/.cookie` |
| `BAL_PUSHER_{NETWORK}_RPC_USER` | Bitcoin Core RPC username (alternative to cookie auth). | - |
| `BAL_PUSHER_{NETWORK}_RPC_PASSWORD` | Bitcoin Core RPC password. | - |
| `BAL_PUSHER_{NETWORK}_HOST` | Bitcoin Core RPC host. | `http://127.0.0.1` |
| `BAL_PUSHER_{NETWORK}_PORT` | Bitcoin Core RPC port. | `8332` (mainnet) |
| `BAL_PUSHER_{NETWORK}_DIR_PATH` | Bitcoin Core data directory subfolder. | `` (mainnet) |
Default ZMQ ports per network:
| Network | ZMQ Port | RPC Port |
| --- | --- | --- |
| `bitcoin` | 28332 | 8332 |
| `regtest` | 21332 | 18443 |
| `testnet` | 23332 | 18332 |
| `testnet4` | 22332 | 48332 |
| `signet` | 24332 | 38332 |

BIN
releases/0.3.2/x86_64/bal-server-0.3.2_x86_64_linux-gnu/bal-pusher Executable file
View File

Binary file not shown.

BIN
releases/0.3.2/x86_64/bal-server-0.3.2_x86_64_linux-gnu/bal-server Executable file
View File

Binary file not shown.

2
sendtx.sh Normal file
View File

@@ -0,0 +1,2 @@
#!/bin/bash
rbitcoin-cli -rpcwallet=default sendrawtransaction $(rbitcoin-cli -rpcwallet=default gettransaction $(rbitcoin-cli -rpcwallet=default -named sendtoaddress address="$1" amount=0.0005 fee_rate=1)|jq -r .hex)

197
src/bin/bal-pusher.rs
View File

@@ -10,7 +10,6 @@ use log::{debug, error, info, trace, warn};
use serde::Deserialize;
use serde::Serialize;
use serde_json::json;
use sqlite::{Connection, Value};
use std::collections::HashMap;
use std::env;
use std::error::Error as StdError;
@@ -18,7 +17,7 @@ use std::str;
use std::{thread, time::Duration};
use zmq::{Context, Socket};
use bal_server::db::open_db;
use bal_server::db::{calculate_and_upsert_stats, get_pending_txs, open_database};
use bal_server::validation::is_valid_welist_url;
use base64::{Engine as _, engine::general_purpose};
use reqwest::Client as rClient;
@@ -29,7 +28,9 @@ const LOCKTIME_THRESHOLD: i64 = 5000000;
const VERSION: &str = env!("CARGO_PKG_VERSION");
#[derive(Debug, Clone, Serialize, Deserialize)]
struct MyConfig {
db_backend: String,
db_file: String,
pg_dsn: String,
bitcoin_dir: String,
regtest: NetworkParams,
testnet: NetworkParams,
@@ -44,7 +45,9 @@ struct MyConfig {
impl Default for MyConfig {
fn default() -> Self {
MyConfig {
db_backend: env::var("BAL_PUSHER_DB_BACKEND").unwrap_or("sqlite".to_string()),
db_file: env::var("BAL_PUSHER_DB_FILE").unwrap_or("bal.db".to_string()),
pg_dsn: env::var("BAL_PUSHER_PG_DSN").unwrap_or_default(),
bitcoin_dir: env::var("BAL_PUSHER_BITCOIN_DIR").unwrap_or("".to_string()),
regtest: get_network_params_default(Network::Regtest),
testnet: get_network_params_default(Network::Testnet),
@@ -206,114 +209,79 @@ async fn main_result(cfg: &MyConfig, network_params: &NetworkParams) -> Result<(
Ok((rpc, bcinfo)) => {
info!("connected");
info!("median time: {}", bcinfo.median_time);
//info!("height time: {}",bcinfo.median_time);
info!("blocks: {}", bcinfo.blocks);
debug!("best block hash: {}", bcinfo.best_block_hash);
let average_time = bcinfo.median_time;
let db = match open_db(&cfg.db_file) {
Ok(c) => c,
let connection_string = match cfg.db_backend.as_str() {
"sqlite" => cfg.db_file.clone(),
"postgresql" => cfg.pg_dsn.clone(),
other => {
error!("Unknown DB backend: {}", other);
return Ok(());
}
};
let db = match open_database(&cfg.db_backend, &connection_string).await {
Ok(pool) => pool,
Err(e) => {
error!("Fatal: {}", e);
std::process::exit(1);
}
};
info!("db open {}", &cfg.db_file);
info!("db open {}", &connection_string);
let sqlquery = "SELECT * FROM tbl_tx WHERE network = :network AND status = :status AND ( locktime < :bestblock_height OR locktime > :locktime_threshold AND locktime < :bestblock_time);";
let query_tx = match db.prepare(sqlquery) {
Ok(q) => q.into_iter(),
let pending_txs = match get_pending_txs(
&db,
&network_params.db_field,
LOCKTIME_THRESHOLD,
bcinfo.blocks as i64,
average_time as i64,
)
.await
{
Ok(txs) => txs,
Err(e) => {
warn!("tbl_tx not ready yet (tables may not exist): {}", e);
warn!("Failed to query pending transactions: {}", e);
return Ok(());
}
};
trace!("query_tx: {}", sqlquery);
trace!(":locktime_threshold: {}", LOCKTIME_THRESHOLD);
trace!(":bestblock_time: {}", average_time);
trace!(":bestblock_height: {}", bcinfo.blocks);
trace!(":network: {}", network_params.db_field.clone());
trace!(":status: {}", 0);
//let query_tx = db.prepare("SELECT * FROM tbl_tx where status = :status").unwrap().into_iter();
let mut pushed_txs: Vec<String> = Vec::new();
let mut invalid_txs: std::collections::HashMap<String, String> = HashMap::new();
for row_result in match query_tx.bind::<&[(_, Value)]>(
&[
(":locktime_threshold", LOCKTIME_THRESHOLD.into()),
(":bestblock_time", (average_time as i64).into()),
(":bestblock_height", (bcinfo.blocks as i64).into()),
(":network", network_params.db_field.clone().into()),
(":status", 0.into()),
][..],
) {
Ok(bound) => bound,
Err(e) => {
error!("Failed to bind query parameters: {}", e);
return Ok(());
}
} {
let row = match row_result {
Ok(r) => r,
Err(e) => {
warn!("Failed to read row: {}", e);
continue;
}
};
let tx = row.read::<&str, _>("tx");
let txid = row.read::<&str, _>("txid");
let locktime = row.read::<i64, _>("locktime");
let mut invalid_txs: HashMap<String, String> = HashMap::new();
for row in &pending_txs {
let txid = &row.txid;
let tx = &row.tx;
let locktime = row.locktime;
info!("to be pushed: {}: {}", txid, locktime);
match rpc.send_raw_transaction(tx) {
match rpc.send_raw_transaction(tx.as_str()) {
Ok(o) => {
info!("tx: {} pusshata PUSHED\n{}", txid, o);
pushed_txs.push(txid.to_string());
pushed_txs.push(txid.clone());
}
Err(err) => {
warn!("Error: {}\n{}", err, txid);
//store err in invalid_txs
invalid_txs.insert(txid.to_string(), err.to_string());
invalid_txs.insert(txid.clone(), err.to_string());
}
};
}
for txid in &pushed_txs {
let sql = "UPDATE tbl_tx SET status = 1 WHERE txid = ?";
match db.prepare(sql) {
Ok(mut stmt) => {
if let Err(e) = stmt.bind((1, Value::String(txid.clone()))) {
error!("Failed to bind txid for status update: {}", e);
continue;
}
let _ = stmt.next();
}
Err(e) => {
error!("Failed to prepare status update: {}", e);
}
if let Err(e) = bal_server::db::update_tx_status(&db, txid, 1, None).await {
error!("Failed to update tx status: {}", e);
}
}
for (txid, txerr) in &invalid_txs {
let sql = "UPDATE tbl_tx SET status = 2, push_err = ? WHERE txid = ?";
match db.prepare(sql) {
Ok(mut stmt) => {
if let Err(e) = stmt.bind((1, Value::String(txerr.clone()))) {
error!("Failed to bind txerr for error update: {}", e);
continue;
}
if let Err(e) = stmt.bind((2, Value::String(txid.clone()))) {
error!("Failed to bind txid for error update: {}", e);
continue;
}
let _ = stmt.next();
}
Err(e) => {
error!("Failed to prepare error update: {}", e);
}
if let Err(e) = bal_server::db::update_tx_status(&db, txid, 2, Some(txerr)).await {
error!("Failed to update tx status: {}", e);
}
}
if let Err(e) = send_stats_report(cfg, bcinfo).await {
error!("send_stats_report failed: {}", e);
}
if let Err(e) = calculate_stats(&db, network_params.db_field.clone()).await {
if let Err(e) = calculate_and_upsert_stats(&db, &network_params.db_field).await {
warn!("calculate_stats failed: {e}");
}
}
@@ -325,65 +293,8 @@ async fn main_result(cfg: &MyConfig, network_params: &NetworkParams) -> Result<(
}
Ok(())
}
async fn calculate_stats(db: &Connection, chain: String) -> Result<(), reqwest::Error> {
// Validate chain to prevent SQL injection via environment variable tampering
if !chain
.chars()
.all(|c| c.is_alphanumeric() || c == '-' || c == '_')
|| chain.is_empty()
{
error!("Invalid chain name: {chain}");
return Ok(());
}
//let sql = "drop table if exists tbl_stats;";
let sql = format!("DELETE FROM tbl_stats WHERE chain = '{chain}';");
if let Err(err) = db.execute(&sql) {
error!("error deleting from tbl_stats where chain:{chain} error: {err}");
}
let sql = format!(
"INSERT INTO tbl_stats (
report_date, chain, totals, waiting, sent, failed,
waiting_profit, sent_profit, missed_profit, unique_inputs
)
VALUES (
CURRENT_TIMESTAMP,
'{chain}',
(SELECT COUNT(*) FROM tbl_tx WHERE network = '{chain}'),
(SELECT COUNT(*) FROM tbl_tx WHERE status = 0 AND network = '{chain}'),
(SELECT COUNT(*) FROM tbl_tx WHERE status = 1 AND network = '{chain}'),
(SELECT COUNT(*) FROM tbl_tx WHERE status = 2 AND network = '{chain}'),
(SELECT IFNULL(SUM(our_fees),0) FROM tbl_tx WHERE status = 0 AND network = '{chain}'),
(SELECT IFNULL(SUM(our_fees),0) FROM tbl_tx WHERE status = 1 AND network = '{chain}'),
(SELECT IFNULL(SUM(our_fees),0) FROM tbl_tx WHERE status = 2 AND network = '{chain}'),
(SELECT COUNT(DISTINCT tbl_inp.in_txid)
FROM tbl_inp
JOIN tbl_tx ON tbl_inp.txid = tbl_tx.txid
WHERE tbl_tx.status = 0 AND tbl_tx.network = '{chain}')
)
ON CONFLICT(chain) DO UPDATE SET
report_date = excluded.report_date,
totals = excluded.totals,
waiting = excluded.waiting,
sent = excluded.sent,
failed = excluded.failed,
waiting_profit = excluded.waiting_profit,
sent_profit = excluded.sent_profit,
missed_profit = excluded.missed_profit,
unique_inputs = excluded.unique_inputs;
"
);
if let Err(err) = db.execute(&sql) {
error!("error inserting creating stats table {err}");
} else {
info!("tbl_stats creation success");
}
Ok(())
}
/// Parse the `(host, port)` pair from a base URL like `https://host[:port]`.
///
/// Falls back to the scheme's well-known default port (443 for `https`,
/// 80 for plain `http`), or to 443 when the scheme is unknown.
fn parse_host_port(base_url: &str) -> Option<(String, u16)> {
let url = Url::parse(base_url).ok()?;
let host = url
@@ -396,8 +307,6 @@ fn parse_host_port(base_url: &str) -> Option<(String, u16)> {
}
/// Resolve `host:port` and return the first IPv6 (AAAA) address, if any.
///
/// Returns `None` when the host has no IPv6 address.
async fn resolve_first_ipv6(host: &str, port: u16) -> Option<SocketAddr> {
use std::net::ToSocketAddrs;
let host = host.to_string();
@@ -412,14 +321,6 @@ async fn resolve_first_ipv6(host: &str, port: u16) -> Option<SocketAddr> {
.flatten()
}
/// Build the HTTP client used for welist reports.
///
/// When `BAL_PUSHER_PREFER_IPV6` is truthy, the welist host is resolved and
/// the client is pinned to its first IPv6 address (the original hostname is
/// still used for the `Host` header and TLS SNI). This works around networks
/// where the IPv4 route to the welist host is broken while IPv6 works: the
/// default connector may otherwise pick the broken family and the request
/// stalls. When the variable is unset (the default), behavior is unchanged.
async fn welist_http_client(welist_url: &str) -> rClient {
let prefer_ipv6 = env::var("BAL_PUSHER_PREFER_IPV6")
.unwrap_or("false".to_string())
@@ -521,6 +422,15 @@ fn sign_message(private_key_path: &str, message: &str) -> String {
}
fn parse_env(cfg: &mut MyConfig) {
if let Ok(value) = env::var("BAL_PUSHER_DB_BACKEND") {
cfg.db_backend = value;
}
if let Ok(value) = env::var("BAL_PUSHER_DB_FILE") {
cfg.db_file = value;
}
if let Ok(value) = env::var("BAL_PUSHER_PG_DSN") {
cfg.pg_dsn = value;
}
cfg.regtest = parse_env_netconfig(cfg, "regtest");
cfg.signet = parse_env_netconfig(cfg, "signet");
cfg.testnet = parse_env_netconfig(cfg, "testnet");
@@ -528,7 +438,6 @@ fn parse_env(cfg: &mut MyConfig) {
drop(parse_env_netconfig(cfg, "bitcoin"));
}
fn parse_env_netconfig(cfg_lock: &mut MyConfig, chain: &str) -> NetworkParams {
//fn parse_env_netconfig(cfg_lock: &MutexGuard<MyConfig>, chain: &str) -> &NetworkParams{
let cfg = match chain {
"regtest" => &mut cfg_lock.regtest,
"signet" => &mut cfg_lock.signet,

426
src/bin/bal-server.rs
View File

@@ -7,16 +7,14 @@ use chrono::Utc;
use hex_conservative::FromHex;
use log::{debug, error, info, trace};
use serde::{Deserialize, Serialize};
use sqlite::State;
use sqlite::{Connection, Value};
use std::collections::{HashMap, HashSet};
use std::env;
use std::fs;
use std::sync::Mutex;
use bal_server::db::{
check_duplicate_txids, create_database, execute_insert, get_all_addresses_by_xpub,
get_last_used_address_by_ip, get_next_address_index, insert_xpub, open_db, save_new_address,
DatabasePool, InsertInpData, InsertOutData, InsertTxData, check_duplicate_txids,
create_database, get_all_addresses_by_xpub, get_last_used_address_by_ip,
get_next_address_index, get_stats, insert_xpub, open_database, save_new_address, search_tx,
};
use bal_server::xpub::new_address_from_xpub;
@@ -56,7 +54,9 @@ struct MyConfig {
info: String,
bind_address: String,
bind_port: u16,
db_backend: String,
db_file: String,
pg_dsn: String,
pub_key_path: String,
expose_stats: bool,
}
@@ -71,7 +71,9 @@ impl Default for MyConfig {
mainnet: NetConfig::default_network("bitcoin".to_string(), Network::Bitcoin),
bind_address: "127.0.0.1".to_string(),
bind_port: 9137,
db_backend: "sqlite".to_string(),
db_file: "bal.db".to_string(),
pg_dsn: String::new(),
info: "Will Executor Server".to_string(),
pub_key_path: "public_key.pem".to_string(),
expose_stats: env::var("BAL_SERVER_EXPOSE_STATS")
@@ -192,7 +194,7 @@ fn parse_actix_config() -> ActixConfig {
}
struct AppState {
db: Mutex<Connection>,
db: DatabasePool,
cfg: MyConfig,
}
@@ -272,22 +274,14 @@ async fn echo_info(
address
}
true => {
// Lock #1: fetch existing address OR atomically claim next index
let next_idx = {
let db = match data.db.lock() {
Ok(g) => g,
Err(_p) => {
error!("DB mutex poisoned in echo_info (lookup phase)");
return HttpResponse::InternalServerError().body("error");
}
};
match get_last_used_address_by_ip(
&db,
if let Some(address) = get_last_used_address_by_ip(
&data.db,
&netconfig.name,
&netconfig.address,
&remote_addr,
) {
Some(address) => {
)
.await
{
return HttpResponse::Ok().json(InfoResponse {
address,
base_fee: netconfig.fixed_fee,
@@ -296,11 +290,10 @@ async fn echo_info(
version: VERSION.to_string(),
});
}
None => get_next_address_index(&db, &netconfig.name, &netconfig.address),
}
}; // lock released
// Derive address (CPU-bound, no lock held)
let next_idx =
get_next_address_index(&data.db, &netconfig.name, &netconfig.address).await;
let derived =
match new_address_from_xpub(&netconfig.address, next_idx.1, netconfig.network) {
Ok(address) => address,
@@ -310,20 +303,10 @@ async fn echo_info(
}
};
// Lock #2: save the newly derived address
{
let db = match data.db.lock() {
Ok(g) => g,
Err(_p) => {
error!("DB mutex poisoned in echo_info (save phase)");
return HttpResponse::InternalServerError().body("error");
}
};
save_new_address(&db, next_idx.0, &derived.0, &derived.1, &remote_addr);
save_new_address(&data.db, next_idx.0, &derived.0, &derived.1, &remote_addr).await;
debug!("save new address {} {}", derived.0, derived.1);
trace!("next {} {}", next_idx.0, next_idx.1);
derived.0
} // lock released
}
};
let info = InfoResponse {
@@ -357,83 +340,31 @@ async fn echo_stats(path: web::Path<String>, data: web::Data<AppState>) -> impl
if !data.cfg.expose_stats {
return HttpResponse::Forbidden().body("error");
}
let mut stats: Vec<StatsResponse> = vec![];
let db = match data.db.lock() {
Ok(g) => g,
Err(_p) => {
error!("DB mutex poisoned in echo_stats");
return HttpResponse::InternalServerError().body("error");
}
};
let mut stmt = match db.prepare(
"SELECT report_date, chain, totals, waiting, sent, failed, waiting_profit, sent_profit, missed_profit, unique_inputs FROM tbl_stats WHERE chain = ?"
) {
Ok(s) => s,
let stats_rows = match get_stats(&data.db, &netconfig.name).await {
Ok(rows) => rows,
Err(e) => {
error!("Failed to prepare stats query: {}", e);
error!("Failed to query stats: {}", e);
return HttpResponse::InternalServerError().body("error");
}
};
if let Err(e) = stmt.bind((1, Value::String(netconfig.name.clone()))) {
error!("Failed to bind chain in stats query: {}", e);
return HttpResponse::InternalServerError().body("error");
}
while let Ok(State::Row) = stmt.next() {
let report_date = stmt.read("report_date").unwrap_or("0".to_string());
let chain = stmt.read("chain").unwrap_or("?".to_string());
let totals = stmt
.read("totals")
.unwrap_or("0".to_string())
.parse::<i64>()
.unwrap_or(0);
let waiting = stmt
.read("waiting")
.unwrap_or("0".to_string())
.parse::<i64>()
.unwrap_or(0);
let sent = stmt
.read("sent")
.unwrap_or("0".to_string())
.parse::<i64>()
.unwrap_or(0);
let failed = stmt
.read("failed")
.unwrap_or("0".to_string())
.parse::<i64>()
.unwrap_or(0);
let waiting_profit = stmt
.read("waiting_profit")
.unwrap_or("0".to_string())
.parse::<i64>()
.unwrap_or(0);
let sent_profit = stmt
.read("sent_profit")
.unwrap_or("0".to_string())
.parse::<i64>()
.unwrap_or(0);
let missed_profit = stmt
.read("missed_profit")
.unwrap_or("0".to_string())
.parse::<i64>()
.unwrap_or(0);
let unique_inputs = stmt
.read("unique_inputs")
.unwrap_or("0".to_string())
.parse::<i64>()
.unwrap_or(0);
stats.push(StatsResponse {
report_date,
chain,
totals,
waiting,
sent,
failed,
waiting_profit,
sent_profit,
missed_profit,
unique_inputs,
});
}
let stats: Vec<StatsResponse> = stats_rows
.into_iter()
.map(|row| StatsResponse {
report_date: row.report_date,
chain: row.chain,
totals: row.totals,
waiting: row.waiting,
sent: row.sent,
failed: row.failed,
waiting_profit: row.waiting_profit,
sent_profit: row.sent_profit,
missed_profit: row.missed_profit,
unique_inputs: row.unique_inputs,
})
.collect();
debug!("echo stats reply for chain: {}", netconfig.name);
HttpResponse::Ok().json(stats)
}
@@ -453,67 +384,15 @@ async fn echo_search(body: Bytes, data: web::Data<AppState>) -> impl Responder {
return HttpResponse::BadRequest().body("error");
}
let db = match data.db.lock() {
Ok(g) => g,
Err(_p) => {
error!("DB mutex poisoned in echo_search");
return HttpResponse::InternalServerError().body("error");
}
};
let mut statement = match db.prepare("SELECT * FROM tbl_tx WHERE txid = ? LIMIT 1") {
Ok(s) => s,
Err(e) => {
error!("Failed to prepare statement: {}", e);
return HttpResponse::InternalServerError().body("error");
}
};
if let Err(e) = statement.bind((1, strbody)) {
error!("Failed to bind parameter: {}", e);
return HttpResponse::InternalServerError().body("error");
}
if let Ok(State::Row) = statement.next() {
match search_tx(&data.db, strbody).await {
Ok(Some(row)) => {
let mut response_data = HashMap::new();
match statement.read::<String, _>("status") {
Ok(value) => {
response_data.insert("status", value);
}
Err(e) => {
error!("Error reading status: {}", e);
}
}
match statement.read::<String, _>("tx") {
Ok(value) => {
response_data.insert("tx", value);
}
Err(e) => {
error!("Error reading tx: {}", e);
}
}
match statement.read::<String, _>("our_address") {
Ok(value) => {
response_data.insert("our_address", value);
}
Err(e) => {
error!("Error reading address: {}", e);
}
}
match statement.read::<String, _>("our_fees") {
Ok(value) => {
response_data.insert("our_fees", value);
}
Err(e) => {
error!("Error reading fees: {}", e);
}
}
match statement.read::<String, _>("reqid") {
Ok(value) => {
response_data.insert("time", value);
}
Err(e) => {
error!("Error reading reqid: {}", e);
}
}
response_data.insert("status", row.status);
response_data.insert("tx", row.tx);
response_data.insert("our_address", row.our_address);
response_data.insert("our_fees", row.our_fees);
response_data.insert("time", row.reqid);
match serde_json::to_string(&response_data) {
Ok(json_data) => {
debug!("echo search reply: {}", json_data);
@@ -521,25 +400,26 @@ async fn echo_search(body: Bytes, data: web::Data<AppState>) -> impl Responder {
}
Err(_) => HttpResponse::BadRequest().body("error"),
}
} else {
HttpResponse::BadRequest().body("error")
}
Ok(None) => HttpResponse::BadRequest().body("error"),
Err(e) => {
error!("Failed to search tx: {}", e);
HttpResponse::InternalServerError().body("error")
}
}
}
/// Holds a transaction that has already been parsed and validated outside the DB lock.
#[derive(Clone)]
struct ParsedTx {
txid: String,
wtxid: String,
ntxid: String,
raw_hex: String, // the original line
raw_hex: String,
locktime: String,
inputs: Vec<(String, String)>, // (in_txid, in_vout)
outputs: Vec<(usize, String, u64)>, // (idx, script_pubkey, amount_sat)
inputs: Vec<(String, String)>,
outputs: Vec<(usize, String, u64)>,
}
/// Parse all transactions from the request body **without** needing the DB lock.
/// Skips transactions that don't have a valid willexecutor output.
fn parse_request_transactions(
strbody: &str,
_req_time: i64,
@@ -576,7 +456,6 @@ fn parse_request_transactions(
let wtxid = tx.compute_wtxid();
let locktime = tx.lock_time.to_string();
// Collect inputs
let mut inputs: Vec<(String, String)> = Vec::with_capacity(tx.input.len());
for input in tx.input {
inputs.push((
@@ -585,7 +464,6 @@ fn parse_request_transactions(
));
}
// Collect outputs and find which one is ours + its amount
let mut outputs: Vec<(usize, String, u64)> = Vec::with_capacity(tx.output.len());
let mut found = false;
let mut our_address = String::new();
@@ -601,13 +479,25 @@ fn parse_request_transactions(
netconfig.network,
) {
Ok(addr) => addr.to_string(),
Err(_) => continue, // skip un-decodable outputs
Err(_) => continue,
};
let expected_ours = if netconfig.xpub {
if known_addresses.contains(&address) {
trace!(
"output {} address {} found in known_addresses (total: {})",
idx,
&address,
known_addresses.len()
);
address.clone()
} else {
trace!(
"output {} address {} NOT in known_addresses (total: {}), skipping",
idx,
&address,
known_addresses.len()
);
continue;
}
} else {
@@ -619,6 +509,11 @@ fn parse_request_transactions(
our_fees = amount;
found = true;
trace!("address and fees are correct {}: {}", our_address, our_fees);
} else if address == expected_ours {
trace!(
"output {} address matches but amount {} < fixed_fee {}, skipping",
idx, amount, netconfig.fixed_fee
);
}
}
@@ -679,16 +574,8 @@ async fn echo_push(
};
// ===== PHASE 1: parse all transactions WITHOUT the DB lock =====
let known_addresses: HashSet<String> = {
let db = match data.db.lock() {
Ok(g) => g,
Err(_p) => {
error!("DB mutex poisoned acquiring addresses in echo_push");
return HttpResponse::InternalServerError().body("error");
}
};
if netconfig.xpub {
match get_all_addresses_by_xpub(&db, &netconfig.address) {
let known_addresses: HashSet<String> = if netconfig.xpub {
match get_all_addresses_by_xpub(&data.db, &netconfig.address).await {
Ok(addrs) => addrs,
Err(e) => {
error!("Failed to load addresses from xpub: {}", e);
@@ -697,8 +584,7 @@ async fn echo_push(
}
} else {
HashSet::new()
}
}; // lock released here
};
// Parse all transactions (CPU-bound, no DB needed)
let parsed = parse_request_transactions(strbody, req_time, netconfig, &known_addresses);
@@ -709,134 +595,86 @@ async fn echo_push(
let all_txids: Vec<String> = parsed.iter().map(|(p, _, _)| p.txid.clone()).collect();
// ===== PHASE 2: check duplicates in a single batch query =====
let duplicates = {
let db = match data.db.lock() {
Ok(g) => g,
Err(_p) => {
error!("DB mutex poisoned in echo_push duplicate check");
return HttpResponse::InternalServerError().body("error");
}
};
match check_duplicate_txids(&db, &all_txids) {
let duplicates = match check_duplicate_txids(&data.db, &all_txids).await {
Ok(dups) => dups,
Err(e) => {
error!("Duplicate check failed: {}", e);
return HttpResponse::InternalServerError().body("error");
}
}
}; // lock released here
};
let all_present = all_txids.iter().all(|t| duplicates.contains(t));
if all_present {
return HttpResponse::Ok().body("already present");
}
// ===== PHASE 3: build insert statements and execute (single DB lock, minimal time) =====
{
let db = match data.db.lock() {
Ok(g) => g,
Err(_p) => {
error!("DB mutex poisoned in echo_push insert phase");
return HttpResponse::InternalServerError().body("error");
}
};
let sqltxshead = "INSERT INTO tbl_tx (txid, wtxid, ntxid, tx, locktime, reqid, network, our_address, our_fees)".to_string();
let mut sqltxs = String::new();
let sqlinpshead = "INSERT INTO tbl_inp (txid, in_txid, in_vout )".to_string();
let mut sqlinps = String::new();
let sqloutshead = "INSERT INTO tbl_out (txid, vout, script_pubkey, amount )".to_string();
let mut sqlouts = String::new();
let mut union_tx = true;
let mut union_inps = true;
let mut union_outs = true;
let mut ptx: Vec<(usize, Value)> = vec![];
let mut pinps: Vec<(usize, Value)> = vec![];
let mut pouts: Vec<(usize, Value)> = vec![];
let mut linenum = 1usize;
let mut lineinp = 1usize;
let mut lineout = 1usize;
// ===== PHASE 3: build insert data and execute (single transaction, minimal time) =====
let mut tx_data = Vec::new();
let mut inp_data = Vec::new();
let mut out_data = Vec::new();
for (parsed, our_address, our_fees) in &parsed {
if duplicates.contains(&parsed.txid) {
continue;
}
if !union_tx {
sqltxs.push_str(" UNION ALL");
} else {
union_tx = false;
}
sqltxs.push_str(" SELECT ?, ?, ?, ?, ?, ?, ?, ?, ?");
ptx.push((linenum, Value::String(parsed.txid.clone())));
ptx.push((linenum + 1, Value::String(parsed.wtxid.clone())));
ptx.push((linenum + 2, Value::String(parsed.ntxid.clone())));
ptx.push((linenum + 3, Value::String(parsed.raw_hex.clone())));
ptx.push((linenum + 4, Value::String(parsed.locktime.clone())));
ptx.push((linenum + 5, Value::String(req_time.to_string())));
ptx.push((linenum + 6, Value::String(netconfig.name.clone())));
ptx.push((linenum + 7, Value::String(our_address.clone())));
ptx.push((linenum + 8, Value::String(our_fees.to_string())));
linenum += 9;
tx_data.push(InsertTxData {
txid: parsed.txid.clone(),
wtxid: parsed.wtxid.clone(),
ntxid: parsed.ntxid.clone(),
raw_hex: parsed.raw_hex.clone(),
locktime: parsed.locktime.clone(),
reqid: req_time.to_string(),
network: netconfig.name.clone(),
our_address: our_address.clone(),
our_fees: our_fees.to_string(),
});
for (in_txid, in_vout) in &parsed.inputs {
if !union_inps {
sqlinps.push_str(" UNION ALL");
} else {
union_inps = false;
}
sqlinps.push_str(" SELECT ?, ?, ?");
pinps.push((lineinp, Value::String(parsed.txid.clone())));
pinps.push((lineinp + 1, Value::String(in_txid.clone())));
pinps.push((lineinp + 2, Value::String(in_vout.clone())));
lineinp += 3;
inp_data.push(InsertInpData {
txid: parsed.txid.clone(),
in_txid: in_txid.clone(),
in_vout: in_vout.clone(),
});
}
for (idx, script, amount) in &parsed.outputs {
if !union_outs {
sqlouts.push_str(" UNION ALL");
} else {
union_outs = false;
}
sqlouts.push_str(" SELECT ?, ?, ?, ?");
pouts.push((lineout, Value::String(parsed.txid.clone())));
pouts.push((
lineout + 1,
Value::Integer(i64::try_from(*idx).unwrap_or(-1)),
));
pouts.push((lineout + 2, Value::String(script.clone())));
pouts.push((
lineout + 3,
Value::Integer(i64::try_from(*amount).unwrap_or(0)),
));
lineout += 4;
out_data.push(InsertOutData {
txid: parsed.txid.clone(),
vout: i64::try_from(*idx).unwrap_or(-1),
script_pubkey: script.clone(),
amount: i64::try_from(*amount).unwrap_or(0),
});
}
}
if sqltxs.is_empty() {
if tx_data.is_empty() {
return HttpResponse::Ok().body("already present");
}
let sqltxs = format!("{}{};", sqltxshead, sqltxs);
let sqlinps = format!("{}{};", sqlinpshead, sqlinps);
let sqlouts = format!("{}{};", sqloutshead, sqlouts);
if let Err(err) = execute_insert(&db, sqltxs, ptx, sqlinps, pinps, sqlouts, pouts) {
if let Err(err) = bal_server::db::execute_insert(&data.db, &tx_data, &inp_data, &out_data).await
{
error!("execute_insert failed: {}", err);
return HttpResponse::BadRequest().body("error");
}
} // lock released
HttpResponse::Ok().body("thx")
}
fn parse_env(data: &MyConfig) -> MyConfig {
let mut cfg = data.clone();
if let Ok(value) = env::var("BAL_SERVER_DB_BACKEND") {
debug!("BAL_SERVER_DB_BACKEND: {}", value);
cfg.db_backend = value;
}
if let Ok(value) = env::var("BAL_SERVER_DB_FILE") {
debug!("BAL_SERVER_DB_FILE: {}", value);
cfg.db_file = value;
}
if let Ok(value) = env::var("BAL_SERVER_PG_DSN") {
debug!("BAL_SERVER_PG_DSN: {}", value);
cfg.pg_dsn = value;
}
if let Ok(value) = env::var("BAL_SERVER_BIND_ADDRESS") {
debug!("BAL_SERVER_BIND_ADDRESS: {}", value);
cfg.bind_address = value;
@@ -889,10 +727,10 @@ fn parse_env_netconfig(cfg: &mut MyConfig, chain: &str) {
}
}
fn init_network(db: &Connection, cfg: &MyConfig) {
async fn init_network(pool: &DatabasePool, cfg: &MyConfig) {
for network in NETWORKS {
let netconfig = cfg.get_net_config(network);
insert_xpub(db, &netconfig.name.to_string(), &netconfig.address);
insert_xpub(pool, &netconfig.name.to_string(), &netconfig.address).await;
}
}
@@ -903,32 +741,44 @@ async fn main() -> std::io::Result<()> {
let actix_cfg = parse_actix_config();
let cfg = parse_env(&cfg);
let db = match open_db(&cfg.db_file) {
Ok(c) => c,
let connection_string = match cfg.db_backend.as_str() {
"sqlite" => cfg.db_file.clone(),
"postgresql" => cfg.pg_dsn.clone(),
other => {
return Err(std::io::Error::other(format!(
"Unknown DB backend: {}",
other
)));
}
};
let db = match open_database(&cfg.db_backend, &connection_string).await {
Ok(pool) => pool,
Err(e) => {
return Err(std::io::Error::other(e));
}
};
// Create database tables
create_database(&db);
create_database(&db)
.await
.map_err(|e| std::io::Error::other(format!("Failed to create database: {}", e)))?;
// Initialize networks
init_network(&db, &cfg);
init_network(&db, &cfg).await;
let data = web::Data::new(AppState {
db: Mutex::new(db),
db,
cfg: cfg.clone(),
});
let bind_address = data.cfg.bind_address.clone();
let bind_port = data.cfg.bind_port;
// Use a single global rate limiter with the most conservative settings (1 req/sec)
// Per-endpoint rate limiting requires advanced configuration with explicit types
let governor_conf = GovernorConfigBuilder::const_default()
.seconds_per_request(actix_cfg.rate_limit_pushtxs.0) // Most restrictive: 1 req/sec
.burst_size(actix_cfg.rate_limit_pushtxs.1) // Burst: 3
.seconds_per_request(actix_cfg.rate_limit_pushtxs.0)
.burst_size(actix_cfg.rate_limit_pushtxs.1)
.finish()
.unwrap();

412
src/db.rs
View File

@@ -1,412 +0,0 @@
use log::{error, info, trace, warn};
use sqlite::{Connection, Error, State, Value};
use std::collections::HashSet;
use std::path::Path;
use std::thread;
use std::time::Duration;
/// Check which txids are already present in the database in a single batch query.
/// Returns a HashSet of txids that already exist (duplicates).
/// This is O(1) per query regardless of the number of txids, replacing the N+1 pattern.
pub fn check_duplicate_txids(db: &Connection, txids: &[String]) -> Result<HashSet<String>, Error> {
if txids.is_empty() {
return Ok(HashSet::new());
}
// Build a single query with all txids using IN clause placeholders
// SQLite supports up to 1000 parameters per statement, so we chunk for safety
let mut duplicates = HashSet::new();
let chunk_size = 500; // Safe chunk size for SQLite parameters
for chunk in txids.chunks(chunk_size) {
let placeholders = chunk.iter().map(|_| "?").collect::<Vec<_>>().join(",");
let sql = format!("SELECT txid FROM tbl_tx WHERE txid IN ({})", placeholders);
let mut stmt = db.prepare(sql)?;
for (i, txid) in chunk.iter().enumerate() {
stmt.bind((i + 1, Value::String(txid.clone())))?;
}
while let Ok(State::Row) = stmt.next() {
if let Ok(txid) = stmt.read::<String, _>("txid") {
duplicates.insert(txid);
}
}
}
Ok(duplicates)
}
/// Validates and opens the SQLite database, enforcing security best practices:
/// - Path must not contain `..` (directory traversal).
/// - Absolute paths must not target known system directories.
/// - If the file exists, it must be a regular file (not a symlink or device).
/// - WAL journal mode is enabled for safe concurrent access.
/// - Synchronous is set to NORMAL for performance with safety.
///
/// Returns `Err` on validation failure or open error to prevent panics.
pub fn open_db(path: &str) -> Result<Connection, String> {
let p = Path::new(path);
// Prevent directory traversal
for component in p.components() {
if component == std::path::Component::ParentDir {
return Err("Database path may not contain '..'".to_string());
}
}
// If absolute, block known sensitive system directories
if p.is_absolute() {
let path_str = p.to_str().unwrap_or("");
let forbidden = [
"/etc", "/proc", "/sys", "/dev", "/usr", "/bin", "/sbin", "/lib", "/opt",
];
for prefix in &forbidden {
if path_str.starts_with(prefix) {
return Err(format!(
"Absolute database path under {} is forbidden",
prefix
));
}
}
}
// If file exists, must be a regular file (not a symlink, device, etc.)
if p.exists() {
if p.is_symlink() {
return Err("Database path must not be a symlink".to_string());
}
let metadata = std::fs::metadata(p)
.map_err(|e| format!("Cannot access database file metadata: {}", e))?;
if !metadata.is_file() {
return Err(
"Database path must point to a regular file, not a directory or device".to_string(),
);
}
}
let conn = sqlite::open(path).map_err(|e| format!("Failed to open SQLite database: {}", e))?;
// Set busy timeout BEFORE WAL mode so SQLite waits instead of failing immediately.
// This handles the race where two processes (server + pusher) open the same DB
// and both try to enable WAL mode concurrently.
conn.execute("PRAGMA busy_timeout = 5000;")
.map_err(|e| format!("Failed to set busy_timeout: {}", e))?;
// Retry WAL mode up to 5 times (handles concurrent open from bal-pusher).
let mut wal_ok = false;
for attempt in 0..5 {
match conn.execute("PRAGMA journal_mode = WAL;") {
Ok(_) => {
wal_ok = true;
break;
}
Err(e) => {
warn!(
"WAL mode attempt {}/5 failed: {}, retrying in 100ms...",
attempt + 1,
e
);
thread::sleep(Duration::from_millis(100));
}
}
}
if !wal_ok {
// WAL might already be enabled by another process; this is not fatal.
warn!("Could not set WAL mode after retries — may already be enabled by another process");
}
conn.execute("PRAGMA synchronous = NORMAL;")
.map_err(|e| format!("Failed to set synchronous NORMAL: {}", e))?;
Ok(conn)
}
/// Loads all known addresses for a given xpub into a HashSet for fast
/// in-memory lookup during transaction validation (replaces N+1 query).
pub fn get_all_addresses_by_xpub(db: &Connection, xpub: &str) -> Result<HashSet<String>, Error> {
let mut stmt = db.prepare(
"SELECT a.address FROM tbl_address a JOIN tbl_xpub x ON a.xpub = x.id WHERE x.xpub = ?",
)?;
stmt.bind((1, Value::String(xpub.to_string())))?;
let mut addresses = HashSet::new();
while let Ok(State::Row) = stmt.next() {
match stmt.read::<String, _>("address") {
Ok(addr) => {
addresses.insert(addr);
}
Err(e) => {
error!("Failed to read address column: {}", e);
}
}
}
Ok(addresses)
}
pub fn create_database(db: &Connection) {
info!("database sanity check");
let _ = db.execute("CREATE TABLE IF NOT EXISTS tbl_tx (txid PRIMARY KEY, date_creation TIMESTAMP DEFAULT CURRENT_TIMESTAMP, date_update TIMESTAMP DEFAULT CURRENT_TIMESTAMP, wtxid, ntxid, tx, locktime integer, network, network_fees, reqid, our_fees, our_address, status integer DEFAULT 0);");
let _ = db.execute("ALTER TABLE tbl_tx ADD COLUMN push_err TEXT");
let _ = db.execute("CREATE TABLE IF NOT EXISTS tbl_inp(id, txid, in_txid, in_vout);");
let _ = db.execute("CREATE UNIQUE INDEX ON tbl_inp(txid,in_txid,in_vout);");
let _ =
db.execute("CREATE TABLE IF NOT EXISTS tbl_out(id, txid, script_pubkey, amount, vout);");
let _ = db.execute("CREATE UNIQUE INDEX ON tbl_out(txid, script_pubkey, amount, vout);");
let _ = db.execute("CREATE TABLE IF NOT EXISTS tbl_xpub (id INTEGER PRIMARY KEY , network TEXT, xpub TEXT, date_create TIMESTAMP DEFAULT CURRENT_TIMESTAMP,path_idx INTEGER DEFAULT -1);");
let _ = db.execute("CREATE UNIQUE INDEX idx_xpub ON tbl_xpub (network, xpub)");
let _ = db.execute("CREATE TABLE IF NOT EXISTS tbl_address (address TEXT PRIMARY_KEY, path TEXT NOT NULL, date_create TIMESTAMP DEFAULT CURRENT_TIMESTAMP, xpub INTEGER,remote_address TEXT);");
let _ = db.execute("CREATE TABLE IF NOT EXISTS tbl_stats (report_date TEXT, chain TEXT, totals INTEGER, waiting INTEGER, sent INTEGER, failed INTEGER, waiting_profit INTEGER, sent_profit INTEGER, missed_profit INTEGER, unique_inputs INTEGER);");
// UNIQUE index required for ON CONFLICT(chain) DO UPDATE in calculate_stats
let _ = db.execute("DROP INDEX IF EXISTS idx_stats_chain;");
let _ = db.execute("CREATE UNIQUE INDEX IF NOT EXISTS idx_stats_chain ON tbl_stats(chain);");
let _ = db.execute("UPDATE tbl_tx set network='bitcoin' where network='mainnet';");
}
/*
pub fn get_xpub_id(db: &Connection, network: &String, xpub: &String) -> Option<i64>{
let mut stmt = db.prepare("SELECT * FROM tbl_xpub where network = ? and xpub = ?;").unwrap();
let _ = stmt.bind((1,Value::String(network.to_string()))).unwrap();
let _ = stmt.bind((2,Value::String(xpub.to_string()))).unwrap();
if let Ok(State::Row) = stmt.next(){
return Some(stmt.read::<i64, _>("id").unwrap());
} else {
return None;
}
}
*/
pub fn insert_xpub(db: &Connection, network: &str, xpub: &str) {
if !xpub.is_empty() {
trace!("going to insert: {} xpub:{}", network, xpub);
let mut stmt =
match db.prepare("INSERT OR IGNORE INTO tbl_xpub(network,xpub) VALUES(?, ?);") {
Ok(s) => s,
Err(e) => {
error!("Failed to prepare xpub insert statement: {}", e);
return;
}
};
if let Err(e) = stmt.bind((1, Value::String(network.to_string()))) {
error!("Failed to bind network parameter for xpub insert: {}", e);
return;
}
if let Err(e) = stmt.bind((2, Value::String(xpub.to_string()))) {
error!("Failed to bind xpub parameter: {}", e);
return;
}
if let Err(e) = stmt.next() {
error!("Failed to insert xpub: {}", e);
}
}
}
pub fn get_last_used_address_by_ip(
db: &Connection,
network: &String,
xpub: &String,
address: &String,
) -> Option<String> {
let mut stmt = match db.prepare("SELECT tbl_address.address FROM tbl_xpub join tbl_address on(tbl_xpub.id = tbl_address.xpub) where tbl_xpub.network = ? and tbl_address.remote_address = ? and tbl_xpub.xpub = ? ORDER BY tbl_address.date_create DESC LIMIT 1;") {
Ok(s) => s,
Err(e) => {
error!("Failed to prepare address query: {}", e);
return None;
}
};
if let Err(e) = stmt.bind((1, Value::String(network.to_string()))) {
error!("Failed to bind network parameter: {}", e);
return None;
}
if let Err(e) = stmt.bind((2, Value::String(address.to_string()))) {
error!("Failed to bind address parameter: {}", e);
return None;
}
if let Err(e) = stmt.bind((3, Value::String(xpub.to_string()))) {
error!("Failed to bind xpub parameter: {}", e);
return None;
}
if let Ok(State::Row) = stmt.next() {
match stmt.read::<String, _>("address") {
Ok(addr) => Some(addr),
Err(e) => {
error!("Failed to read address column: {}", e);
None
}
}
} else {
None
}
}
pub fn get_next_address_index(db: &Connection, network: &String, xpub: &String) -> (i64, i64) {
let mut stmt = match db.prepare("UPDATE tbl_xpub SET path_idx = path_idx + 1 WHERE network = ? and xpub= ? RETURNING path_idx,id;") {
Ok(s) => s,
Err(e) => {
error!("Failed to prepare xpub index update: {}", e);
return (0, 0);
}
};
if let Err(e) = stmt.bind((1, Value::String(network.to_string()))) {
error!("Failed to bind network parameter: {}", e);
return (0, 0);
}
if let Err(e) = stmt.bind((2, Value::String(xpub.to_string()))) {
error!("Failed to bind xpub parameter: {}", e);
return (0, 0);
}
match stmt.next() {
Ok(State::Row) => match stmt.read::<i64, _>("path_idx") {
Ok(next) => match stmt.read::<i64, _>("id") {
Ok(id) => (id, next),
Err(e) => {
error!("Failed to read id column: {}", e);
(0, 0)
}
},
Err(e) => {
error!("Failed to read path_idx column: {}", e);
(0, 0)
}
},
Err(e) => {
error!("Failed to execute xpub index update: {}", e);
(0, 0)
}
Ok(State::Done) => (0, 0),
}
}
pub fn save_new_address(
db: &Connection,
xpub: i64,
address: &String,
path: &String,
remote_addr: &String,
) {
let mut stmt = match db.prepare(
"INSERT INTO tbl_address(address,path,xpub,remote_address) VALUES(?,?,?,?);
",
) {
Ok(s) => s,
Err(e) => {
error!("Failed to prepare address insert statement: {}", e);
return;
}
};
if let Err(e) = stmt.bind((1, Value::String(address.to_string()))) {
error!("Failed to bind address parameter: {}", e);
return;
}
if let Err(e) = stmt.bind((2, Value::String(path.to_string()))) {
error!("Failed to bind path parameter: {}", e);
return;
}
if let Err(e) = stmt.bind((3, Value::Integer(xpub))) {
error!("Failed to bind xpub parameter: {}", e);
return;
}
if let Err(e) = stmt.bind((4, Value::String(remote_addr.to_string()))) {
error!("Failed to bind remote_addr parameter: {}", e);
return;
}
if let Err(e) = stmt.next() {
error!("Failed to insert address: {}", e);
}
}
pub fn execute_insert(
db: &Connection,
sqltxs: String,
ptx: Vec<(usize, Value)>,
sqlinp: String,
pinp: Vec<(usize, Value)>,
sqlout: String,
pout: Vec<(usize, Value)>,
) -> Result<(), Error> {
let _ = db.execute("BEGIN TRANSACTION");
let mut stmt = match db.prepare(sqltxs.as_str()) {
Ok(s) => s,
Err(err) => {
error!("error preparing sqltxs: {}", err);
let _ = db.execute("ROLLBACK");
return Err(err);
}
};
if let Err(err) = stmt.bind::<&[(_, Value)]>(&ptx[..]) {
error!("error binding transaction parameters: {}", err);
let _ = db.execute("ROLLBACK");
return Err(err);
}
if let Err(err) = stmt.next() {
error!("error inserting transactions {}", err);
let _ = db.execute("ROLLBACK");
} else {
let mut stmt = match db.prepare(sqlinp.as_str()) {
Ok(s) => s,
Err(err) => {
error!("error preparing sqlinp: {}", err);
let _ = db.execute("ROLLBACK");
return Err(err);
}
};
if let Err(err) = stmt.bind::<&[(_, Value)]>(&pinp[..]) {
error!("error binding inputs parameters {}", err);
let _ = db.execute("ROLLBACK");
return Err(err);
}
if let Err(err) = stmt.next() {
error!("error inserting inputs {}", err);
let _ = db.execute("ROLLBACK");
return Err(err);
} else {
let mut stmt = match db.prepare(sqlout.as_str()) {
Ok(s) => s,
Err(err) => {
error!("error preparing sqlout: {}", err);
let _ = db.execute("ROLLBACK");
return Err(err);
}
};
if let Err(err) = stmt.bind::<&[(_, Value)]>(&pout[..]) {
error!("error binding outs parameters {}", err);
let _ = db.execute("ROLLBACK");
return Err(err);
}
if let Err(err) = stmt.next() {
error!("error inserting outs {}", err);
let _ = db.execute("ROLLBACK");
return Err(err);
}
}
}
let _ = db.execute("COMMIT");
Ok(())
}
pub fn get_total_transaction_number(db: Connection, network: &String) -> Result<i64, Error> {
let mut stmt = db
.prepare("SELECT COUNT(*) as total_number FROM tbl_tx where network = ?;")
.map_err(|e| {
error!("Failed to prepare statement: {}", e);
e
})?;
if let Err(e) = stmt.bind((1, Value::String(network.to_string()))) {
error!("Failed to bind network parameter: {}", e);
return Err(e);
}
match stmt.next() {
Ok(State::Row) => match stmt.read::<i64, _>("total_number") {
Ok(val) => Ok(val),
Err(e) => {
error!("Failed to read total_number column: {}", e);
Err(e)
}
},
Ok(sqlite::State::Done) => Ok(0),
Err(err) => {
error!("Failed to execute query: {}", err);
Err(err)
}
}
}

868
src/db/mod.rs Normal file
View File

@@ -0,0 +1,868 @@
pub mod schema;
use log::{error, info, trace};
use sqlx::Row;
use std::collections::HashSet;
use std::path::Path;
#[derive(Clone)]
pub enum DatabasePool {
SQLite(sqlx::SqlitePool),
PostgreSQL(sqlx::PgPool),
}
fn validate_sqlite_path(dsn: &str) -> Result<(), String> {
// Extract file path from DSN formats like "sqlite:path" or "file:path?mode=rwc"
let path_str = if let Some(rest) = dsn.strip_prefix("sqlite:") {
rest.split('?').next().unwrap_or(rest)
} else if let Some(rest) = dsn.strip_prefix("file:") {
rest.split('?').next().unwrap_or(rest)
} else {
dsn
};
// Skip validation for in-memory databases
if path_str == ":memory:" || path_str.is_empty() {
return Ok(());
}
let p = Path::new(path_str);
// Prevent directory traversal
for component in p.components() {
if component == std::path::Component::ParentDir {
return Err("Database path may not contain '..'".to_string());
}
}
// If absolute, block known sensitive system directories
if p.is_absolute() {
let forbidden = [
"/etc", "/proc", "/sys", "/dev", "/usr", "/bin", "/sbin", "/lib", "/opt",
];
for prefix in &forbidden {
if path_str.starts_with(prefix) {
return Err(format!(
"Absolute database path under {} is forbidden",
prefix
));
}
}
}
// If file exists, must be a regular file (not a symlink, device, etc.)
if p.exists() {
if p.is_symlink() {
return Err("Database path must not be a symlink".to_string());
}
let metadata = std::fs::metadata(p)
.map_err(|e| format!("Cannot access database file metadata: {}", e))?;
if !metadata.is_file() {
return Err(
"Database path must point to a regular file, not a directory or device".to_string(),
);
}
}
Ok(())
}
pub async fn open_database(backend: &str, connection_string: &str) -> Result<DatabasePool, String> {
match backend {
"sqlite" => {
validate_sqlite_path(connection_string)?;
let pool = sqlx::sqlite::SqlitePoolOptions::new()
.max_connections(1)
.connect(connection_string)
.await
.map_err(|e| format!("Failed to open SQLite database: {}", e))?;
sqlx::query("PRAGMA journal_mode=WAL")
.execute(&pool)
.await
.map_err(|e| format!("Failed to set WAL mode: {}", e))?;
sqlx::query("PRAGMA busy_timeout=5000")
.execute(&pool)
.await
.map_err(|e| format!("Failed to set busy_timeout: {}", e))?;
Ok(DatabasePool::SQLite(pool))
}
"postgresql" => {
let pool = sqlx::postgres::PgPoolOptions::new()
.max_connections(10)
.connect(connection_string)
.await
.map_err(|e| format!("Failed to open PostgreSQL database: {}", e))?;
Ok(DatabasePool::PostgreSQL(pool))
}
other => Err(format!("Unknown database backend: {}", other)),
}
}
pub async fn create_database(pool: &DatabasePool) -> Result<(), sqlx::Error> {
info!("database sanity check");
match pool {
DatabasePool::SQLite(p) => schema::create_sqlite_schema(p).await,
DatabasePool::PostgreSQL(p) => schema::create_pg_schema(p).await,
}
}
pub async fn check_duplicate_txids(
pool: &DatabasePool,
txids: &[String],
) -> Result<HashSet<String>, sqlx::Error> {
if txids.is_empty() {
return Ok(HashSet::new());
}
let mut duplicates = HashSet::new();
let chunk_size = 500;
for chunk in txids.chunks(chunk_size) {
match pool {
DatabasePool::SQLite(p) => {
let placeholders: Vec<String> = chunk.iter().map(|_| "?".to_string()).collect();
let sql = format!(
"SELECT txid FROM tbl_tx WHERE txid IN ({})",
placeholders.join(",")
);
let mut query = sqlx::query(&sql);
for txid in chunk {
query = query.bind(txid);
}
let rows = query.fetch_all(p).await?;
for row in rows {
if let Ok(txid) = row.try_get::<String, _>("txid") {
duplicates.insert(txid);
}
}
}
DatabasePool::PostgreSQL(p) => {
let placeholders: Vec<String> = chunk
.iter()
.enumerate()
.map(|(i, _)| format!("${}", i + 1))
.collect();
let sql = format!(
"SELECT txid FROM tbl_tx WHERE txid IN ({})",
placeholders.join(",")
);
let mut query = sqlx::query(&sql);
for txid in chunk {
query = query.bind(txid);
}
let rows = query.fetch_all(p).await?;
for row in rows {
if let Ok(txid) = row.try_get::<String, _>("txid") {
duplicates.insert(txid);
}
}
}
}
}
Ok(duplicates)
}
pub async fn get_all_addresses_by_xpub(
pool: &DatabasePool,
xpub: &str,
) -> Result<HashSet<String>, sqlx::Error> {
let mut addresses = HashSet::new();
trace!("get_all_addresses_by_xpub: querying for xpub={}", xpub);
match pool {
DatabasePool::SQLite(p) => {
let rows = sqlx::query(
"SELECT a.address FROM tbl_address a JOIN tbl_xpub x ON a.xpub = x.id WHERE x.xpub = ?",
)
.bind(xpub)
.fetch_all(p)
.await?;
trace!(
"get_all_addresses_by_xpub: SQLite returned {} rows",
rows.len()
);
for row in rows {
if let Ok(addr) = row.try_get::<String, _>("address") {
trace!("get_all_addresses_by_xpub: address={}", &addr);
addresses.insert(addr);
}
}
}
DatabasePool::PostgreSQL(p) => {
let rows = sqlx::query(
"SELECT a.address FROM tbl_address a JOIN tbl_xpub x ON a.xpub = x.id WHERE x.xpub = $1",
)
.bind(xpub)
.fetch_all(p)
.await?;
trace!(
"get_all_addresses_by_xpub: PostgreSQL returned {} rows",
rows.len()
);
for row in rows {
if let Ok(addr) = row.try_get::<String, _>("address") {
trace!("get_all_addresses_by_xpub: address={}", &addr);
addresses.insert(addr);
}
}
}
}
trace!(
"get_all_addresses_by_xpub: returning {} addresses",
addresses.len()
);
Ok(addresses)
}
pub async fn insert_xpub(pool: &DatabasePool, network: &str, xpub: &str) {
if xpub.is_empty() {
return;
}
trace!("going to insert: {} xpub:{}", network, xpub);
match pool {
DatabasePool::SQLite(p) => {
if let Err(e) =
sqlx::query("INSERT OR IGNORE INTO tbl_xpub(network, xpub) VALUES(?, ?)")
.bind(network)
.bind(xpub)
.execute(p)
.await
{
error!("Failed to insert xpub: {}", e);
}
}
DatabasePool::PostgreSQL(p) => {
if let Err(e) = sqlx::query(
"INSERT INTO tbl_xpub(network, xpub) VALUES($1, $2) ON CONFLICT DO NOTHING",
)
.bind(network)
.bind(xpub)
.execute(p)
.await
{
error!("Failed to insert xpub: {}", e);
}
}
}
}
pub async fn get_last_used_address_by_ip(
pool: &DatabasePool,
network: &str,
xpub: &str,
address: &str,
) -> Option<String> {
match pool {
DatabasePool::SQLite(p) => {
let result = sqlx::query(
"SELECT tbl_address.address FROM tbl_xpub JOIN tbl_address ON(tbl_xpub.id = tbl_address.xpub) WHERE tbl_xpub.network = ? AND tbl_address.remote_address = ? AND tbl_xpub.xpub = ? ORDER BY tbl_address.date_create DESC LIMIT 1",
)
.bind(network)
.bind(address)
.bind(xpub)
.fetch_optional(p)
.await;
match result {
Ok(Some(row)) => row.try_get::<String, _>("address").ok(),
Ok(None) => None,
Err(e) => {
error!("Failed to query last used address: {}", e);
None
}
}
}
DatabasePool::PostgreSQL(p) => {
let result = sqlx::query(
"SELECT tbl_address.address FROM tbl_xpub JOIN tbl_address ON(tbl_xpub.id = tbl_address.xpub) WHERE tbl_xpub.network = $1 AND tbl_address.remote_address = $2 AND tbl_xpub.xpub = $3 ORDER BY tbl_address.date_create DESC LIMIT 1",
)
.bind(network)
.bind(address)
.bind(xpub)
.fetch_optional(p)
.await;
match result {
Ok(Some(row)) => row.try_get::<String, _>("address").ok(),
Ok(None) => None,
Err(e) => {
error!("Failed to query last used address: {}", e);
None
}
}
}
}
}
pub async fn get_next_address_index(pool: &DatabasePool, network: &str, xpub: &str) -> (i64, i64) {
match pool {
DatabasePool::SQLite(p) => {
let result = sqlx::query(
"UPDATE tbl_xpub SET path_idx = path_idx + 1 WHERE network = ? AND xpub = ? RETURNING path_idx, id",
)
.bind(network)
.bind(xpub)
.fetch_optional(p)
.await;
match result {
Ok(Some(row)) => {
let idx = row.try_get::<i64, _>("path_idx").unwrap_or(0);
let id = row.try_get::<i64, _>("id").unwrap_or(0);
(id, idx)
}
Ok(None) => (0, 0),
Err(e) => {
error!("Failed to get next address index: {}", e);
(0, 0)
}
}
}
DatabasePool::PostgreSQL(p) => {
let result = sqlx::query(
"UPDATE tbl_xpub SET path_idx = path_idx + 1 WHERE network = $1 AND xpub = $2 RETURNING path_idx, id",
)
.bind(network)
.bind(xpub)
.fetch_optional(p)
.await;
match result {
Ok(Some(row)) => {
let idx = row.try_get::<i32, _>("path_idx").unwrap_or(0) as i64;
let id = row.try_get::<i32, _>("id").unwrap_or(0) as i64;
(id, idx)
}
Ok(None) => (0, 0),
Err(e) => {
error!("Failed to get next address index: {}", e);
(0, 0)
}
}
}
}
}
pub async fn save_new_address(
pool: &DatabasePool,
xpub: i64,
address: &str,
path: &str,
remote_addr: &str,
) {
match pool {
DatabasePool::SQLite(p) => {
if let Err(e) = sqlx::query(
"INSERT INTO tbl_address(address, path, xpub, remote_address) VALUES(?, ?, ?, ?)",
)
.bind(address)
.bind(path)
.bind(xpub)
.bind(remote_addr)
.execute(p)
.await
{
error!("Failed to save address: {}", e);
}
}
DatabasePool::PostgreSQL(p) => {
if let Err(e) = sqlx::query(
"INSERT INTO tbl_address(address, path, xpub, remote_address) VALUES($1, $2, $3, $4)",
)
.bind(address)
.bind(path)
.bind(xpub as i32)
.bind(remote_addr)
.execute(p)
.await
{
error!("Failed to save address: {}", e);
}
}
}
}
#[derive(Debug, Clone)]
pub struct InsertTxData {
pub txid: String,
pub wtxid: String,
pub ntxid: String,
pub raw_hex: String,
pub locktime: String,
pub reqid: String,
pub network: String,
pub our_address: String,
pub our_fees: String,
}
#[derive(Debug, Clone)]
pub struct InsertInpData {
pub txid: String,
pub in_txid: String,
pub in_vout: String,
}
#[derive(Debug, Clone)]
pub struct InsertOutData {
pub txid: String,
pub vout: i64,
pub script_pubkey: String,
pub amount: i64,
}
pub async fn execute_insert(
pool: &DatabasePool,
txs: &[InsertTxData],
inps: &[InsertInpData],
outs: &[InsertOutData],
) -> Result<(), sqlx::Error> {
match pool {
DatabasePool::SQLite(p) => {
let mut tx = p.begin().await?;
for item in txs {
sqlx::query(
"INSERT INTO tbl_tx (txid, wtxid, ntxid, tx, locktime, reqid, network, our_address, our_fees) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)"
)
.bind(&item.txid)
.bind(&item.wtxid)
.bind(&item.ntxid)
.bind(&item.raw_hex)
.bind(&item.locktime)
.bind(&item.reqid)
.bind(&item.network)
.bind(&item.our_address)
.bind(&item.our_fees)
.execute(&mut *tx)
.await?;
}
for item in inps {
sqlx::query("INSERT INTO tbl_inp (txid, in_txid, in_vout) VALUES (?, ?, ?)")
.bind(&item.txid)
.bind(&item.in_txid)
.bind(&item.in_vout)
.execute(&mut *tx)
.await?;
}
for item in outs {
sqlx::query(
"INSERT INTO tbl_out (txid, vout, script_pubkey, amount) VALUES (?, ?, ?, ?)",
)
.bind(&item.txid)
.bind(item.vout)
.bind(&item.script_pubkey)
.bind(item.amount)
.execute(&mut *tx)
.await?;
}
tx.commit().await?;
}
DatabasePool::PostgreSQL(p) => {
let mut tx = p.begin().await?;
for item in txs {
let locktime_i64: i64 = item.locktime.parse().unwrap_or(0);
sqlx::query(
"INSERT INTO tbl_tx (txid, wtxid, ntxid, tx, locktime, reqid, network, our_address, our_fees) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9)"
)
.bind(&item.txid)
.bind(&item.wtxid)
.bind(&item.ntxid)
.bind(&item.raw_hex)
.bind(locktime_i64)
.bind(&item.reqid)
.bind(&item.network)
.bind(&item.our_address)
.bind(&item.our_fees)
.execute(&mut *tx)
.await?;
}
for item in inps {
let in_vout_i32: i32 = item.in_vout.parse().unwrap_or(0);
sqlx::query("INSERT INTO tbl_inp (txid, in_txid, in_vout) VALUES ($1, $2, $3)")
.bind(&item.txid)
.bind(&item.in_txid)
.bind(in_vout_i32)
.execute(&mut *tx)
.await?;
}
for item in outs {
sqlx::query(
"INSERT INTO tbl_out (txid, vout, script_pubkey, amount) VALUES ($1, $2, $3, $4)",
)
.bind(&item.txid)
.bind(item.vout as i32)
.bind(&item.script_pubkey)
.bind(item.amount.to_string())
.execute(&mut *tx)
.await?;
}
tx.commit().await?;
}
}
Ok(())
}
pub async fn get_total_transaction_number(
pool: &DatabasePool,
network: &str,
) -> Result<i64, sqlx::Error> {
match pool {
DatabasePool::SQLite(p) => {
let row = sqlx::query("SELECT COUNT(*) as total_number FROM tbl_tx WHERE network = ?")
.bind(network)
.fetch_one(p)
.await?;
Ok(row.try_get::<i64, _>("total_number").unwrap_or(0))
}
DatabasePool::PostgreSQL(p) => {
let row = sqlx::query("SELECT COUNT(*) as total_number FROM tbl_tx WHERE network = $1")
.bind(network)
.fetch_one(p)
.await?;
Ok(row.try_get::<i64, _>("total_number").unwrap_or(0))
}
}
}
pub async fn update_tx_status(
pool: &DatabasePool,
txid: &str,
status: i32,
push_err: Option<&str>,
) -> Result<(), sqlx::Error> {
match pool {
DatabasePool::SQLite(p) => {
if let Some(err) = push_err {
sqlx::query("UPDATE tbl_tx SET status = ?, push_err = ? WHERE txid = ?")
.bind(status)
.bind(err)
.bind(txid)
.execute(p)
.await?;
} else {
sqlx::query("UPDATE tbl_tx SET status = ? WHERE txid = ?")
.bind(status)
.bind(txid)
.execute(p)
.await?;
}
}
DatabasePool::PostgreSQL(p) => {
if let Some(err) = push_err {
sqlx::query("UPDATE tbl_tx SET status = $1, push_err = $2 WHERE txid = $3")
.bind(status)
.bind(err)
.bind(txid)
.execute(p)
.await?;
} else {
sqlx::query("UPDATE tbl_tx SET status = $1 WHERE txid = $2")
.bind(status)
.bind(txid)
.execute(p)
.await?;
}
}
}
Ok(())
}
#[derive(Debug, Clone)]
pub struct TxRow {
pub txid: String,
pub tx: String,
pub locktime: i64,
pub network: String,
pub status: i64,
}
pub async fn get_pending_txs(
pool: &DatabasePool,
network: &str,
locktime_threshold: i64,
bestblock_height: i64,
bestblock_time: i64,
) -> Result<Vec<TxRow>, sqlx::Error> {
let mut results = Vec::new();
match pool {
DatabasePool::SQLite(p) => {
let rows = sqlx::query(
"SELECT txid, tx, locktime, network, status FROM tbl_tx WHERE network = ? AND status = 0 AND (locktime < ? OR (locktime > ? AND locktime < ?))",
)
.bind(network)
.bind(bestblock_height)
.bind(locktime_threshold)
.bind(bestblock_time)
.fetch_all(p)
.await?;
for row in rows {
results.push(TxRow {
txid: row.try_get("txid")?,
tx: row.try_get("tx")?,
locktime: row.try_get("locktime")?,
network: row.try_get("network")?,
status: row.try_get("status")?,
});
}
}
DatabasePool::PostgreSQL(p) => {
let rows = sqlx::query(
"SELECT txid, tx, locktime, network, status FROM tbl_tx WHERE network = $1 AND status = 0 AND (locktime < $2 OR (locktime > $3 AND locktime < $4))",
)
.bind(network)
.bind(bestblock_height as i32)
.bind(locktime_threshold as i32)
.bind(bestblock_time as i32)
.fetch_all(p)
.await?;
for row in rows {
results.push(TxRow {
txid: row.try_get("txid")?,
tx: row.try_get("tx")?,
locktime: row.try_get::<i32, _>("locktime").unwrap_or(0) as i64,
network: row.try_get("network")?,
status: row.try_get::<i32, _>("status").unwrap_or(0) as i64,
});
}
}
}
Ok(results)
}
#[derive(Debug, Clone)]
pub struct StatsRow {
pub report_date: String,
pub chain: String,
pub totals: i64,
pub waiting: i64,
pub sent: i64,
pub failed: i64,
pub waiting_profit: i64,
pub sent_profit: i64,
pub missed_profit: i64,
pub unique_inputs: i64,
}
pub async fn get_stats(pool: &DatabasePool, chain: &str) -> Result<Vec<StatsRow>, sqlx::Error> {
let mut results = Vec::new();
match pool {
DatabasePool::SQLite(p) => {
let rows = sqlx::query(
"SELECT report_date, chain, totals, waiting, sent, failed, waiting_profit, sent_profit, missed_profit, unique_inputs FROM tbl_stats WHERE chain = ?",
)
.bind(chain)
.fetch_all(p)
.await?;
for row in rows {
results.push(StatsRow {
report_date: row.try_get("report_date").unwrap_or_default(),
chain: row.try_get("chain").unwrap_or_default(),
totals: row.try_get("totals").unwrap_or(0),
waiting: row.try_get("waiting").unwrap_or(0),
sent: row.try_get("sent").unwrap_or(0),
failed: row.try_get("failed").unwrap_or(0),
waiting_profit: row.try_get("waiting_profit").unwrap_or(0),
sent_profit: row.try_get("sent_profit").unwrap_or(0),
missed_profit: row.try_get("missed_profit").unwrap_or(0),
unique_inputs: row.try_get("unique_inputs").unwrap_or(0),
});
}
}
DatabasePool::PostgreSQL(p) => {
let rows = sqlx::query(
"SELECT report_date, chain, totals, waiting, sent, failed, waiting_profit, sent_profit, missed_profit, unique_inputs FROM tbl_stats WHERE chain = $1",
)
.bind(chain)
.fetch_all(p)
.await?;
for row in rows {
results.push(StatsRow {
report_date: row.try_get("report_date").unwrap_or_default(),
chain: row.try_get("chain").unwrap_or_default(),
totals: row.try_get::<i32, _>("totals").unwrap_or(0) as i64,
waiting: row.try_get::<i32, _>("waiting").unwrap_or(0) as i64,
sent: row.try_get::<i32, _>("sent").unwrap_or(0) as i64,
failed: row.try_get::<i32, _>("failed").unwrap_or(0) as i64,
waiting_profit: row.try_get::<i32, _>("waiting_profit").unwrap_or(0) as i64,
sent_profit: row.try_get::<i32, _>("sent_profit").unwrap_or(0) as i64,
missed_profit: row.try_get::<i32, _>("missed_profit").unwrap_or(0) as i64,
unique_inputs: row.try_get::<i32, _>("unique_inputs").unwrap_or(0) as i64,
});
}
}
}
Ok(results)
}
#[derive(Debug, Clone)]
pub struct SearchTxRow {
pub status: String,
pub tx: String,
pub our_address: String,
pub our_fees: String,
pub reqid: String,
}
pub async fn search_tx(
pool: &DatabasePool,
txid: &str,
) -> Result<Option<SearchTxRow>, sqlx::Error> {
match pool {
DatabasePool::SQLite(p) => {
let result = sqlx::query("SELECT * FROM tbl_tx WHERE txid = ? LIMIT 1")
.bind(txid)
.fetch_optional(p)
.await?;
Ok(result.map(|row| SearchTxRow {
status: row.try_get::<i64, _>("status").unwrap_or(0).to_string(),
tx: row.try_get::<String, _>("tx").unwrap_or_default(),
our_address: row.try_get::<String, _>("our_address").unwrap_or_default(),
our_fees: row.try_get::<String, _>("our_fees").unwrap_or_default(),
reqid: row.try_get::<String, _>("reqid").unwrap_or_default(),
}))
}
DatabasePool::PostgreSQL(p) => {
let result = sqlx::query("SELECT * FROM tbl_tx WHERE txid = $1 LIMIT 1")
.bind(txid)
.fetch_optional(p)
.await?;
Ok(result.map(|row| SearchTxRow {
status: row.try_get::<i32, _>("status").unwrap_or(0).to_string(),
tx: row.try_get::<String, _>("tx").unwrap_or_default(),
our_address: row.try_get::<String, _>("our_address").unwrap_or_default(),
our_fees: row.try_get::<String, _>("our_fees").unwrap_or_default(),
reqid: row.try_get::<String, _>("reqid").unwrap_or_default(),
}))
}
}
}
pub async fn calculate_and_upsert_stats(
pool: &DatabasePool,
chain: &str,
) -> Result<(), sqlx::Error> {
if !chain
.chars()
.all(|c| c.is_alphanumeric() || c == '-' || c == '_')
|| chain.is_empty()
{
error!("Invalid chain name: {chain}");
return Ok(());
}
match pool {
DatabasePool::SQLite(p) => {
sqlx::query("DELETE FROM tbl_stats WHERE chain = ?")
.bind(chain)
.execute(p)
.await?;
sqlx::query(
"INSERT INTO tbl_stats (
report_date, chain, totals, waiting, sent, failed,
waiting_profit, sent_profit, missed_profit, unique_inputs
)
SELECT
CURRENT_TIMESTAMP,
?,
(SELECT COUNT(*) FROM tbl_tx WHERE network = ?),
(SELECT COUNT(*) FROM tbl_tx WHERE status = 0 AND network = ?),
(SELECT COUNT(*) FROM tbl_tx WHERE status = 1 AND network = ?),
(SELECT COUNT(*) FROM tbl_tx WHERE status = 2 AND network = ?),
(SELECT IFNULL(SUM(our_fees),0) FROM tbl_tx WHERE status = 0 AND network = ?),
(SELECT IFNULL(SUM(our_fees),0) FROM tbl_tx WHERE status = 1 AND network = ?),
(SELECT IFNULL(SUM(our_fees),0) FROM tbl_tx WHERE status = 2 AND network = ?),
(SELECT COUNT(DISTINCT tbl_inp.in_txid)
FROM tbl_inp
JOIN tbl_tx ON tbl_inp.txid = tbl_tx.txid
WHERE tbl_tx.status = 0 AND tbl_tx.network = ?)
ON CONFLICT(chain) DO UPDATE SET
report_date = excluded.report_date,
totals = excluded.totals,
waiting = excluded.waiting,
sent = excluded.sent,
failed = excluded.failed,
waiting_profit = excluded.waiting_profit,
sent_profit = excluded.sent_profit,
missed_profit = excluded.missed_profit,
unique_inputs = excluded.unique_inputs",
)
.bind(chain)
.bind(chain)
.bind(chain)
.bind(chain)
.bind(chain)
.bind(chain)
.bind(chain)
.bind(chain)
.bind(chain)
.execute(p)
.await?;
}
DatabasePool::PostgreSQL(p) => {
sqlx::query("DELETE FROM tbl_stats WHERE chain = $1")
.bind(chain)
.execute(p)
.await?;
sqlx::query(
"INSERT INTO tbl_stats (
report_date, chain, totals, waiting, sent, failed,
waiting_profit, sent_profit, missed_profit, unique_inputs
)
SELECT
CURRENT_TIMESTAMP,
$1,
(SELECT COUNT(*) FROM tbl_tx WHERE network = $1),
(SELECT COUNT(*) FROM tbl_tx WHERE status = 0 AND network = $1),
(SELECT COUNT(*) FROM tbl_tx WHERE status = 1 AND network = $1),
(SELECT COUNT(*) FROM tbl_tx WHERE status = 2 AND network = $1),
(SELECT COALESCE(SUM(our_fees),0) FROM tbl_tx WHERE status = 0 AND network = $1),
(SELECT COALESCE(SUM(our_fees),0) FROM tbl_tx WHERE status = 1 AND network = $1),
(SELECT COALESCE(SUM(our_fees),0) FROM tbl_tx WHERE status = 2 AND network = $1),
(SELECT COUNT(DISTINCT tbl_inp.in_txid)
FROM tbl_inp
JOIN tbl_tx ON tbl_inp.txid = tbl_tx.txid
WHERE tbl_tx.status = 0 AND tbl_tx.network = $1)
ON CONFLICT(chain) DO UPDATE SET
report_date = excluded.report_date,
totals = excluded.totals,
waiting = excluded.waiting,
sent = excluded.sent,
failed = excluded.failed,
waiting_profit = excluded.waiting_profit,
sent_profit = excluded.sent_profit,
missed_profit = excluded.missed_profit,
unique_inputs = excluded.unique_inputs",
)
.bind(chain)
.execute(p)
.await?;
}
}
info!("tbl_stats creation success");
Ok(())
}

233
src/db/schema.rs Normal file
View File

@@ -0,0 +1,233 @@
use sqlx::{PgPool, SqlitePool};
pub async fn create_sqlite_schema(pool: &SqlitePool) -> Result<(), sqlx::Error> {
sqlx::query(
"CREATE TABLE IF NOT EXISTS tbl_tx (
txid TEXT PRIMARY KEY,
date_creation TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
date_update TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
wtxid TEXT,
ntxid TEXT,
tx TEXT,
locktime INTEGER,
network TEXT,
network_fees TEXT,
reqid TEXT,
our_fees TEXT,
our_address TEXT,
status INTEGER DEFAULT 0
)",
)
.execute(pool)
.await?;
let _ = sqlx::query("ALTER TABLE tbl_tx ADD COLUMN push_err TEXT")
.execute(pool)
.await;
sqlx::query(
"CREATE TABLE IF NOT EXISTS tbl_inp (
id INTEGER,
txid TEXT,
in_txid TEXT,
in_vout INTEGER
)",
)
.execute(pool)
.await?;
sqlx::query(
"CREATE UNIQUE INDEX IF NOT EXISTS idx_inp_unique ON tbl_inp(txid, in_txid, in_vout)",
)
.execute(pool)
.await?;
sqlx::query(
"CREATE TABLE IF NOT EXISTS tbl_out (
id INTEGER,
txid TEXT,
script_pubkey TEXT,
amount TEXT,
vout INTEGER
)",
)
.execute(pool)
.await?;
sqlx::query(
"CREATE UNIQUE INDEX IF NOT EXISTS idx_out_unique ON tbl_out(txid, script_pubkey, amount, vout)",
)
.execute(pool)
.await?;
sqlx::query(
"CREATE TABLE IF NOT EXISTS tbl_xpub (
id INTEGER PRIMARY KEY,
network TEXT,
xpub TEXT,
date_create TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
path_idx INTEGER DEFAULT -1
)",
)
.execute(pool)
.await?;
sqlx::query("CREATE UNIQUE INDEX IF NOT EXISTS idx_xpub ON tbl_xpub(network, xpub)")
.execute(pool)
.await?;
sqlx::query(
"CREATE TABLE IF NOT EXISTS tbl_address (
address TEXT PRIMARY KEY,
path TEXT NOT NULL,
date_create TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
xpub INTEGER,
remote_address TEXT
)",
)
.execute(pool)
.await?;
sqlx::query(
"CREATE TABLE IF NOT EXISTS tbl_stats (
report_date TEXT,
chain TEXT,
totals INTEGER,
waiting INTEGER,
sent INTEGER,
failed INTEGER,
waiting_profit INTEGER,
sent_profit INTEGER,
missed_profit INTEGER,
unique_inputs INTEGER
)",
)
.execute(pool)
.await?;
sqlx::query("DROP INDEX IF EXISTS idx_stats_chain")
.execute(pool)
.await?;
sqlx::query("CREATE UNIQUE INDEX IF NOT EXISTS idx_stats_chain ON tbl_stats(chain)")
.execute(pool)
.await?;
sqlx::query("UPDATE tbl_tx SET network='bitcoin' WHERE network='mainnet'")
.execute(pool)
.await?;
Ok(())
}
pub async fn create_pg_schema(pool: &PgPool) -> Result<(), sqlx::Error> {
sqlx::query(
"CREATE TABLE IF NOT EXISTS tbl_tx (
txid TEXT PRIMARY KEY,
date_creation TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
date_update TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
wtxid TEXT,
ntxid TEXT,
tx TEXT,
locktime INTEGER,
network TEXT,
network_fees TEXT,
reqid TEXT,
our_fees TEXT,
our_address TEXT,
status INTEGER DEFAULT 0,
push_err TEXT
)",
)
.execute(pool)
.await?;
sqlx::query(
"CREATE TABLE IF NOT EXISTS tbl_inp (
id SERIAL PRIMARY KEY,
txid TEXT,
in_txid TEXT,
in_vout INTEGER
)",
)
.execute(pool)
.await?;
sqlx::query(
"CREATE UNIQUE INDEX IF NOT EXISTS idx_inp_unique ON tbl_inp(txid, in_txid, in_vout)",
)
.execute(pool)
.await?;
sqlx::query(
"CREATE TABLE IF NOT EXISTS tbl_out (
id SERIAL PRIMARY KEY,
txid TEXT,
script_pubkey TEXT,
amount TEXT,
vout INTEGER
)",
)
.execute(pool)
.await?;
sqlx::query(
"CREATE UNIQUE INDEX IF NOT EXISTS idx_out_unique ON tbl_out(txid, script_pubkey, amount, vout)",
)
.execute(pool)
.await?;
sqlx::query(
"CREATE TABLE IF NOT EXISTS tbl_xpub (
id SERIAL PRIMARY KEY,
network TEXT,
xpub TEXT,
date_create TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
path_idx INTEGER DEFAULT -1
)",
)
.execute(pool)
.await?;
sqlx::query("CREATE UNIQUE INDEX IF NOT EXISTS idx_xpub ON tbl_xpub(network, xpub)")
.execute(pool)
.await?;
sqlx::query(
"CREATE TABLE IF NOT EXISTS tbl_address (
address TEXT PRIMARY KEY,
path TEXT NOT NULL,
date_create TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
xpub INTEGER,
remote_address TEXT
)",
)
.execute(pool)
.await?;
sqlx::query(
"CREATE TABLE IF NOT EXISTS tbl_stats (
report_date TEXT,
chain TEXT,
totals INTEGER,
waiting INTEGER,
sent INTEGER,
failed INTEGER,
waiting_profit INTEGER,
sent_profit INTEGER,
missed_profit INTEGER,
unique_inputs INTEGER
)",
)
.execute(pool)
.await?;
sqlx::query("CREATE UNIQUE INDEX IF NOT EXISTS idx_stats_chain ON tbl_stats(chain)")
.execute(pool)
.await?;
sqlx::query("UPDATE tbl_tx SET network='bitcoin' WHERE network='mainnet'")
.execute(pool)
.await?;
Ok(())
}

242
src/xpub.rs2 Normal file
View File

@@ -0,0 +1,242 @@
//use bs58;
use bitcoin::Address;
use bitcoin::Network;
use bitcoin::ScriptBuf;
use bitcoin::WPubkeyHash;
use bitcoin::bip32::DerivationPath;
use bitcoin::bip32::Xpub;
use bitcoin::hashes::Hash;
use bitcoin::key::Secp256k1;
use sha2::{Digest, Sha256};
use std::str::FromStr;
// Mainnet (BIP44/BIP49/BIP84)
enum BS58Prefix {
Xpub,
Ypub,
Zpub,
Tpub,
Vpub,
Upub,
}
const XPUB_PREFIX: [u8; 4] = [0x04, 0x88, 0xB2, 0x1E]; // xpub (Legacy P2PKH)
const YPUB_PREFIX: [u8; 4] = [0x04, 0x9D, 0x7C, 0xB2]; // ypub (Nested SegWit P2SH-P2WPKH)
const ZPUB_PREFIX: [u8; 4] = [0x04, 0xB2, 0x47, 0x46]; // zpub (Native SegWit P2WPKH)
const TPUB_PREFIX: [u8; 4] = [0x04, 0x35, 0x87, 0xCF]; // tpub (Testnet Legacy P2PKH)
const VPUB_PREFIX: [u8; 4] = [0x04, 0x5F, 0x1C, 0xF6]; // vpub (Testnet Nested SegWit)
const UPUB_PREFIX: [u8; 4] = [0x04, 0x4A, 0x52, 0x62]; // upub (RegTest Nested SegWit)
// Constants from Bitcoin Core's checksum algorithm
const INPUT_CHARSET: &[u8] = b"0123456789()[],'/*abcdefgh@:$%{}IJKLMNOPQRSTUVWXYZ&+-.;<=>?!^_|~ijklmnopqrstuvwxyzABCDEFGH`#\"\\ ";
const CHECKSUM_CHARSET: &[u8] = b"qpzry9x8gf2tvdw0s3jn54khce6mua7l";
// Polynomial modulo function used in checksum calculation (same as in Bitcoin Core)
fn poly_mod(mut c: u64, val: u64) -> u64 {
let c0 = c >> 35;
c = ((c & 0x7ffffffff) << 5) ^ val;
if c0 & 1 > 0 {
c ^= 0xf5dee51989
};
if c0 & 2 > 0 {
c ^= 0xa9fdca3312
};
if c0 & 4 > 0 {
c ^= 0x1bab10e32d
};
if c0 & 8 > 0 {
c ^= 0x3706b1677a
};
if c0 & 16 > 0 {
c ^= 0x644d626ffd
};
c
}
// Calculate checksum for a descriptor string
fn calc_checksum(desc: &str) -> Result<String, String> {
// Separate descriptor from any existing checksum
let desc = match desc.split_once('#') {
Some((d, _)) => d,
None => desc,
};
let mut c: u64 = 1;
let mut cls: u64 = 0;
let mut clscount: u64 = 0;
// Process each character in the descriptor
for ch in desc.as_bytes() {
let pos = match INPUT_CHARSET.iter().position(|b| b == ch) {
Some(p) => p as u64,
None => return Err(format!("Invalid character in descriptor: {}", *ch as char)),
};
c = poly_mod(c, pos & 31);
cls = cls * 3 + (pos >> 5);
clscount += 1;
if clscount == 3 {
c = poly_mod(c, cls);
cls = 0;
clscount = 0;
}
}
if clscount > 0 {
c = poly_mod(c, cls);
}
// Final steps in checksum calculation
for _ in 0..8 {
c = poly_mod(c, 0);
}
c ^= 1;
// Convert checksum to characters
let mut checksum = String::with_capacity(8);
for j in 0..8 {
let idx = ((c >> (5 * (7 - j))) & 31) as usize;
checksum.push(CHECKSUM_CHARSET[idx] as char);
}
Ok(checksum)
}
pub fn get_bitcoincore_descriptor(xpub: &String) -> String {
let fingerprint = calculate_fingerprint(xpub);
let mut bip = 84;
let cpub = xpub.to_string();
match &xpub[0..4] {
"vpub" => {
bip = 84;
}
"zpub" => {
bip = 84;
}
&_ => {
bip = 84;
}
};
let descriptor = format!(
"wpkh([{}/84h/0h/0h]{}/0/*)",
fingerprint,
convert_xpub(xpub)
);
let descriptor = match calc_checksum(&descriptor) {
Ok(checksum) => {
let clean_descriptor = descriptor.split('#').next().unwrap_or(&descriptor);
format!("{}#{}", clean_descriptor, checksum)
}
Err(err) => {
eprintln!("Error: {}", err);
"".to_string()
}
};
descriptor
//format!("{}#{}",descriptor,checksum)
}
fn convert_xpub(xpub: &String) -> String {
if xpub[0..4] == *"xpub" || xpub[0..4] == *"ypub" || xpub[0..4] == *"zpub" {
return convert_to(xpub, BS58Prefix::Xpub).unwrap();
} else {
return convert_to(xpub, BS58Prefix::Tpub).unwrap();
}
}
pub fn calculate_fingerprint(tpub: &str) -> String {
let xpub = Xpub::from_str(&convert_to(tpub, BS58Prefix::Xpub).unwrap()).unwrap();
let fp = xpub.fingerprint();
let pp = xpub.parent_fingerprint;
format!("{}", fp)
}
fn base58check_decode(s: &str) -> Result<Vec<u8>, String> {
let data = bs58::decode(s).into_vec().map_err(|e| e.to_string())?;
if data.len() < 4 {
return Err("Data troppo corta".to_string());
}
let (payload, checksum) = data.split_at(data.len() - 4);
let hash = Sha256::digest(&Sha256::digest(payload));
if hash[0..4] != checksum[..] {
return Err("Checksum invalido".to_string());
}
Ok(payload.to_vec())
}
fn base58check_encode(data: &[u8]) -> String {
let checksum = &Sha256::digest(&Sha256::digest(data))[0..4];
let full = [data, checksum].concat();
bs58::encode(full).into_string()
}
fn convert_to(zpub: &str, prefix: BS58Prefix) -> Result<String, String> {
let mut data = base58check_decode(zpub)?;
if data.len() < 4 {
return Err("Non è una zpub valida.".to_string());
}
data.splice(
0..4,
match prefix {
BS58Prefix::Xpub => XPUB_PREFIX,
BS58Prefix::Ypub => YPUB_PREFIX,
BS58Prefix::Zpub => ZPUB_PREFIX,
BS58Prefix::Vpub => VPUB_PREFIX,
BS58Prefix::Tpub => TPUB_PREFIX,
BS58Prefix::Upub => UPUB_PREFIX,
},
);
Ok(base58check_encode(&data))
}
pub fn new_address_from_xpub(
zpub: &str,
index: i64,
network: Network,
) -> Result<(String, String), Box<dyn std::error::Error>> {
let xpub = Xpub::from_str(&convert_to(zpub, BS58Prefix::Xpub)?)?;
let path = format!("m/0/{}", index);
let derivation_path = DerivationPath::from_str(&path.as_str())?;
let secp = Secp256k1::new();
let derived_xpub = xpub.derive_pub(&secp, &derivation_path)?;
let public_key = derived_xpub.public_key;
let pubkey_bytes = public_key.serialize();
let witness_program = WPubkeyHash::hash(&pubkey_bytes);
let redeem_script = ScriptBuf::new_p2wpkh(&witness_program);
//let script_pubkey = ScriptBuf::new_p2sh(&redeem_script.script_hash());
let address = Address::from_script(&redeem_script, network)?;
//let address = Address::from_script(&script_pubkey, network)?;
Ok((address.to_string(), path.to_string()))
}
/*
fn main() -> Result<(), Box<dyn std::error::Error>>{
//let zpub = "xpub6C29v8gxCXREHUzoGNfqqFqZWxTVEmYtmZshuzfSwBKNmfYQxoizRziCkkUUA4WwJZkJs2i7nttRiC6MQG7mxZpouXeYkTZe3U52RyPAeo2";
//let zpub = "vpub5Ut36m34VebUUjdhYaxJCjSPqk3ZR8bA2MXLmbHRQCycAxy5Q1GFPJspLkJywJjBgQnvU3rmwPKTPp1ELLWeXrve3zBufpZR4MRCCTNHzsn";
let zpub = "zpub6qdfveGrxBQN3z8paZ88EHpCn5MGXpUoHwQmHhPbj4rPQtUjbWyCHrJFYZGVY7MsmVbDaeu4JYqRqcdLzMx78wZFEWbLrF9FG3gr2MPQC5H";
match convert_to(zpub,BS58Prefix::Tpub) {
Ok(tpub) => println!("XPUB: {}", tpub),
Err(e) => eprintln!("Errore: {}", e),
}
let fingerprint = base58check_encode(&calculate_fingerprint(zpub));
println!("ZPUB: {}, FINGERPRINT: {}",zpub,fingerprint);
let xpub = Xpub::from_str(&convert_to(zpub,BS58Prefix::Xpub)?)?;
let tpub = convert_to(zpub,BS58Prefix::Tpub)?;
let fingerprint = base58check_encode(&calculate_fingerprint(&tpub));
println!("TPUB: {}, FINGERPRINT: {}",tpub,fingerprint);
let derivation_path = DerivationPath::from_str("m/0/0")?;
let secp = Secp256k1::new();
let derived_xpub = xpub.derive_pub(&secp, &derivation_path)?;
let public_key = derived_xpub.public_key;
let pubkey_bytes = public_key.serialize();
let witness_program = WPubkeyHash::hash(&pubkey_bytes);
let redeem_script = ScriptBuf::new_p2wpkh(&witness_program);
let script_pubkey = ScriptBuf::new_p2sh(&redeem_script.script_hash());
// Generate the Bitcoin SegWit (BIP49) address
let network = Network::Bitcoin;
let address = Address::from_script(&redeem_script, network)?;
let address = Address::from_script(&script_pubkey, network)?;
Ok(())
}*/

1
start_postgres_docker.sh Normal file
View File

@@ -0,0 +1 @@
docker run -d --name test-pg -e POSTGRES_PASSWORD=testpass -e POSTGRES_USER=testuser -e POSTGRES_DB=testdb -p 5432:5432 postgres:16-alpine

55
tbitcoind.service Normal file
View File

@@ -0,0 +1,55 @@
# /etc/systemd/system/tbitcoind.service
[Unit]
Description=Bitcoin Testnet daemon
After=network.target
[Service]
# Service execution
###################
ExecStart=/usr/local/bin/bitcoind -daemon -testnet \
-pid=/run/tbitcoind/tbitcoind.pid \
-conf=/home/bitcoin/.bitcoin/bitcoin.conf \
-datadir=/home/bitcoin/.bitcoin \
-startupnotify="chmod g+r /home/bitcoin/.bitcoin/testnet3/.cookie"
# Process management
####################
Type=forking
PIDFile=/run/tbitcoind/tbitcoind.pid
Restart=on-failure
TimeoutSec=300
RestartSec=30
# Directory creation and permissions
####################################
User=bitcoin
UMask=0027
# /run/tbitcoind
RuntimeDirectory=tbitcoind
RuntimeDirectoryMode=0710
# Hardening measures
####################
# Provide a private /tmp and /var/tmp.
PrivateTmp=true
# Mount /usr, /boot/ and /etc read-only for the process.
ProtectSystem=full
# Disallow the process and all of its children to gain
# new privileges through execve().
NoNewPrivileges=true
# Use a new /dev namespace only populated with API pseudo devices
# such as /dev/null, /dev/zero and /dev/random.
PrivateDevices=true
# Deny the creation of writable and executable memory mappings.
MemoryDenyWriteExecute=true
[Install]
WantedBy=multi-user.target

30
test Normal file
View File

@@ -0,0 +1,30 @@
{
"amount": -0.00050000,
"fee": -0.00000141,
"confirmations": 0,
"trusted": false,
"txid": "931dd22dca06fee28c0bdeb404ce055bbd4250c5a2ea0f9dee1d43def1838e0e",
"wtxid": "87c53a3f39a8874c4a70b65108f81c9899f199d79e378ce254198aaf25b185e6",
"walletconflicts": [
],
"mempoolconflicts": [
],
"time": 1747802029,
"timereceived": 1747802029,
"bip125-replaceable": "yes",
"details": [
{
"address": "bcrt1qh2c83yulvs7kgw0g6q3lkxqws4cnf0uxpcgcpt",
"category": "send",
"amount": -0.00050000,
"vout": 1,
"fee": -0.00000141,
"abandoned": false
}
],
"hex": "02000000000101f54628aceb97a6140c5266ae6367c22db0e451c5683ef67455f99c2a92f6fece0000000000fdffffff02a3b8804a00000000160014fc2a81aebc081721099e0f394b82736f6651535650c3000000000000160014bab078939f643d6439e8d023fb180e857134bf8602473044022055da1e26c408892a5c8e1ac04fd7e1ce7a2fbd92ee10d977228a845423073d6402200f05d658ebf3a64e2f33e59a1378aa4f3966381917437b3a7627698f332b3c97012102614ec7c018916f137ea4137a05c328c1c0b6cc9ffb6b7a7aca249bff15124ceaa7010000",
"lastprocessedblock": {
"hash": "469ae576ee05edae3fe7dfc800f24d61165759f4ac57415adc5b14afcdf4a9ff",
"height": 423
}
}

146
tests/db_path_validation.rs
View File

@@ -1,15 +1,62 @@
use bal_server::db::open_db;
use sqlite::State;
use std::fs;
use bal_server::db::{DatabasePool, create_database, open_database};
use sqlx::Row;
#[test]
fn test_open_db_blocks_traversal() {
let res = open_db("../etc/passwd");
#[tokio::test]
async fn test_open_database_sqlite() {
let pool = open_database("sqlite", "sqlite::memory:").await;
assert!(pool.is_ok(), "Opening SQLite in-memory should succeed");
}
#[tokio::test]
async fn test_open_database_invalid_backend() {
let pool = open_database("oracle", "connection_string").await;
assert!(pool.is_err(), "Unknown backend should fail");
}
#[tokio::test]
async fn test_create_database_sqlite() {
let pool = open_database("sqlite", "sqlite::memory:").await.unwrap();
let result = create_database(&pool).await;
assert!(result.is_ok(), "Creating SQLite schema should succeed");
}
#[tokio::test]
async fn test_create_database_is_idempotent() {
let pool = open_database("sqlite", "sqlite::memory:").await.unwrap();
create_database(&pool).await.unwrap();
let result = create_database(&pool).await;
assert!(
result.is_ok(),
"Creating schema twice should succeed (idempotent)"
);
}
#[tokio::test]
async fn test_sqlite_wal_mode() {
let tmp_path = std::env::temp_dir().join("tmp_test_wal_mode.db");
let path_str = tmp_path.to_str().unwrap();
let _ = std::fs::remove_file(&tmp_path);
let dsn = format!("sqlite:{}?mode=rwc", path_str);
let pool = open_database("sqlite", &dsn).await.unwrap();
if let DatabasePool::SQLite(p) = &pool {
let row = sqlx::query("PRAGMA journal_mode")
.fetch_one(p)
.await
.unwrap();
let mode: String = row.try_get(0).unwrap();
assert_eq!(mode, "wal", "SQLite journal mode should be WAL");
}
drop(pool);
let _ = std::fs::remove_file(&tmp_path);
let _ = std::fs::remove_file(format!("{}-shm", path_str));
let _ = std::fs::remove_file(format!("{}-wal", path_str));
}
#[tokio::test]
async fn test_open_database_blocks_traversal() {
let res = open_database("sqlite", "sqlite:../etc/passwd").await;
assert!(res.is_err(), "Path with '..' should be rejected");
let err = match res {
Err(e) => e,
Ok(_) => panic!("Expected error for traversal path"),
};
let err = res.err().unwrap();
assert!(
err.contains("'..'"),
"Error should mention directory traversal: {}",
@@ -17,15 +64,17 @@ fn test_open_db_blocks_traversal() {
);
}
#[test]
fn test_open_db_blocks_forbidden_absolute() {
for path in ["/etc/passwd", "/proc/self/mem", "/dev/null", "/usr/bin/ls"] {
let res = open_db(path);
#[tokio::test]
async fn test_open_database_blocks_forbidden_absolute() {
for path in [
"sqlite:/etc/passwd",
"sqlite:/proc/self/mem",
"sqlite:/dev/null",
"sqlite:/usr/bin/ls",
] {
let res = open_database("sqlite", path).await;
assert!(res.is_err(), "Absolute path {} should be rejected", path);
let err = match res {
Err(e) => e,
Ok(_) => panic!("Expected error for forbidden path {}", path),
};
let err = res.err().unwrap();
assert!(
err.contains("forbidden"),
"Error should mention forbidden prefix: {}",
@@ -34,59 +83,26 @@ fn test_open_db_blocks_forbidden_absolute() {
}
}
#[test]
fn test_open_db_allows_relative() {
let test_path = "tmp_test_bal.db";
let _ = fs::remove_file(test_path);
let res = open_db(test_path);
assert!(res.is_ok(), "Valid relative path should be allowed");
let db = res.unwrap();
drop(db);
let _ = fs::remove_file(test_path);
}
#[tokio::test]
async fn test_open_database_rejects_symlink() {
let tmp_dir = std::env::temp_dir();
let real = tmp_dir.join("tmp_test_real_symlink.db");
let link = tmp_dir.join("tmp_test_link_symlink.db");
let _ = std::fs::remove_file(&real);
let _ = std::fs::remove_file(&link);
std::fs::File::create(&real).unwrap();
std::os::unix::fs::symlink(&real, &link).unwrap();
#[test]
fn test_open_db_wal_pragmas_set() {
let test_path = "tmp_test_wal.db";
let _ = fs::remove_file(test_path);
let _ = fs::remove_file(format!("{}-shm", test_path));
let _ = fs::remove_file(format!("{}-wal", test_path));
let db = open_db(test_path).expect("Should open DB");
let mut stmt = db.prepare("PRAGMA journal_mode;").unwrap();
if let Ok(State::Row) = stmt.next() {
let mode: String = stmt.read(0).unwrap();
assert_eq!(mode, "wal", "SQLite journal mode should be WAL");
} else {
panic!("Could not read journal_mode pragma");
}
let _ = fs::remove_file(test_path);
let _ = fs::remove_file(format!("{}-shm", test_path));
let _ = fs::remove_file(format!("{}-wal", test_path));
}
#[test]
fn test_open_db_rejects_symlink() {
let real = "tmp_test_real.db";
let link = "tmp_test_link.db";
let _ = fs::remove_file(real);
let _ = fs::remove_file(link);
fs::File::create(real).unwrap();
std::os::unix::fs::symlink(real, link).unwrap();
let res = open_db(link);
let dsn = format!("sqlite:{}?mode=rwc", link.to_str().unwrap());
let res = open_database("sqlite", &dsn).await;
assert!(res.is_err(), "Symlink DB path should be rejected");
let err = match res {
Err(e) => e,
Ok(_) => panic!("Expected error for symlink"),
};
let err = res.err().unwrap();
assert!(
err.contains("symlink"),
"Error should mention symlink: {}",
err
);
let _ = fs::remove_file(real);
let _ = fs::remove_file(link);
let _ = std::fs::remove_file(&real);
let _ = std::fs::remove_file(&link);
}

66
tests/input_validation_tests.rs
View File

@@ -1,60 +1,48 @@
use bal_server::db::{get_all_addresses_by_xpub, open_db};
use sqlite::Value;
use bal_server::db::{DatabasePool, create_database, get_all_addresses_by_xpub, open_database};
fn setup_db_with_xpub() -> sqlite::Connection {
let db = open_db(":memory:").unwrap();
let _ = db.execute(
"CREATE TABLE tbl_xpub (id INTEGER PRIMARY KEY, network TEXT, xpub TEXT, path_idx INTEGER DEFAULT -1);"
);
let _ = db.execute(
"CREATE TABLE tbl_address (address TEXT PRIMARY KEY, path TEXT, xpub INTEGER, remote_address TEXT);"
);
// Insert test xpub
let mut stmt = db
.prepare("INSERT INTO tbl_xpub(id, network, xpub) VALUES(?, ?, ?);")
async fn setup_db_with_xpub() -> DatabasePool {
let pool = open_database("sqlite", "sqlite::memory:").await.unwrap();
create_database(&pool).await.unwrap();
if let DatabasePool::SQLite(p) = &pool {
sqlx::query("INSERT INTO tbl_xpub(id, network, xpub) VALUES(1, 'testnet', 'tpub_test')")
.execute(p)
.await
.unwrap();
stmt.bind((1, Value::Integer(1))).unwrap();
stmt.bind((2, Value::String("testnet".to_string())))
.unwrap();
stmt.bind((3, Value::String("tpub_test".to_string())))
.unwrap();
let _ = stmt.next();
drop(stmt);
// Insert test addresses
for addr in ["addr1", "addr2", "addr3"] {
let mut stmt = db
.prepare("INSERT INTO tbl_address(address, path, xpub) VALUES(?, ?, ?);")
sqlx::query("INSERT INTO tbl_address(address, path, xpub) VALUES(?, 'm/0/1', 1)")
.bind(addr)
.execute(p)
.await
.unwrap();
stmt.bind((1, Value::String(addr.to_string()))).unwrap();
stmt.bind((2, Value::String("m/0/1".to_string()))).unwrap();
stmt.bind((3, Value::Integer(1))).unwrap();
let _ = stmt.next();
drop(stmt);
}
db
}
#[test]
fn test_get_all_addresses_by_xpub_returns_known() {
let db = setup_db_with_xpub();
let addresses = get_all_addresses_by_xpub(&db, "tpub_test").unwrap();
pool
}
#[tokio::test]
async fn test_get_all_addresses_by_xpub_returns_known() {
let pool = setup_db_with_xpub().await;
let addresses = get_all_addresses_by_xpub(&pool, "tpub_test").await.unwrap();
assert!(addresses.contains("addr1"));
assert!(addresses.contains("addr2"));
assert!(addresses.contains("addr3"));
assert_eq!(addresses.len(), 3);
}
#[test]
fn test_get_all_addresses_by_xpub_empty_for_missing() {
let db = setup_db_with_xpub();
let addresses = get_all_addresses_by_xpub(&db, "tpub_nonexistent").unwrap();
#[tokio::test]
async fn test_get_all_addresses_by_xpub_empty_for_missing() {
let pool = setup_db_with_xpub().await;
let addresses = get_all_addresses_by_xpub(&pool, "tpub_nonexistent")
.await
.unwrap();
assert!(addresses.is_empty());
}
#[test]
fn test_network_unknown_returns_404() {
// This is a code-level check; the actual HTTP test requires actix-web setup.
// Verify that the NETWORKS constant includes the expected set.
let networks = ["bitcoin", "testnet", "testnet4", "signet", "regtest"];
for n in networks {
assert!(networks.contains(&n), "{} should be a valid network", n);

54
tests/panic_regression_tests.rs
View File

@@ -1,5 +1,4 @@
use sqlite::Connection;
use std::collections::HashMap;
use sqlx::Row;
use std::sync::{Arc, Mutex};
use std::thread;
@@ -8,39 +7,44 @@ fn test_mutex_poisoning_recovery() {
let data = Arc::new(Mutex::new(0));
let c = data.clone();
let handle = thread::spawn(move || {
let _guard = c.lock(); // Acquire lock
panic!("test panic"); // Panic while holding the lock
// _guard is dropped during panic unwinding, poisoning the mutex
let _guard = c.lock();
panic!("test panic");
});
let result = handle.join();
assert!(result.is_err()); // Thread panicked
assert!(result.is_err());
// Recovery: the same pattern used in bal-server.rs
let guard = match data.lock() {
Ok(g) => g,
Err(p) => {
p.into_inner() // Should not panic
}
Err(p) => p.into_inner(),
};
assert_eq!(*guard, 0);
}
#[test]
fn test_db_null_unwrap_or() {
let db = Connection::open(":memory:").unwrap();
let _ = db.execute(
#[tokio::test]
async fn test_db_null_unwrap_or() {
let pool = bal_server::db::open_database("sqlite", "sqlite::memory:")
.await
.unwrap();
if let bal_server::db::DatabasePool::SQLite(p) = &pool {
sqlx::query(
"CREATE TABLE test_stats (report_date TEXT, chain TEXT, totals TEXT, waiting TEXT);",
);
let _ =
db.execute("INSERT INTO test_stats (report_date, chain) VALUES ('2024-01-01', 'testnet');");
)
.execute(p)
.await
.unwrap();
let mut found_value = None;
let _ = db.iterate("SELECT * FROM test_stats;", |pairs| {
let row: HashMap<_, _> = pairs.iter().map(|(k, v)| (k.to_string(), *v)).collect();
let totals = row["totals"].unwrap_or("0").to_string();
found_value = Some(totals);
true
});
sqlx::query("INSERT INTO test_stats (report_date, chain) VALUES ('2024-01-01', 'testnet')")
.execute(p)
.await
.unwrap();
assert_eq!(found_value.unwrap(), "0");
let row = sqlx::query("SELECT * FROM test_stats")
.fetch_one(p)
.await
.unwrap();
let totals: Option<String> = row.try_get("totals").unwrap_or(None);
let totals_value = totals.unwrap_or_else(|| "0".to_string());
assert_eq!(totals_value, "0");
}
}

335
tests/postgresql_integration.rs Normal file
View File

@@ -0,0 +1,335 @@
use bal_server::db::{
DatabasePool, calculate_and_upsert_stats, check_duplicate_txids, create_database,
get_all_addresses_by_xpub, get_next_address_index, get_pending_txs, get_stats, insert_xpub,
open_database, save_new_address, search_tx, update_tx_status,
};
use sqlx::Row;
fn pg_dsn() -> Option<String> {
std::env::var("BAL_TEST_PG_DSN").ok()
}
async fn setup_pg() -> Option<DatabasePool> {
let dsn = pg_dsn()?;
let pool = open_database("postgresql", &dsn).await.ok()?;
// Drop and recreate schema for clean test
if let DatabasePool::PostgreSQL(p) = &pool {
sqlx::query("DROP SCHEMA public CASCADE; CREATE SCHEMA public")
.execute(p)
.await
.ok()?;
}
create_database(&pool).await.ok()?;
Some(pool)
}
#[tokio::test]
async fn test_pg_open_database() {
let Some(dsn) = pg_dsn() else {
eprintln!("skipped: BAL_TEST_PG_DSN not set");
return;
};
let pool = open_database("postgresql", &dsn).await;
assert!(
pool.is_ok(),
"Opening PostgreSQL should succeed: {:?}",
pool.err()
);
}
#[tokio::test]
async fn test_pg_create_schema() {
let Some(pool) = setup_pg().await else {
eprintln!("skipped: PostgreSQL not available");
return;
};
// Second call should also succeed (idempotent)
let result = create_database(&pool).await;
assert!(
result.is_ok(),
"Creating PG schema twice should be idempotent: {:?}",
result.err()
);
}
#[tokio::test]
async fn test_pg_insert_xpub() {
let Some(pool) = setup_pg().await else {
eprintln!("skipped: PostgreSQL not available");
return;
};
insert_xpub(&pool, "testnet", "tpub_test123").await;
insert_xpub(&pool, "testnet", "tpub_test123").await; // duplicate should be ignored
if let DatabasePool::PostgreSQL(p) = &pool {
let row = sqlx::query("SELECT COUNT(*) as cnt FROM tbl_xpub WHERE xpub = 'tpub_test123'")
.fetch_one(p)
.await
.unwrap();
let count: i64 = row.try_get("cnt").unwrap();
assert_eq!(count, 1, "INSERT OR IGNORE should prevent duplicates");
}
}
#[tokio::test]
async fn test_pg_get_next_address_index() {
let Some(pool) = setup_pg().await else {
eprintln!("skipped: PostgreSQL not available");
return;
};
insert_xpub(&pool, "testnet", "tpub_addr_test").await;
let (id, idx) = get_next_address_index(&pool, "testnet", "tpub_addr_test").await;
assert!(id > 0, "Should return valid xpub id, got {}", id);
assert_eq!(idx, 0, "First index should be 0, got {}", idx);
let (id2, idx2) = get_next_address_index(&pool, "testnet", "tpub_addr_test").await;
assert_eq!(id, id2, "xpub id should be stable");
assert_eq!(idx2, 1, "Second index should be 1, got {}", idx2);
}
#[tokio::test]
async fn test_pg_save_and_get_address() {
let Some(pool) = setup_pg().await else {
eprintln!("skipped: PostgreSQL not available");
return;
};
insert_xpub(&pool, "testnet", "tpub_addr_test2").await;
let (xpub_id, _idx) = get_next_address_index(&pool, "testnet", "tpub_addr_test2").await;
save_new_address(&pool, xpub_id, "tb1qtestaddr", "m/0/0", "1.2.3.4").await;
let addrs = get_all_addresses_by_xpub(&pool, "tpub_addr_test2")
.await
.unwrap();
assert!(
addrs.contains("tb1qtestaddr"),
"Should find saved address: {:?}",
addrs
);
}
#[tokio::test]
async fn test_pg_check_duplicate_txids() {
let Some(pool) = setup_pg().await else {
eprintln!("skipped: PostgreSQL not available");
return;
};
// Insert a transaction first via raw SQL (to have a txid to check)
if let DatabasePool::PostgreSQL(p) = &pool {
sqlx::query(
"INSERT INTO tbl_tx (txid, wtxid, ntxid, tx, locktime, network, status)
VALUES ('txid_dup_test', 'wtx1', 'ntx1', 'rawtx', 100, 'testnet', 0)",
)
.execute(p)
.await
.unwrap();
}
let dups = check_duplicate_txids(
&pool,
&["txid_dup_test".to_string(), "txid_new".to_string()],
)
.await
.unwrap();
assert!(
dups.contains("txid_dup_test"),
"Should detect existing txid"
);
assert!(
!dups.contains("txid_new"),
"Should not report non-existing txid"
);
}
#[tokio::test]
async fn test_pg_search_tx() {
let Some(pool) = setup_pg().await else {
eprintln!("skipped: PostgreSQL not available");
return;
};
if let DatabasePool::PostgreSQL(p) = &pool {
sqlx::query(
"INSERT INTO tbl_tx (txid, wtxid, ntxid, tx, locktime, network, status, our_address, our_fees, reqid)
VALUES ('txid_search', 'wtx', 'ntx', 'rawhex', 500, 'testnet', 1, 'tb1ouraddr', '0.0001', 'req123')"
)
.execute(p)
.await
.unwrap();
}
let result = search_tx(&pool, "txid_search").await.unwrap();
assert!(result.is_some(), "Should find the transaction");
let row = result.unwrap();
assert_eq!(row.status, "1", "Status should be read as string '1'");
assert_eq!(row.tx, "rawhex");
assert_eq!(row.our_address, "tb1ouraddr");
assert_eq!(row.our_fees, "0.0001");
assert_eq!(row.reqid, "req123");
let not_found = search_tx(&pool, "txid_nonexistent").await.unwrap();
assert!(not_found.is_none(), "Should return None for missing txid");
}
#[tokio::test]
async fn test_pg_update_tx_status() {
let Some(pool) = setup_pg().await else {
eprintln!("skipped: PostgreSQL not available");
return;
};
if let DatabasePool::PostgreSQL(p) = &pool {
sqlx::query(
"INSERT INTO tbl_tx (txid, wtxid, ntxid, tx, locktime, network, status)
VALUES ('txid_status', 'wtx', 'ntx', 'raw', 100, 'testnet', 0)",
)
.execute(p)
.await
.unwrap();
}
update_tx_status(&pool, "txid_status", 1, None)
.await
.unwrap();
if let DatabasePool::PostgreSQL(p) = &pool {
let row = sqlx::query("SELECT status FROM tbl_tx WHERE txid = 'txid_status'")
.fetch_one(p)
.await
.unwrap();
let status: i64 = row.try_get("status").unwrap();
assert_eq!(status, 1, "Status should be updated to 1");
}
update_tx_status(&pool, "txid_status", 2, Some("test error"))
.await
.unwrap();
if let DatabasePool::PostgreSQL(p) = &pool {
let row = sqlx::query("SELECT status, push_err FROM tbl_tx WHERE txid = 'txid_status'")
.fetch_one(p)
.await
.unwrap();
let status: i64 = row.try_get("status").unwrap();
let push_err: Option<String> = row.try_get("push_err").unwrap();
assert_eq!(status, 2);
assert_eq!(push_err.as_deref(), Some("test error"));
}
}
#[tokio::test]
async fn test_pg_get_pending_txs() {
let Some(pool) = setup_pg().await else {
eprintln!("skipped: PostgreSQL not available");
return;
};
if let DatabasePool::PostgreSQL(p) = &pool {
// Insert pending tx (status=0, locktime < height)
sqlx::query(
"INSERT INTO tbl_tx (txid, wtxid, ntxid, tx, locktime, network, status)
VALUES ('pending1', 'w', 'n', 'rawtx1', 100, 'testnet', 0)",
)
.execute(p)
.await
.unwrap();
// Insert already pushed tx (status=1)
sqlx::query(
"INSERT INTO tbl_tx (txid, wtxid, ntxid, tx, locktime, network, status)
VALUES ('pushed1', 'w', 'n', 'rawtx2', 100, 'testnet', 1)",
)
.execute(p)
.await
.unwrap();
}
let txs = get_pending_txs(&pool, "testnet", 5000000, 200, 1000)
.await
.unwrap();
assert_eq!(txs.len(), 1, "Should only return pending txs");
assert_eq!(txs[0].txid, "pending1");
assert_eq!(txs[0].tx, "rawtx1");
}
#[tokio::test]
async fn test_pg_stats() {
let Some(pool) = setup_pg().await else {
eprintln!("skipped: PostgreSQL not available");
return;
};
// Insert test data
if let DatabasePool::PostgreSQL(p) = &pool {
sqlx::query(
"INSERT INTO tbl_tx (txid, wtxid, ntxid, tx, locktime, network, status, our_fees)
VALUES
('stx1', 'w', 'n', 'r', 100, 'testnet', 0, '0.0001'),
('stx2', 'w', 'n', 'r', 100, 'testnet', 1, '0.0002'),
('stx3', 'w', 'n', 'r', 100, 'testnet', 2, '0.0003')",
)
.execute(p)
.await
.unwrap();
}
calculate_and_upsert_stats(&pool, "testnet").await.unwrap();
let stats = get_stats(&pool, "testnet").await.unwrap();
assert_eq!(stats.len(), 1, "Should have one stats row");
assert_eq!(stats[0].totals, 3);
assert_eq!(stats[0].waiting, 1);
assert_eq!(stats[0].sent, 1);
assert_eq!(stats[0].failed, 1);
}
#[tokio::test]
async fn test_pg_sql_injection_via_push_err() {
let Some(pool) = setup_pg().await else {
eprintln!("skipped: PostgreSQL not available");
return;
};
if let DatabasePool::PostgreSQL(p) = &pool {
sqlx::query(
"CREATE TABLE test_inject (txid TEXT PRIMARY KEY, status INTEGER, push_err TEXT)",
)
.execute(p)
.await
.unwrap();
sqlx::query("INSERT INTO test_inject (txid, status, push_err) VALUES ($1, $2, $3)")
.bind("dummy")
.bind(0_i64)
.bind("")
.execute(p)
.await
.unwrap();
let malicious = "'; DROP TABLE test_inject; --";
sqlx::query("UPDATE test_inject SET status = 2, push_err = $1 WHERE txid = $2")
.bind(malicious)
.bind("dummy")
.execute(p)
.await
.unwrap();
let row = sqlx::query("SELECT status, push_err FROM test_inject WHERE txid = 'dummy'")
.fetch_one(p)
.await
.unwrap();
let status: i64 = row.try_get("status").unwrap();
let push_err: String = row.try_get("push_err").unwrap();
assert_eq!(status, 2);
assert_eq!(push_err, malicious);
// Table should still exist
let cnt = sqlx::query("SELECT COUNT(*) as cnt FROM test_inject")
.fetch_one(p)
.await
.unwrap();
let count: i64 = cnt.try_get("cnt").unwrap();
assert_eq!(count, 1);
}
}

187
tests/sql_injection_tests.rs
View File

@@ -1,91 +1,90 @@
use sqlite::{Connection, Value};
use bal_server::db::{DatabasePool, open_database};
use sqlx::Row;
#[test]
fn test_sql_injection_via_push_err_update() {
// Create an in-memory database and the required table
let db = Connection::open(":memory:").unwrap();
let _ =
db.execute("CREATE TABLE tbl_tx (txid TEXT PRIMARY KEY, status INTEGER, push_err TEXT);");
async fn setup_db() -> DatabasePool {
open_database("sqlite", "sqlite::memory:").await.unwrap()
}
// Insert a dummy transaction
let mut stmt = db
.prepare("INSERT INTO tbl_tx (txid, status, push_err) VALUES (?, ?, ?);")
#[tokio::test]
async fn test_sql_injection_via_push_err_update() {
let pool = setup_db().await;
if let DatabasePool::SQLite(p) = &pool {
sqlx::query("CREATE TABLE tbl_tx (txid TEXT PRIMARY KEY, status INTEGER, push_err TEXT);")
.execute(p)
.await
.unwrap();
sqlx::query("INSERT INTO tbl_tx (txid, status, push_err) VALUES (?, ?, ?)")
.bind("dummy_txid")
.bind(0_i64)
.bind("")
.execute(p)
.await
.unwrap();
stmt.bind((1, Value::String("dummy_txid".to_string())))
.unwrap();
stmt.bind((2, Value::Integer(0))).unwrap();
stmt.bind((3, Value::String("".to_string()))).unwrap();
let _ = stmt.next();
drop(stmt);
// Malicious error payload containing a single quote (SQL injection attempt)
let malicious_error = "'; DROP TABLE tbl_tx; --";
let txid = "dummy_txid";
// Execute the fixed query using parameter binding (safe)
let sql = "UPDATE tbl_tx SET status = 2, push_err = ? WHERE txid = ?";
let mut stmt = db.prepare(sql).unwrap();
stmt.bind((1, Value::String(malicious_error.to_string())))
sqlx::query("UPDATE tbl_tx SET status = 2, push_err = ? WHERE txid = ?")
.bind(malicious_error)
.bind(txid)
.execute(p)
.await
.unwrap();
stmt.bind((2, Value::String(txid.to_string()))).unwrap();
let _ = stmt.next();
// Verify the table still exists and the row was updated correctly
let mut check = db
.prepare("SELECT status, push_err FROM tbl_tx WHERE txid = ?;")
let row = sqlx::query("SELECT status, push_err FROM tbl_tx WHERE txid = ?")
.bind("dummy_txid")
.fetch_one(p)
.await
.unwrap();
check
.bind((1, Value::String("dummy_txid".to_string())))
.unwrap();
assert!(check.next().unwrap() == sqlite::State::Row);
let status: i64 = check.read("status").unwrap();
let push_err: String = check.read("push_err").unwrap();
let status: i64 = row.try_get("status").unwrap();
let push_err: String = row.try_get("push_err").unwrap();
assert_eq!(status, 2);
assert_eq!(push_err, malicious_error);
// Ensure no second row was created (injection would have failed or produced extra rows)
let mut count_stmt = db.prepare("SELECT COUNT(*) FROM tbl_tx;").unwrap();
assert!(count_stmt.next().unwrap() == sqlite::State::Row);
let count: i64 = count_stmt.read(0).unwrap();
let row = sqlx::query("SELECT COUNT(*) as cnt FROM tbl_tx")
.fetch_one(p)
.await
.unwrap();
let count: i64 = row.try_get("cnt").unwrap();
assert_eq!(count, 1);
}
#[test]
fn test_sql_injection_via_txid_update() {
let db = Connection::open(":memory:").unwrap();
let _ = db.execute("CREATE TABLE tbl_tx (txid TEXT PRIMARY KEY, status INTEGER);");
// Insert multiple dummy transactions
for i in 0..3 {
let mut stmt = db
.prepare("INSERT INTO tbl_tx (txid, status) VALUES (?, ?);")
.unwrap();
stmt.bind((1, Value::String(format!("txid_{}", i))))
.unwrap();
stmt.bind((2, Value::Integer(0))).unwrap();
let _ = stmt.next();
}
// Malicious txid payload
#[tokio::test]
async fn test_sql_injection_via_txid_update() {
let pool = setup_db().await;
if let DatabasePool::SQLite(p) = &pool {
sqlx::query("CREATE TABLE tbl_tx (txid TEXT PRIMARY KEY, status INTEGER);")
.execute(p)
.await
.unwrap();
for i in 0..3 {
sqlx::query("INSERT INTO tbl_tx (txid, status) VALUES (?, ?)")
.bind(format!("txid_{}", i))
.bind(0_i64)
.execute(p)
.await
.unwrap();
}
let malicious_txid = "' OR '1'='1";
// The fixed query parameterizes the txid, so this should only update zero rows
let sql = "UPDATE tbl_tx SET status = 1 WHERE txid = ?";
let mut stmt = db.prepare(sql).unwrap();
stmt.bind((1, Value::String(malicious_txid.to_string())))
sqlx::query("UPDATE tbl_tx SET status = 1 WHERE txid = ?")
.bind(malicious_txid)
.execute(p)
.await
.unwrap();
let _ = stmt.next();
// Verify no rows were updated (status should still be 0 for all)
for i in 0..3 {
let mut check = db
.prepare("SELECT status FROM tbl_tx WHERE txid = ?;")
let row = sqlx::query("SELECT status FROM tbl_tx WHERE txid = ?")
.bind(format!("txid_{}", i))
.fetch_one(p)
.await
.unwrap();
check
.bind((1, Value::String(format!("txid_{}", i))))
.unwrap();
assert!(check.next().unwrap() == sqlite::State::Row);
let status: i64 = check.read("status").unwrap();
let status: i64 = row.try_get("status").unwrap();
assert_eq!(
status, 0,
"Row txid_{} should not be updated by malicious txid",
@@ -93,45 +92,45 @@ fn test_sql_injection_via_txid_update() {
);
}
}
}
#[test]
fn test_sql_injection_via_txid_with_comment() {
let db = Connection::open(":memory:").unwrap();
let _ = db.execute("CREATE TABLE tbl_tx (txid TEXT PRIMARY KEY, status INTEGER);");
let mut stmt = db
.prepare("INSERT INTO tbl_tx (txid, status) VALUES (?, ?);")
#[tokio::test]
async fn test_sql_injection_via_txid_with_comment() {
let pool = setup_db().await;
if let DatabasePool::SQLite(p) = &pool {
sqlx::query("CREATE TABLE tbl_tx (txid TEXT PRIMARY KEY, status INTEGER);")
.execute(p)
.await
.unwrap();
sqlx::query("INSERT INTO tbl_tx (txid, status) VALUES (?, ?)")
.bind("safe_txid")
.bind(0_i64)
.execute(p)
.await
.unwrap();
stmt.bind((1, Value::String("safe_txid".to_string())))
.unwrap();
stmt.bind((2, Value::Integer(0))).unwrap();
let _ = stmt.next();
// Another common injection pattern
let malicious_txid = "safe_txid'; UPDATE tbl_tx SET status = 99; --";
let sql = "UPDATE tbl_tx SET status = 1 WHERE txid = ?";
let mut stmt = db.prepare(sql).unwrap();
stmt.bind((1, Value::String(malicious_txid.to_string())))
sqlx::query("UPDATE tbl_tx SET status = 1 WHERE txid = ?")
.bind(malicious_txid)
.execute(p)
.await
.unwrap();
let _ = stmt.next();
// Verify the original row was NOT updated (because it was looking for the full malicious string)
// and no rows have status 99 (the injected update did not execute)
let mut check = db
.prepare("SELECT status FROM tbl_tx WHERE txid = ?;")
let row = sqlx::query("SELECT status FROM tbl_tx WHERE txid = ?")
.bind("safe_txid")
.fetch_one(p)
.await
.unwrap();
check
.bind((1, Value::String("safe_txid".to_string())))
.unwrap();
assert!(check.next().unwrap() == sqlite::State::Row);
let status: i64 = check.read("status").unwrap();
let status: i64 = row.try_get("status").unwrap();
assert_eq!(status, 0, "Original row should not be updated");
let mut count_stmt = db
.prepare("SELECT COUNT(*) FROM tbl_tx WHERE status = 99;")
let row = sqlx::query("SELECT COUNT(*) as cnt FROM tbl_tx WHERE status = 99")
.fetch_one(p)
.await
.unwrap();
assert!(count_stmt.next().unwrap() == sqlite::State::Row);
let count: i64 = count_stmt.read(0).unwrap();
let count: i64 = row.try_get("cnt").unwrap();
assert_eq!(count, 0, "No rows should have status 99");
}
}

287
tests/test_endpoints.sh Executable file
View File

@@ -0,0 +1,287 @@
#!/bin/bash
# tests/test_endpoints.sh — Integration tests for bal-server endpoints
# Usage: ./tests/test_endpoints.sh <base_url>
# Example: ./tests/test_endpoints.sh http://127.0.0.1:9133
# Returns 0 if all tests pass, 1 otherwise
set -uo pipefail
BASE_URL="${1:?Usage: $0 <base_url>}"
# Delay between requests to avoid rate limiter (actix-governor)
DELAY=1.0
PASS=0
FAIL=0
TOTAL=0
# Colors
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
NC='\033[0m'
# Known network for testing (must be enabled in server config)
TEST_NETWORK="regtest"
# A valid 64-char hex txid that does NOT exist in the database
NONEXISTENT_TXID="551dc4841830e457b0932b81eb458a00f87e5342b70333bb92df7475d6ca90f4"
# A valid raw transaction hex (minimal valid tx for testing)
VALID_TX_HEX="020000000100000000000000000000000000000000000000000000000000000000000000000000000000ffffffff0100f2052a0100000043410496b538e853519c726a2c91e61ec112f826d36b0c8080a01e2894d24b051f05e0f03ed7790b09fd327518756ff2a55ecee44b5e08d76f994a7c5f3ffcac88bac"
# ─────────────────────────────────────────────
# Helpers
# ─────────────────────────────────────────────
log_pass() {
PASS=$((PASS + 1))
TOTAL=$((TOTAL + 1))
echo -e " ${GREEN}PASS${NC} $1"
}
log_fail() {
FAIL=$((FAIL + 1))
TOTAL=$((TOTAL + 1))
echo -e " ${RED}FAIL${NC} $1"
if [ -n "${2:-}" ]; then
echo -e " Expected: ${YELLOW}$2${NC}"
echo -e " Got: ${YELLOW}$3${NC}"
fi
}
# Perform a GET request with rate-limit delay, split into BODY and STATUS
curl_get() {
sleep "$DELAY"
local resp
resp=$(curl -s -w "\n%{http_code}" "$@")
BODY=$(echo "$resp" | sed '$d')
STATUS=$(echo "$resp" | tail -1)
}
# Perform a POST request with rate-limit delay, split into BODY and STATUS
curl_post() {
sleep "$DELAY"
local resp
resp=$(curl -s -w "\n%{http_code}" "$@")
BODY=$(echo "$resp" | sed '$d')
STATUS=$(echo "$resp" | tail -1)
}
assert_status() {
local expected="$1" actual="$2" name="$3"
if [ "$actual" = "$expected" ]; then
log_pass "$name (HTTP $actual)"
else
log_fail "$name" "HTTP $expected" "HTTP $actual"
fi
}
assert_body_contains() {
local pattern="$1" body="$2" name="$3"
if echo "$body" | grep -q "$pattern"; then
log_pass "$name (contains '$pattern')"
else
log_fail "$name" "body contains '$pattern'" "body: '$(echo "$body" | head -c 80)'"
fi
}
assert_body_not_empty() {
local body="$1" name="$2"
if [ -n "$body" ]; then
log_pass "$name (non-empty response)"
else
log_fail "$name" "non-empty body" "empty body"
fi
}
assert_json_valid() {
local body="$1" name="$2"
if echo "$body" | jq . >/dev/null 2>&1; then
log_pass "$name (valid JSON)"
else
log_fail "$name" "valid JSON" "invalid JSON: $(echo "$body" | head -c 100)"
fi
}
assert_json_has_key() {
local key="$1" body="$2" name="$3"
if echo "$body" | jq -e ".$key" >/dev/null 2>&1; then
log_pass "$name (has key '$key')"
else
log_fail "$name" "JSON has key '$key'" "key not found"
fi
}
# ─────────────────────────────────────────────
# 1. GET /
# ─────────────────────────────────────────────
echo ""
echo -e "${YELLOW}[1] GET /${NC}"
curl_get "$BASE_URL/"
assert_status "200" "$STATUS" "GET / returns 200"
assert_body_not_empty "$BODY" "GET / returns non-empty body"
# ─────────────────────────────────────────────
# 2. GET /.pub_key.pem
# ─────────────────────────────────────────────
echo ""
echo -e "${YELLOW}[2] GET /.pub_key.pem${NC}"
curl_get "$BASE_URL/.pub_key.pem"
assert_status "200" "$STATUS" "GET /.pub_key.pem returns 200"
assert_body_contains "BEGIN PUBLIC KEY" "$BODY" "GET /.pub_key.pem contains PEM header"
# ─────────────────────────────────────────────
# 3. GET /version
# ─────────────────────────────────────────────
echo ""
echo -e "${YELLOW}[3] GET /version${NC}"
curl_get "$BASE_URL/version"
assert_status "200" "$STATUS" "GET /version returns 200"
assert_body_not_empty "$BODY" "GET /version returns non-empty body"
# ─────────────────────────────────────────────
# 4. GET /{network}/info
# ─────────────────────────────────────────────
echo ""
echo -e "${YELLOW}[4] GET /$TEST_NETWORK/info${NC}"
curl_get "$BASE_URL/$TEST_NETWORK/info"
assert_status "200" "$STATUS" "GET /$TEST_NETWORK/info returns 200"
assert_json_valid "$BODY" "GET /$TEST_NETWORK/info returns valid JSON"
assert_json_has_key "chain" "$BODY" "GET /$TEST_NETWORK/info has 'chain' key"
assert_json_has_key "address" "$BODY" "GET /$TEST_NETWORK/info has 'address' key"
assert_json_has_key "base_fee" "$BODY" "GET /$TEST_NETWORK/info has 'base_fee' key"
assert_json_has_key "info" "$BODY" "GET /$TEST_NETWORK/info has 'info' key"
assert_json_has_key "version" "$BODY" "GET /$TEST_NETWORK/info has 'version' key"
echo ""
echo -e "${YELLOW}[4b] GET /invalidnet/info${NC}"
curl_get "$BASE_URL/invalidnet/info"
assert_status "404" "$STATUS" "GET /invalidnet/info returns 404"
assert_body_contains "Unknown network" "$BODY" "GET /invalidnet/info body says 'Unknown network'"
# ─────────────────────────────────────────────
# 5. GET /{network}/stats
# ─────────────────────────────────────────────
echo ""
echo -e "${YELLOW}[5] GET /$TEST_NETWORK/stats${NC}"
curl_get "$BASE_URL/$TEST_NETWORK/stats"
assert_status "200" "$STATUS" "GET /$TEST_NETWORK/stats returns 200"
assert_json_valid "$BODY" "GET /$TEST_NETWORK/stats returns valid JSON"
echo ""
echo -e "${YELLOW}[5b] GET /invalidnet/stats${NC}"
curl_get "$BASE_URL/invalidnet/stats"
assert_status "404" "$STATUS" "GET /invalidnet/stats returns 404"
assert_body_contains "Unknown network" "$BODY" "GET /invalidnet/stats body says 'Unknown network'"
# ─────────────────────────────────────────────
# 6. POST /searchtx
# ─────────────────────────────────────────────
echo ""
echo -e "${YELLOW}[6] POST /searchtx — empty body${NC}"
curl_post -X POST -d "" "$BASE_URL/searchtx"
assert_status "400" "$STATUS" "POST /searchtx empty body returns 400"
assert_body_contains "Invalid txid" "$BODY" "POST /searchtx empty body says 'Invalid txid'"
echo ""
echo -e "${YELLOW}[6b] POST /searchtx — short txid${NC}"
curl_post -X POST -d "abc123" "$BASE_URL/searchtx"
assert_status "400" "$STATUS" "POST /searchtx short txid returns 400"
assert_body_contains "Invalid txid" "$BODY" "POST /searchtx short txid says 'Invalid txid'"
echo ""
echo -e "${YELLOW}[6c] POST /searchtx — non-hex txid${NC}"
curl_post -X POST -d "zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz" "$BASE_URL/searchtx"
assert_status "400" "$STATUS" "POST /searchtx non-hex txid returns 400"
assert_body_contains "Invalid txid" "$BODY" "POST /searchtx non-hex txid says 'Invalid txid'"
echo ""
echo -e "${YELLOW}[6d] POST /searchtx — nonexistent valid hex txid${NC}"
curl_post -X POST -d "$NONEXISTENT_TXID" "$BASE_URL/searchtx"
# When txid is valid hex but not in DB: either 200 with empty JSON or 404
if [ "$STATUS" = "200" ] || [ "$STATUS" = "404" ]; then
log_pass "POST /searchtx nonexistent txid returns HTTP $STATUS (expected)"
else
log_fail "POST /searchtx nonexistent txid" "HTTP 200 or 404" "HTTP $STATUS"
fi
# Verify Content-Type is application/json for successful searchtx responses
if [ "$STATUS" = "200" ]; then
CONTENT_TYPE=$(curl -s -D - --max-time 3 -X POST -d "$NONEXISTENT_TXID" "$BASE_URL/searchtx" 2>/dev/null | grep -i "^content-type:" | tr -d '\r')
if echo "$CONTENT_TYPE" | grep -q "application/json"; then
log_pass "POST /searchtx Content-Type is application/json"
else
log_fail "POST /searchtx Content-Type" "application/json" "$CONTENT_TYPE"
fi
fi
echo ""
echo -e "${YELLOW}[6e] POST /searchtx — binary non-UTF8 body${NC}"
curl_post -X POST --data-binary $'\xff\xfe\xfd' "$BASE_URL/searchtx"
assert_status "400" "$STATUS" "POST /searchtx binary body returns 400"
assert_body_contains "Invalid UTF-8 body" "$BODY" "POST /searchtx binary body says 'Invalid UTF-8 body'"
# ─────────────────────────────────────────────
# 7. POST /{network}/pushtxs
# ─────────────────────────────────────────────
echo ""
echo -e "${YELLOW}[7] POST /invalidnet/pushtxs${NC}"
curl_post -X POST -d "" "$BASE_URL/invalidnet/pushtxs"
assert_status "404" "$STATUS" "POST /invalidnet/pushtxs returns 404"
assert_body_contains "Unknown network" "$BODY" "POST /invalidnet/pushtxs body says 'Unknown network'"
echo ""
echo -e "${YELLOW}[7b] POST /$TEST_NETWORK/pushtxs — empty body${NC}"
curl_post -X POST -d "" "$BASE_URL/$TEST_NETWORK/pushtxs"
assert_status "200" "$STATUS" "POST /$TEST_NETWORK/pushtxs empty body returns 200"
echo ""
echo -e "${YELLOW}[7c] POST /$TEST_NETWORK/pushtxs — valid hex tx${NC}"
curl_post -X POST -d "$VALID_TX_HEX" "$BASE_URL/$TEST_NETWORK/pushtxs"
# Server returns 200 with "thx" or "already present" or "Bad data received"
if [ "$STATUS" = "200" ]; then
if echo "$BODY" | grep -qE "^(thx|already present|Bad data received)$"; then
log_pass "POST /$TEST_NETWORK/pushtxs valid hex returns HTTP 200 with '$BODY'"
else
log_pass "POST /$TEST_NETWORK/pushtxs valid hex returns HTTP 200 (body: $(echo "$BODY" | head -c 50))"
fi
else
log_fail "POST /$TEST_NETWORK/pushtxs valid hex" "HTTP 200" "HTTP $STATUS"
fi
echo ""
echo -e "${YELLOW}[7d] POST /$TEST_NETWORK/pushtxs — non-hex garbage${NC}"
curl_post -X POST -d "not-a-transaction" "$BASE_URL/$TEST_NETWORK/pushtxs"
assert_status "200" "$STATUS" "POST /$TEST_NETWORK/pushtxs garbage returns 200"
echo ""
echo -e "${YELLOW}[7e] POST /$TEST_NETWORK/pushtxs — binary non-UTF8 body${NC}"
curl_post -X POST --data-binary $'\xff\xfe\xfd' "$BASE_URL/$TEST_NETWORK/pushtxs"
# Invalid UTF-8 should be rejected
if [ "$STATUS" = "400" ]; then
assert_body_contains "Invalid UTF-8 body" "$BODY" "POST /$TEST_NETWORK/pushtxs binary body says 'Invalid UTF-8 body'"
else
# Server may accept binary as latin-1 and skip invalid lines gracefully
log_pass "POST /$TEST_NETWORK/pushtxs binary body returns HTTP $STATUS (skipped gracefully)"
fi
# ─────────────────────────────────────────────
# 8. GET on unknown routes
# ─────────────────────────────────────────────
echo ""
echo -e "${YELLOW}[8] GET /nonexistent${NC}"
curl_get "$BASE_URL/nonexistent"
assert_status "404" "$STATUS" "GET /nonexistent returns 404"
# ─────────────────────────────────────────────
# Summary
# ─────────────────────────────────────────────
echo ""
echo "========================================="
echo -e "Results: ${GREEN}$PASS passed${NC}, ${RED}$FAIL failed${NC} ($TOTAL total)"
echo "========================================="
if [ "$FAIL" -gt 0 ]; then
exit 1
fi
exit 0

271
update Normal file
View File

@@ -0,0 +1,271 @@
Per creare un Webservice in Rust che utilizza Axis (un framework per la definizione di servizi web basato su XML Schema) e Diesel (un ORM per database SQLite), dobbiamo seguire alcuni passaggi. Tuttavia, è
importante notare che l'uso diretto di Axis con Diesel non è il modo più comune o consigliato per creare servizi web in Rust. Più spesso si usa Axum o Rocket come framework principale.
Tuttavia, posso mostrarti un esempio molto simplificato di come potresti iniziare a strutturare il tuo progetto utilizzando Diesel con SQLite3 e una libreria esterna per la definizione dei servizi web. Per
brevità, utilizzeremo `axum`, che è uno dei framework più popolari per creare servizi web in Rust.
### Prerequisiti
1. Installa Rust: https://www.rust-lang.org/tools/install
2. Crea un nuovo progetto con Cargo:
```sh
cargo new rust_sqlite_webserver
cd rust_sqlite_webserver
```
3. Aggiungi le dipendenze al file `Cargo.toml`:
```toml
[dependencies]
axum = "0.6"
tokio = { version = "1", features = ["full"] }
diesel = { version = "2.0", features = ["sqlite"] }
dotenv = "0.15"
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
[build-dependencies]
diesel_cli = { version = "2.0", features = ["sqlite"] }
[[bin]]
name = "main"
path = "src/main.rs"
```
### Schema di Database
Creiamo un semplice schema per un database SQLite3 con Diesel.
1. Crea il file `schema.rs` in una nuova directory `src/schema/`.
```rust
// src/schema/users.sql
CREATE TABLE users (
id SERIAL PRIMARY KEY,
name VARCHAR(255) NOT NULL,
email TEXT NOT NULL UNIQUE
);
```
2. Genera i modelli Diesel:
```sh
cargo run --bin diesel_cli setup
cargo run --bin diesel_cli migrate
```
### Creazione del Servizio Web con Axum
1. Crea un file `main.rs` nella directory `src/`.
```rust
use axum::{
routing::get,
Router,
};
use diesel::prelude::*;
use serde::{Deserialize, Serialize};
use std::env;
mod schema;
mod models;
// Importiamo i modelli generati da Diesel
use self::schema::users::dsl::*;
#[derive(Serialize)]
struct User {
id: i32,
name: String,
email: String,
}
impl From<models::User> for User {
fn from(user: models::User) -> Self {
User {
id: user.id,
name: user.name,
email: user.email,
}
}
}
#[derive(Deserialize, Insertable)]
#[diesel(table_name = users)]
struct NewUser {
name: String,
email: String,
}
async fn get_users() -> impl axum::response::IntoResponse {
let conn = establish_connection();
let results = users.load::<models::User>(&conn).expect("Error loading users");
(axum::http::StatusCode::OK, axum::Json(results.into_iter().map(User::from).collect()))
}
async fn create_user(user: axum::Json<NewUser>) -> impl axum::response::IntoResponse {
let conn = establish_connection();
let new_user = user.0;
diesel::insert_into(users)
.values(&new_user)
.get_result::<models::User>(&conn)
.expect("Error creating new user");
(axum::http::StatusCode::CREATED, axum::Json(User::from(new_user)))
}
fn establish_connection() -> SqliteConnection {
dotenv().ok();
let database_url = env::var("DATABASE_URL").expect("DATABASE_URL must be set");
SqliteConnection::establish(&database_url).expect(&format!("Error connecting to {}", database_url))
}
#[tokio::main]
async fn main() {
// Set the `RUST_LOG` environment variable if you want to see logs
env_logger::init();
let app = Router::new()
.route("/users", get(get_users).post(create_user));
axum::Server::bind(&"0.0.0.0:3000".parse().unwrap())
.serve(app.into_make_service())
.await
.unwrap();
}
```
### Esecuzione del Progetto
1. Crea un file `.env` nella directory root con la seguente riga:
```sh
DATABASE_URL=sqlite://db.sqlite3
```
2. Esegui il progetto:
```sh
cargo run
```
Ora, puoi accedere al tuo servizio web:
- Per ottenere tutti gli utenti: `GET http://localhost:3000/users`
- Per creare un nuovo utente: `POST http://localhost:3000/users` con il corpo JSON:
```json
{
"name": "John Doe",
"email": "john.doe@example.com"
}
```
Questo esempio mostra come iniziare a strutturare un servizio web in Rust utilizzando Axum, Diesel con SQLite3. Ricorda che questo è solo un punto di partenza e potrebbe essere esteso o modificato per
soddisfare le tue specifiche esigenze.
>>> il mio progetto prevede vari file binari, oltre al webservice come posso gestire piu' binari?
Se il tuo progetto in Rust contiene diversi file binari (ad esempio, un client e un server separati), puoi organizzarli in moduli all'interno del tuo progetto. Ogni modulo può essere compilato come un file
binario separato.
Ecco come farlo:
1. **Struttura del Progetto**:
Organizza il tuo progetto in una directory `src` con sottodirectory per i vari binari e moduli. Ad esempio:
```
rust_project/
├── src/
│ ├── main.rs
│ ├── client/
│ │ └── main.rs
│ ├── server/
│ │ └── main.rs
│ └── common/
│ └── lib.rs
├── Cargo.toml
└── .env
```
2. **Creazione dei Binari**:
Ogni sottodirectory con un file `main.rs` può essere considerata un binario separato.
3. **Cargo.toml**:
Configura il tuo `Cargo.toml` per gestire i vari binari.
```toml
[package]
name = "rust_project"
version = "0.1.0"
edition = "2018"
[dependencies]
axum = "0.6"
tokio = { version = "1", features = ["full"] }
diesel = { version = "2.0", features = ["sqlite"] }
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
[build-dependencies]
diesel_cli = { version = "2.0", features = ["sqlite"] }
[[bin]]
name = "server"
path = "src/server/main.rs"
[[bin]]
name = "client"
path = "src/client/main.rs"
```
4. **Esecuzione dei Binari**:
Puoi ora compilare e eseguire i binari separati.
```sh
cargo build --release
./target/release/server
./target/release/client
```
5. **Moduli Comuni**:
Se hai moduli comuni che vengono utilizzati da più binari, puoi organizzarli in una directory `common` e importarli nei vari binari.
```rust
// src/common/lib.rs
pub mod common_module {
pub fn common_function() {
println!("This is a common function");
}
}
```
Allora, puoi importare questo modulo in altri binari:
```rust
// src/server/main.rs
extern crate rust_project;
use rust_project::common::lib as common;
fn main() {
common::common_function();
}
```
6. **File `.env`**:
Puoi avere un file `.env` nella directory root del progetto per gestire le variabili d'ambiente comuni, e poi specificare variabili d'ambiente separate per ogni binario in sottodirectory.
```sh
# .env
DATABASE_URL=sqlite://db.sqlite3
# src/server/.env
SERVER_PORT=3000
# src/client/.env
CLIENT_IP=localhost
```
Questo approccio ti permette di mantenere una struttura organizzata per i tuoi progetti multi-binari in Rust, facilitando la gestione e il sviluppo di diverse parti del tuo progetto.

271
update_codebase.txt Normal file
View File

@@ -0,0 +1,271 @@
Per creare un Webservice in Rust che utilizza Axis (un framework per la definizione di servizi web basato su XML Schema) e Diesel (un ORM per database SQLite), dobbiamo seguire alcuni passaggi. Tuttavia, è
importante notare che l'uso diretto di Axis con Diesel non è il modo più comune o consigliato per creare servizi web in Rust. Più spesso si usa Axum o Rocket come framework principale.
Tuttavia, posso mostrarti un esempio molto simplificato di come potresti iniziare a strutturare il tuo progetto utilizzando Diesel con SQLite3 e una libreria esterna per la definizione dei servizi web. Per
brevità, utilizzeremo `axum`, che è uno dei framework più popolari per creare servizi web in Rust.
### Prerequisiti
1. Installa Rust: https://www.rust-lang.org/tools/install
2. Crea un nuovo progetto con Cargo:
```sh
cargo new rust_sqlite_webserver
cd rust_sqlite_webserver
```
3. Aggiungi le dipendenze al file `Cargo.toml`:
```toml
[dependencies]
axum = "0.6"
tokio = { version = "1", features = ["full"] }
diesel = { version = "2.0", features = ["sqlite"] }
dotenv = "0.15"
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
[build-dependencies]
diesel_cli = { version = "2.0", features = ["sqlite"] }
[[bin]]
name = "main"
path = "src/main.rs"
```
### Schema di Database
Creiamo un semplice schema per un database SQLite3 con Diesel.
1. Crea il file `schema.rs` in una nuova directory `src/schema/`.
```rust
// src/schema/users.sql
CREATE TABLE users (
id SERIAL PRIMARY KEY,
name VARCHAR(255) NOT NULL,
email TEXT NOT NULL UNIQUE
);
```
2. Genera i modelli Diesel:
```sh
cargo run --bin diesel_cli setup
cargo run --bin diesel_cli migrate
```
### Creazione del Servizio Web con Axum
1. Crea un file `main.rs` nella directory `src/`.
```rust
use axum::{
routing::get,
Router,
};
use diesel::prelude::*;
use serde::{Deserialize, Serialize};
use std::env;
mod schema;
mod models;
// Importiamo i modelli generati da Diesel
use self::schema::users::dsl::*;
#[derive(Serialize)]
struct User {
id: i32,
name: String,
email: String,
}
impl From<models::User> for User {
fn from(user: models::User) -> Self {
User {
id: user.id,
name: user.name,
email: user.email,
}
}
}
#[derive(Deserialize, Insertable)]
#[diesel(table_name = users)]
struct NewUser {
name: String,
email: String,
}
async fn get_users() -> impl axum::response::IntoResponse {
let conn = establish_connection();
let results = users.load::<models::User>(&conn).expect("Error loading users");
(axum::http::StatusCode::OK, axum::Json(results.into_iter().map(User::from).collect()))
}
async fn create_user(user: axum::Json<NewUser>) -> impl axum::response::IntoResponse {
let conn = establish_connection();
let new_user = user.0;
diesel::insert_into(users)
.values(&new_user)
.get_result::<models::User>(&conn)
.expect("Error creating new user");
(axum::http::StatusCode::CREATED, axum::Json(User::from(new_user)))
}
fn establish_connection() -> SqliteConnection {
dotenv().ok();
let database_url = env::var("DATABASE_URL").expect("DATABASE_URL must be set");
SqliteConnection::establish(&database_url).expect(&format!("Error connecting to {}", database_url))
}
#[tokio::main]
async fn main() {
// Set the `RUST_LOG` environment variable if you want to see logs
env_logger::init();
let app = Router::new()
.route("/users", get(get_users).post(create_user));
axum::Server::bind(&"0.0.0.0:3000".parse().unwrap())
.serve(app.into_make_service())
.await
.unwrap();
}
```
### Esecuzione del Progetto
1. Crea un file `.env` nella directory root con la seguente riga:
```sh
DATABASE_URL=sqlite://db.sqlite3
```
2. Esegui il progetto:
```sh
cargo run
```
Ora, puoi accedere al tuo servizio web:
- Per ottenere tutti gli utenti: `GET http://localhost:3000/users`
- Per creare un nuovo utente: `POST http://localhost:3000/users` con il corpo JSON:
```json
{
"name": "John Doe",
"email": "john.doe@example.com"
}
```
Questo esempio mostra come iniziare a strutturare un servizio web in Rust utilizzando Axum, Diesel con SQLite3. Ricorda che questo è solo un punto di partenza e potrebbe essere esteso o modificato per
soddisfare le tue specifiche esigenze.
>>> il mio progetto prevede vari file binari, oltre al webservice come posso gestire piu' binari?
Se il tuo progetto in Rust contiene diversi file binari (ad esempio, un client e un server separati), puoi organizzarli in moduli all'interno del tuo progetto. Ogni modulo può essere compilato come un file
binario separato.
Ecco come farlo:
1. **Struttura del Progetto**:
Organizza il tuo progetto in una directory `src` con sottodirectory per i vari binari e moduli. Ad esempio:
```
rust_project/
├── src/
│ ├── main.rs
│ ├── client/
│ │ └── main.rs
│ ├── server/
│ │ └── main.rs
│ └── common/
│ └── lib.rs
├── Cargo.toml
└── .env
```
2. **Creazione dei Binari**:
Ogni sottodirectory con un file `main.rs` può essere considerata un binario separato.
3. **Cargo.toml**:
Configura il tuo `Cargo.toml` per gestire i vari binari.
```toml
[package]
name = "rust_project"
version = "0.1.0"
edition = "2018"
[dependencies]
axum = "0.6"
tokio = { version = "1", features = ["full"] }
diesel = { version = "2.0", features = ["sqlite"] }
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
[build-dependencies]
diesel_cli = { version = "2.0", features = ["sqlite"] }
[[bin]]
name = "server"
path = "src/server/main.rs"
[[bin]]
name = "client"
path = "src/client/main.rs"
```
4. **Esecuzione dei Binari**:
Puoi ora compilare e eseguire i binari separati.
```sh
cargo build --release
./target/release/server
./target/release/client
```
5. **Moduli Comuni**:
Se hai moduli comuni che vengono utilizzati da più binari, puoi organizzarli in una directory `common` e importarli nei vari binari.
```rust
// src/common/lib.rs
pub mod common_module {
pub fn common_function() {
println!("This is a common function");
}
}
```
Allora, puoi importare questo modulo in altri binari:
```rust
// src/server/main.rs
extern crate rust_project;
use rust_project::common::lib as common;
fn main() {
common::common_function();
}
```
6. **File `.env`**:
Puoi avere un file `.env` nella directory root del progetto per gestire le variabili d'ambiente comuni, e poi specificare variabili d'ambiente separate per ogni binario in sottodirectory.
```sh
# .env
DATABASE_URL=sqlite://db.sqlite3
# src/server/.env
SERVER_PORT=3000
# src/client/.env
CLIENT_IP=localhost
```
Questo approccio ti permette di mantenere una struttura organizzata per i tuoi progetti multi-binari in Rust, facilitando la gestione e il sviluppo di diverse parti del tuo progetto.

1
valid_txs Normal file
View File

@@ -0,0 +1 @@
4df113c0d6c4f31cb01841edb1ce4434ec45bff78eef0b9fcf96129b0b21bb30 : 1752650277 : 1710573876