Compare commits
4 Commits
5e18a2e06c
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
|
6690343aad
|
|||
|
c957dfff6b
|
|||
|
8e5b921771
|
|||
|
f106beeb62
|
@@ -112,6 +112,7 @@ The `bal-server` application can be configured using environment variables.
|
||||
| `BAL_SERVER_ACTIX_DEFAULT_BURST` | Rate limit: default burst size. | `100` |
|
||||
| `BAL_SERVER_ACTIX_WORKERS` | Number of Actix worker threads. | `4` |
|
||||
| `BAL_SERVER_ACTIX_MAX_CONNECTIONS` | Maximum concurrent connections. | `100` |
|
||||
| `BAL_SERVER_TRUSTED_PROXY` | Trusted reverse proxy IP for rate-limiting client identification. | `127.0.0.1` |
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -117,7 +117,7 @@ The main application binary that provides an async HTTP server.
|
||||
|
||||
### Architecture
|
||||
- **Runtime:** `actix-web 4.9.0` with `actix-rt` (`#[actix_web::main]`).
|
||||
- **Rate Limiting:** `actix-governor` middleware with token-bucket algorithm per endpoint.
|
||||
- **Rate Limiting:** `actix-governor` middleware with token-bucket algorithm. Uses `RealIpKeyExtractor` to identify clients by real IP behind reverse proxy (via `X-Real-IP` / `X-Forwarded-For` headers).
|
||||
- **Response Compression:** `actix_web::middleware::Compress`.
|
||||
- **Request Logging:** `actix_web::middleware::Logger::default()`.
|
||||
- **Shared State:** `Arc<Mutex<Connection>>` for database access, `MyConfig` for configuration.
|
||||
@@ -132,7 +132,7 @@ The main application binary that provides an async HTTP server.
|
||||
- `address` (xpub or address), `fixed_fee` (sats), `xpub` (bool), `network` (bitcoin::Network), `name`, `enabled`
|
||||
|
||||
**`ActixConfig`** (server tuning):
|
||||
- `max_body_size`, `timeout_secs`, per-endpoint rate limits (`pushtxs`, `searchtx`, `info`, `default`), `workers`, `max_connections`
|
||||
- `max_body_size`, `timeout_secs`, rate limits (`pushtxs` per sec/burst), `workers`, `max_connections`, `trusted_proxy`
|
||||
|
||||
### Key Routes
|
||||
| Method | Path | Handler | Description |
|
||||
|
||||
@@ -10,16 +10,11 @@
|
||||
|
||||
### Rate Limiting
|
||||
|
||||
All endpoints are rate-limited via `actix-governor` with a token-bucket algorithm. Defaults:
|
||||
All endpoints are rate-limited via `actix-governor` with a token-bucket algorithm. The rate limit key is the **real client IP address**, extracted from `X-Real-IP` / `X-Forwarded-For` headers when the request comes from a trusted proxy (default `127.0.0.1`, configurable via `BAL_SERVER_TRUSTED_PROXY`). Direct connections (non-proxy) use the TCP peer IP.
|
||||
|
||||
| Endpoint | Rate (req/s) | Burst |
|
||||
|----------|-------------|-------|
|
||||
| `POST /{network}/pushtxs` | 1 | 3 |
|
||||
| `POST /searchtx` | 5 | 10 |
|
||||
| `GET /{network}/info` | 20 | 30 |
|
||||
| All others | 50 | 100 |
|
||||
Default: 1 req/s with burst of 3 (configurable via `BAL_SERVER_ACTIX_PUSHTXS_PER_SEC` / `BAL_SERVER_ACTIX_PUSHTXS_BURST`).
|
||||
|
||||
Rate limits are configurable via `BAL_SERVER_ACTIX_*` environment variables.
|
||||
When behind Nginx, ensure `proxy_set_header X-Real-IP $remote_addr` is set so the server can identify individual clients.
|
||||
|
||||
### `GET /`
|
||||
- **Description:** Returns a static identification string (default: "Will Executor Server").
|
||||
|
||||
@@ -40,8 +40,9 @@ Example: `BAL_SERVER_REGTEST_ADDRESS=tpub...`, `BAL_SERVER_BITCOIN_FIXED_FEE=500
|
||||
| `BAL_SERVER_ACTIX_TIMEOUT_SECS` | `5` | Request timeout in seconds |
|
||||
| `BAL_SERVER_ACTIX_WORKERS` | `4` | Number of actix-web worker threads |
|
||||
| `BAL_SERVER_ACTIX_MAX_CONNECTIONS` | `100` | Maximum concurrent connections |
|
||||
| `BAL_SERVER_ACTIX_PUSHTXS_PER_SEC` | `1` | Rate limit: pushtxs requests per second |
|
||||
| `BAL_SERVER_ACTIX_PUSHTXS_BURST` | `3` | Rate limit: pushtxs burst size |
|
||||
| `BAL_SERVER_TRUSTED_PROXY` | `127.0.0.1` | Trusted reverse proxy IP for rate-limiting client identification |
|
||||
| `BAL_SERVER_ACTIX_PUSHTXS_PER_SEC` | `1` | Rate limit: requests per second (applied to all endpoints) |
|
||||
| `BAL_SERVER_ACTIX_PUSHTXS_BURST` | `3` | Rate limit: burst size (applied to all endpoints) |
|
||||
| `BAL_SERVER_ACTIX_SEARCHTX_PER_SEC` | `5` | Rate limit: searchtx requests per second |
|
||||
| `BAL_SERVER_ACTIX_SEARCHTX_BURST` | `10` | Rate limit: searchtx burst size |
|
||||
| `BAL_SERVER_ACTIX_INFO_PER_SEC` | `20` | Rate limit: info requests per second |
|
||||
|
||||
@@ -52,7 +52,7 @@
|
||||
**Description:** All DoS vectors mitigated via actix-web migration.
|
||||
**Mitigation Applied:**
|
||||
- Body size limit: `PayloadConfig::default().limit(max_body_size)` via `BAL_SERVER_ACTIX_MAX_BODY_SIZE` (default 1 MiB).
|
||||
- Rate limiting: `actix-governor` with token-bucket per endpoint (`BAL_SERVER_ACTIX_PUSHTXS_PER_SEC`/`BURST`).
|
||||
- Rate limiting: `actix-governor` with token-bucket per client IP (`BAL_SERVER_ACTIX_PUSHTXS_PER_SEC`/`BURST`). Uses `RealIpKeyExtractor` to extract real client IP from proxy headers.
|
||||
- Connection limits: `workers(4)` and `max_connections(100)` via `BAL_SERVER_ACTIX_WORKERS`/`MAX_CONNECTIONS`.
|
||||
- Body timeout: configurable via `BAL_SERVER_ACTIX_TIMEOUT_SECS`.
|
||||
- ZMQ timeout: `set_rcvtimeo(5000)` prevents infinite blocking.
|
||||
@@ -130,7 +130,7 @@
|
||||
6. Use read-only filesystem for the server binary.
|
||||
|
||||
### Application-Level
|
||||
1. **Rate Limiting:** Implemented via `actix-governor` with per-endpoint token-bucket configuration.
|
||||
1. **Rate Limiting:** Implemented via `actix-governor` with token-bucket per client IP. `RealIpKeyExtractor` identifies clients behind reverse proxy using `X-Real-IP` / `X-Forwarded-For` headers. Trusted proxy IP configurable via `BAL_SERVER_TRUSTED_PROXY`.
|
||||
2. **Input Validation:** Network enum check, txid hex validation, body size limits.
|
||||
3. **HTTPS:** Via Nginx reverse proxy with Let's Encrypt.
|
||||
4. **WAL Mode:** Enabled with retry logic for concurrent access.
|
||||
|
||||
@@ -24,7 +24,8 @@ use reqwest::Client as rClient;
|
||||
use std::net::SocketAddr;
|
||||
use url::Url;
|
||||
|
||||
const LOCKTIME_THRESHOLD: i64 = 5000000;
|
||||
// BIP-65: locktime values below this are block heights, at or above are UNIX timestamps.
|
||||
const LOCKTIME_THRESHOLD: i64 = 500_000_000;
|
||||
const VERSION: &str = env!("CARGO_PKG_VERSION");
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
struct MyConfig {
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
use actix_governor::{Governor, GovernorConfigBuilder};
|
||||
use actix_governor::{Governor, GovernorConfigBuilder, KeyExtractor, SimpleKeyExtractionError};
|
||||
use actix_web::dev::ServiceRequest;
|
||||
use actix_web::middleware;
|
||||
use actix_web::web::Bytes;
|
||||
use actix_web::{App, HttpResponse, HttpServer, Responder, web};
|
||||
@@ -10,6 +11,8 @@ use serde::{Deserialize, Serialize};
|
||||
use std::collections::{HashMap, HashSet};
|
||||
use std::env;
|
||||
use std::fs;
|
||||
use std::net::IpAddr;
|
||||
use std::str::FromStr;
|
||||
|
||||
use bal_server::db::{
|
||||
DatabasePool, InsertInpData, InsertOutData, InsertTxData, check_duplicate_txids,
|
||||
@@ -130,6 +133,7 @@ struct ActixConfig {
|
||||
rate_limit_default: (u64, u32),
|
||||
workers: usize,
|
||||
max_connections: usize,
|
||||
trusted_proxy: IpAddr,
|
||||
}
|
||||
|
||||
fn parse_actix_config() -> ActixConfig {
|
||||
@@ -190,6 +194,10 @@ fn parse_actix_config() -> ActixConfig {
|
||||
.unwrap_or("100".to_string())
|
||||
.parse::<usize>()
|
||||
.unwrap_or(100),
|
||||
trusted_proxy: env::var("BAL_SERVER_TRUSTED_PROXY")
|
||||
.unwrap_or_else(|_| "127.0.0.1".to_string())
|
||||
.parse::<IpAddr>()
|
||||
.unwrap_or(IpAddr::from_str("127.0.0.1").unwrap()),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -219,41 +227,70 @@ async fn echo_version() -> impl Responder {
|
||||
HttpResponse::Ok().body(VERSION)
|
||||
}
|
||||
|
||||
fn is_valid_ip(ip: &str) -> bool {
|
||||
ip.parse::<std::net::IpAddr>().is_ok()
|
||||
fn extract_real_ip(req: &actix_web::HttpRequest, trusted_proxy: IpAddr) -> String {
|
||||
let peer_ip = req.peer_addr().map(|socket| socket.ip());
|
||||
let connection_info = req.connection_info();
|
||||
|
||||
let ip = match peer_ip {
|
||||
Some(peer) if peer == trusted_proxy => {
|
||||
connection_info.realip_remote_addr().unwrap_or("unknown")
|
||||
}
|
||||
_ => connection_info.peer_addr().unwrap_or("unknown"),
|
||||
};
|
||||
|
||||
debug!("client IP: {}", ip);
|
||||
ip.to_string()
|
||||
}
|
||||
|
||||
fn extract_client_ip(req: &actix_web::HttpRequest) -> String {
|
||||
if let Some(val) = req.headers().get("X-Real-IP")
|
||||
&& let Ok(s) = val.to_str()
|
||||
{
|
||||
let ip = s.split(',').next().unwrap_or(s).trim();
|
||||
if is_valid_ip(ip) {
|
||||
debug!("client IP from X-Real-IP: {}", ip);
|
||||
return ip.to_string();
|
||||
}
|
||||
}
|
||||
if let Some(val) = req.headers().get("X-Forwarded-For")
|
||||
&& let Ok(s) = val.to_str()
|
||||
{
|
||||
let ip = s.split(',').next().unwrap_or(s).trim();
|
||||
if is_valid_ip(ip) {
|
||||
debug!("client IP from X-Forwarded-For: {}", ip);
|
||||
return ip.to_string();
|
||||
}
|
||||
}
|
||||
let fallback = req
|
||||
.connection_info()
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
struct RealIpKeyExtractor;
|
||||
|
||||
impl KeyExtractor for RealIpKeyExtractor {
|
||||
type Key = IpAddr;
|
||||
type KeyExtractionError = SimpleKeyExtractionError<&'static str>;
|
||||
|
||||
fn extract(&self, req: &ServiceRequest) -> Result<Self::Key, Self::KeyExtractionError> {
|
||||
let proxy_ip = req
|
||||
.app_data::<web::Data<IpAddr>>()
|
||||
.map(|ip| *ip.get_ref())
|
||||
.unwrap_or_else(|| IpAddr::from_str("0.0.0.0").unwrap());
|
||||
|
||||
let peer_ip = req.peer_addr().map(|socket| socket.ip());
|
||||
let connection_info = req.connection_info();
|
||||
|
||||
match peer_ip {
|
||||
Some(peer) if peer == proxy_ip => connection_info
|
||||
.realip_remote_addr()
|
||||
.ok_or_else(|| {
|
||||
SimpleKeyExtractionError::new("Could not extract real IP address from request")
|
||||
})
|
||||
.and_then(|str| {
|
||||
str.parse::<IpAddr>().map_err(|_| {
|
||||
SimpleKeyExtractionError::new(
|
||||
"Could not extract real IP address from request",
|
||||
)
|
||||
})
|
||||
}),
|
||||
_ => connection_info
|
||||
.peer_addr()
|
||||
.unwrap_or("unknown")
|
||||
.to_string();
|
||||
debug!("client IP from peer_addr fallback: {}", fallback);
|
||||
fallback
|
||||
.ok_or_else(|| {
|
||||
SimpleKeyExtractionError::new("Could not extract peer IP address from request")
|
||||
})
|
||||
.and_then(|str| {
|
||||
str.parse::<IpAddr>().map_err(|_| {
|
||||
SimpleKeyExtractionError::new(
|
||||
"Could not extract peer IP address from request",
|
||||
)
|
||||
})
|
||||
}),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async fn echo_info(
|
||||
path: web::Path<String>,
|
||||
data: web::Data<AppState>,
|
||||
proxy: web::Data<IpAddr>,
|
||||
req: actix_web::HttpRequest,
|
||||
) -> impl Responder {
|
||||
let param = path.into_inner();
|
||||
@@ -266,7 +303,7 @@ async fn echo_info(
|
||||
debug!("network disabled {}", param);
|
||||
return HttpResponse::BadRequest().body("error");
|
||||
}
|
||||
let remote_addr = extract_client_ip(&req);
|
||||
let remote_addr = extract_real_ip(&req, **proxy);
|
||||
let address = match netconfig.xpub {
|
||||
false => {
|
||||
let address = netconfig.address.to_string();
|
||||
@@ -776,9 +813,10 @@ async fn main() -> std::io::Result<()> {
|
||||
let bind_address = data.cfg.bind_address.clone();
|
||||
let bind_port = data.cfg.bind_port;
|
||||
|
||||
let governor_conf = GovernorConfigBuilder::const_default()
|
||||
let governor_conf = GovernorConfigBuilder::default()
|
||||
.seconds_per_request(actix_cfg.rate_limit_pushtxs.0)
|
||||
.burst_size(actix_cfg.rate_limit_pushtxs.1)
|
||||
.key_extractor(RealIpKeyExtractor)
|
||||
.finish()
|
||||
.unwrap();
|
||||
|
||||
@@ -788,6 +826,7 @@ async fn main() -> std::io::Result<()> {
|
||||
App::new()
|
||||
.app_data(web::PayloadConfig::default().limit(actix_cfg.max_body_size))
|
||||
.app_data(data.clone())
|
||||
.app_data(web::Data::new(actix_cfg.trusted_proxy))
|
||||
.wrap(middleware::Logger::default())
|
||||
.wrap(middleware::Compress::default())
|
||||
.wrap(Governor::new(&governor_conf))
|
||||
|
||||
@@ -624,9 +624,9 @@ pub async fn get_pending_txs(
|
||||
"SELECT txid, tx, locktime, network, status FROM tbl_tx WHERE network = $1 AND status = 0 AND (locktime < $2 OR (locktime > $3 AND locktime < $4))",
|
||||
)
|
||||
.bind(network)
|
||||
.bind(bestblock_height as i32)
|
||||
.bind(locktime_threshold as i32)
|
||||
.bind(bestblock_time as i32)
|
||||
.bind(bestblock_height)
|
||||
.bind(locktime_threshold)
|
||||
.bind(bestblock_time)
|
||||
.fetch_all(p)
|
||||
.await?;
|
||||
|
||||
@@ -634,7 +634,7 @@ pub async fn get_pending_txs(
|
||||
results.push(TxRow {
|
||||
txid: row.try_get("txid")?,
|
||||
tx: row.try_get("tx")?,
|
||||
locktime: row.try_get::<i32, _>("locktime").unwrap_or(0) as i64,
|
||||
locktime: row.try_get::<i64, _>("locktime").unwrap_or(0),
|
||||
network: row.try_get("network")?,
|
||||
status: row.try_get::<i32, _>("status").unwrap_or(0) as i64,
|
||||
});
|
||||
@@ -839,9 +839,9 @@ pub async fn calculate_and_upsert_stats(
|
||||
(SELECT COUNT(*) FROM tbl_tx WHERE status = 0 AND network = $1),
|
||||
(SELECT COUNT(*) FROM tbl_tx WHERE status = 1 AND network = $1),
|
||||
(SELECT COUNT(*) FROM tbl_tx WHERE status = 2 AND network = $1),
|
||||
(SELECT COALESCE(SUM(our_fees),0) FROM tbl_tx WHERE status = 0 AND network = $1),
|
||||
(SELECT COALESCE(SUM(our_fees),0) FROM tbl_tx WHERE status = 1 AND network = $1),
|
||||
(SELECT COALESCE(SUM(our_fees),0) FROM tbl_tx WHERE status = 2 AND network = $1),
|
||||
(SELECT COALESCE(SUM(CAST(our_fees AS BIGINT)),0) FROM tbl_tx WHERE status = 0 AND network = $1),
|
||||
(SELECT COALESCE(SUM(CAST(our_fees AS BIGINT)),0) FROM tbl_tx WHERE status = 1 AND network = $1),
|
||||
(SELECT COALESCE(SUM(CAST(our_fees AS BIGINT)),0) FROM tbl_tx WHERE status = 2 AND network = $1),
|
||||
(SELECT COUNT(DISTINCT tbl_inp.in_txid)
|
||||
FROM tbl_inp
|
||||
JOIN tbl_tx ON tbl_inp.txid = tbl_tx.txid
|
||||
|
||||
@@ -128,7 +128,7 @@ pub async fn create_pg_schema(pool: &PgPool) -> Result<(), sqlx::Error> {
|
||||
wtxid TEXT,
|
||||
ntxid TEXT,
|
||||
tx TEXT,
|
||||
locktime INTEGER,
|
||||
locktime BIGINT,
|
||||
network TEXT,
|
||||
network_fees TEXT,
|
||||
reqid TEXT,
|
||||
@@ -141,6 +141,13 @@ pub async fn create_pg_schema(pool: &PgPool) -> Result<(), sqlx::Error> {
|
||||
.execute(pool)
|
||||
.await?;
|
||||
|
||||
// Migrate pre-existing deployments where locktime was created as INTEGER.
|
||||
// nLockTime is a u32 (up to 4_294_967_295); INTEGER (i32) cannot hold
|
||||
// timestamp-based locktimes after 2038-01-19.
|
||||
let _ = sqlx::query("ALTER TABLE tbl_tx ALTER COLUMN locktime TYPE BIGINT")
|
||||
.execute(pool)
|
||||
.await;
|
||||
|
||||
sqlx::query(
|
||||
"CREATE TABLE IF NOT EXISTS tbl_inp (
|
||||
id SERIAL PRIMARY KEY,
|
||||
|
||||
@@ -148,12 +148,12 @@ pub fn calculate_fingerprint(tpub: &str) -> Result<String, String> {
|
||||
fn base58check_decode(s: &str) -> Result<Vec<u8>, String> {
|
||||
let data = bs58::decode(s).into_vec().map_err(|e| e.to_string())?;
|
||||
if data.len() < 4 {
|
||||
return Err("Data troppo corta".to_string());
|
||||
return Err("Data too short".to_string());
|
||||
}
|
||||
let (payload, checksum) = data.split_at(data.len() - 4);
|
||||
let hash = Sha256::digest(Sha256::digest(payload));
|
||||
if hash[0..4] != checksum[..] {
|
||||
return Err("Checksum invalido".to_string());
|
||||
return Err("Invalid checksum".to_string());
|
||||
}
|
||||
Ok(payload.to_vec())
|
||||
}
|
||||
@@ -168,7 +168,7 @@ fn convert_to(zpub: &str, prefix: BS58Prefix) -> Result<String, String> {
|
||||
let mut data = base58check_decode(zpub)?;
|
||||
|
||||
if data.len() < 4 {
|
||||
return Err("Non è una zpub valida.".to_string());
|
||||
return Err("Not a valid zpub".to_string());
|
||||
}
|
||||
data.splice(
|
||||
0..4,
|
||||
@@ -207,7 +207,7 @@ pub fn new_address_from_xpub(
|
||||
fn main() -> Result<(), Box<dyn std::error::Error>>{
|
||||
match convert_to(zpub,BS58Prefix::Tpub) {
|
||||
Ok(tpub) => println!("XPUB: {}", tpub),
|
||||
Err(e) => eprintln!("Errore: {}", e),
|
||||
Err(e) => eprintln!("Error: {}", e),
|
||||
}
|
||||
let fingerprint = base58check_encode(&calculate_fingerprint(zpub));
|
||||
println!("ZPUB: {}, FINGERPRINT: {}",zpub,fingerprint);
|
||||
|
||||
Reference in New Issue
Block a user