Engine and storage

The futures addon is one long-lived component and one keyspace. This page describes both, because most operational problems here are one of two things: the engine is not running where you think it is, or Scylla is not answering.

The matching engine

FuturesMatchingEngine is a single instance per process, created at boot on the process that is allowed to own it. It holds the order queue, the per-symbol ticker and the candle map, and it drives matching.

The leadership lease

The engine claims a lease named futures-matching. Exactly one process in a deployment holds it.

• The leader owns the order queue, runs matching cycles, re-marks positions, enforces stop-loss and take-profit, and liquidates.
• A follower serves getTicker and getTickers from the candle tables and nothing else. No queue, no boot-time re-mark (which writes), no mark sweep, and every mutating call refused.

The backend starts the engine at boot when the futures extension is enabled and it is the main thread. It used to arrive only through a lazy first request, and under a split cron deployment that meant nothing held the lease at all and no leveraged position was marked anywhere until somebody hit a futures endpoint.

The mark sweep

Every two seconds, the leader re-marks every open position against its symbol's live ticker.

For each position, in order: check the trader's stop loss and take profit, re-read the position (the exit may have just closed it), then evaluate liquidation. Each position is isolated in its own error handling — one bad row never stops the sweep for everyone else.

Two guards worth knowing:

• One ticker read per symbol, not per position. A busy desk holds many positions on the same market and the ticker cannot change between them.
• A symbol whose last price is zero is skipped. A market that has never traded would otherwise read as a 100% adverse move and liquidate every position on it.

The sweep is single-flight across callers: the 2-second timer and the 60-second cron backstop share one running pass rather than overlapping. A later caller joins the running sweep instead of queueing behind it, because "a sweep has happened recently" is satisfied by one happening right now.

Matching

Matching is serialised — two placements landing in the same tick cannot walk the same book snapshot. A cycle that triggers risk checks queues them for after the pass has persisted, so a liquidation can place an order and have it matched by the next pass rather than deadlocking inside the one it was called from. A single driver invocation runs at most 16 back-to-back passes before yielding.

Only LIMIT and MARKET orders that are OPEN with a non-zero remainder are matchable. Anything else is filtered out before the loop.

The cron jobs

Three jobs register under System → Cron, category futures.

Job	Period	Writes?
sweepFuturesPositions	60s	Yes — closes and liquidates positions
reconcileFuturesPositions	5 min	Scylla only; never touches wallets
reconcileFuturesOrders	5 min	Scylla only; never touches wallets

The sweep cron is a backstop, not the clock. On a dedicated cron process it is a structural no-op, because that process cannot hold the matching lease and the sweep returns at its leader check. Inline, it shares a realm with the engine's own 2-second timer and the single-flight guard keeps the two from overlapping.

The reconcilers exist because every settlement path is MySQL-first. The trader's wallet is credited inside a MySQL transaction, and only then is the Scylla row flipped. If the Scylla write fails after retries, the money is already right and the row still shows OPEN. The wallet credit leaves a durable breadcrumb in the transaction table — an idempotency key — and that is what the reconciler keys off to replay the status write.

Keys used, so you can recognise them in the transaction metadata:

Prefix	Written by
futures_order_<orderId>	The placement debit
futures_order_<orderId>_cancel	A cancellation refund
futures_order_<orderId>_slippage_refund_<fill>	Releasing an over-hold on a fill
futures_position_<positionId>_close	A manual close
futures_exit_<positionId>	A stop-loss or take-profit close
futures_liquidation_<positionId>	A full liquidation settled against the mark
futures_liq_fill_<orderId>_<fill>	A reduce-only liquidation fill

Reconciled rows are stamped so later ticks do not re-scan them, the scan window is rolling, and work is capped per tick so a backlog drains across ticks instead of being redone.

Storage

MySQL

One table: futures_market. Id, currency, pair, isTrending, isHot, a metadata TEXT column holding JSON, status, and timestamps. The pair is unique. The table supports soft deletes, but the delete route forces a hard one.

This is the only futures data in MySQL — and therefore the only futures data the platform's built-in backup covers.

ScyllaDB

Everything else lives in the keyspace named by SCYLLA_FUTURES_KEYSPACE (default futures), created by the client on first connection along with its tables and views. You never run DDL.

Table	Primary key	Holds
orders	((userId), createdAt, id)	Every order, newest first within a user
position	((userId), id)	Every position, open and historical
orderbook	((symbol, side), price)	Aggregated resting depth per price level
candles	(symbol, interval, createdAt)	The candle series every ticker derives from

Five materialised views support the queries the engine and the admin tables need: open_order, latest_candles, orders_by_symbol, orderbook_by_symbol and positions_by_symbol.

How numbers are stored

Amounts, prices, costs, fees and PnL are VARINT columns holding fixed-point values scaled by 10^18, from the Ecosystem extension's helpers.

leverage is the exception. It is also a VARINT column, but it holds a plain integer: a 10x position stores 10, not 10e18. It is written as a string because the driver refuses a JavaScript BigInt for a varint — and when it did, positions opened from a reloaded order threw, the fill was still recorded, and the trader was debited with no position at all. Anything that de-scales leverage yields 1e-17 and displays as 0.00x.

Environment variables

None of these are in .env.example; you add the ones you need by hand.

Variable	Default	Effect
SCYLLA_ENABLED	true	false disables the connection entirely. The reconcilers read this directly, because a disabled client is not falsy and the import guard cannot see the kill switch.
SCYLLA_CONNECT_POINTS	127.0.0.1:9042	Contact points
SCYLLA_DATACENTER	datacenter1	Local datacenter name
SCYLLA_FUTURES_KEYSPACE	futures	The futures keyspace
SCYLLA_USERNAME / SCYLLA_PASSWORD	unset	Credentials, if the cluster requires them
FUTURES_DASHBOARD_SCAN_CAP	50000	Rows one dashboard scan reads before stopping and reporting truncation. Minimum 1,000.
FUTURES_RECONCILER_WINDOW_HOURS	24	How far back the reconcilers look
FUTURES_RECONCILER_BATCH_SIZE	500	Rows per reconciler tick

SCYLLA_KEYSPACE (default trading) belongs to Ecosystem, not to Futures, but both must be set because the futures code loads Ecosystem's client.

Degraded behaviour

The addon fails closed rather than quietly.

Condition	What happens
Ecosystem not installed	Order placement and position close return 503 with a message naming Ecosystem. The engine logs "Ecosystem extension not available, futures matching engine disabled" and does not start.
Scylla unreachable at boot	The engine boot is non-fatal — the server still starts. The lazy path retries on the next futures request.
Scylla unreachable at request time	Position and order queries return 500; the dashboard reports the store as unavailable rather than showing zeros.
Extension row disabled	Routes disappear, the engine does not boot, and the cron jobs are not registered.
No lease holder	Everything looks healthy and nothing is ever liquidated. See Troubleshooting.