Troubleshooting

Two problems account for most futures support tickets: the matching engine is not running where you assume it is, and a market has no depth. Start with those.

Nothing is ever liquidated

The most expensive failure, and the quietest: everything looks healthy. Markets load, orders fill, positions open. They just never end.

Cause. The mark sweep runs only on the process holding the futures-matching lease, and nothing else closes a losing position. If no process claimed it, or the process that did is not the one you think, positions sail past their liquidation price indefinitely.

How it happens. The engine boots at start-up only when the futures extension is enabled and the process is the main thread. A dedicated cron process (CRON_MODE=only) is structurally refused the lease, so the 60-second sweepFuturesPositions job returns at its leader check and does nothing at all. If your web process did not boot an engine, there is no leader anywhere.

Quick test. Open a small position on a test market, move the price against it past entry × (1 − 0.9 ÷ leverage), and watch. If the position is still OPEN five seconds later, no leader is sweeping.

Every futures call returns 503

{ "message": "This feature requires the Ecosystem extension." }

The Ecosystem addon is not installed, not enabled, or failed to load. Futures imports its wallet ledger, its fixed-point maths and its Scylla client from Ecosystem; there is no fallback. Fix Ecosystem first — see Install.

A 503 from position or order queries specifically, with Ecosystem clearly present, points at Scylla instead: the client could not be constructed. Check SCYLLA_ENABLED, SCYLLA_CONNECT_POINTS, and that port 9042 is reachable from the app box.

Orders will not place

Every rejection carries its real status code and names the thing to change. Match the message:

A position shows OPEN after the trader was paid

Expected, briefly, and it must not be corrected by hand.

Every settlement path here is MySQL-first: the wallet is credited inside a MySQL transaction, and only then is the Scylla row flipped. If the Scylla write fails, the money is already right and the row still reads OPEN. A loud error is logged and reconcileFuturesPositions replays the status write within five minutes.

Search the log for the idempotency key in the error line — futures_close_…, futures_exit_…, futures_liquidation_… — and confirm the reconciler picked it up on its next run. Editing the Scylla row yourself risks a second payout on replay.

FUTURES_SHORTFALL in the log

FUTURES_SHORTFALL  The book could not absorb a liquidation; … was settled against
the mark instead. … shortfall=812.34 USDT

A liquidation could not be traded out, so the position was settled against the mark and the platform absorbed the difference. There is no insurance fund.

This is a configuration signal, not a bug. The fix is upstream:

• Lower the top leverage rung on that market — at 100x the liquidation band is a 0.9% move, and there must be resting depth inside it.
• Set a real cost.max so a single position cannot exceed what the book can take.
• Improve depth on the market, or stop offering it.

Monitor for this string. A run of them across one symbol means that market's leverage is writing cheques its order book cannot cash.

Every position on a market liquidated at once

Almost always a mark price problem. Positions are marked against your own last traded price — there is no external index. One print at an absurd price becomes the mark for the whole symbol.

Defences, all in the market's metadata: price.min and price.max, amount.max, cost.max. Set them. See Funding and the mark price.

The engine does protect against one related case: a market whose last price is zero, because nothing has ever traded, is skipped by the sweep rather than marked at zero.

Positions exist on a market that is gone

You deleted the market instead of disabling it.

Deletion is a forced hard delete with no restore, and it does not touch the position table. The surviving positions have no ticker, so the sweep skips them and they can never be stopped out or liquidated. Traders can still close them manually — at the entry price, since there is no mark — but nothing else will.

Recreate the market with the same currency and pair to restore a ticker, then let the positions drain, then disable it. And from now on: switch markets off, delete only markets that never traded.

Leverage shows as 0.00x

A display bug in whatever is rendering it. leverage is stored as a plain integer varint — a 10x position stores 10, not 10e18 — so de-scaling it by 10^18 yields 1e-17. The admin position and order tables both had this and both were fixed; a custom table or an integration may still have it.

Traders ask why their long was not closed by their sell

Hedge mode. A sell placed while holding a long opens a second, independently margined short; it does not reduce the long. Both positions post their own margin and both can be liquidated. The position API marks every row mode: "HEDGE" so the interface can say so — make sure your own help text does too.

Liquidation emails never arrive

The engine queues two templates by name: LiquidationNotification and PartialLiquidationNotification. Both are looked up in System → Notifications → Templates, and a missing or disabled row means the send fails with a "template not found" error and the trader hears nothing.

Check both exist and are enabled. Note also that no pre-liquidation warning email is sent — the template name exists in the code but nothing calls it.

The dashboard totals look wrong

Check the truncated indicator on the page. The position table is partitioned by user id, so the dashboard has to scan, and the scan stops at FUTURES_DASHBOARD_SCAN_CAP (50,000 rows by default). When it stops early the page says so and reports how many rows it saw. That is a sample, not a total — raise the cap and accept the read cost, or read it as a sample.

Two more things that are not bugs: the positions and orders tables can only produce row counts, because the Scylla aggregator understands nothing but "count rows where a column equals a literal"; and unrealizedPnl on a closed row is frozen at whatever the last mark was, so it must never be summed across statuses.