How a Trade Replication Engine Works: The Reconcile Loop

At 03:14 a copier process restarts after a routine host reboot. The provider account holds six open positions. Follower A holds five, because one copy failed during the outage and nobody was awake. Follower B holds seven, because a retry fired twice the previous evening.

What happens in the next second decides whether this is a footnote or an incident. A system built around "when a trade opens, send the order" has no idea any of that is true — it was not listening when those things happened, and it has no mechanism for noticing afterwards. A system built around reconciliation treats the three accounts as three facts to be compared, works out the smallest set of actions that makes them agree, and takes them.

That comparison is the engine. Everything else is plumbing around it. This is how that class of system is put together — the architecture our own copier uses, described at the level of the design rather than the code.

Desired state, not instructions

The mental shift is from messaging to control.

A messaging design says: an event happened, deliver it. Delivery is the promise, and when delivery fails the system has lost information permanently. Every outage leaves a scar you have to find by hand.

A control design says: here is the state the world should be in; make it so. The engine repeatedly computes a desired state for each follower by applying the replication policy to the provider's current state, compares it with the follower's observed state, and emits only the difference.

  provider state ──▶ [ replication policy ] ──▶ desired follower state
                                                        │
  follower state ───────────────────────────────────────┤
                                                        ▼
                                              [ diff ] ──▶ actions
                                                            open
                                                            close
                                                            modify
                                                            cancel

Three properties fall out of this for free, and they are exactly the properties a copier needs.

It is idempotent. Running the loop twice changes nothing the second time, because the second diff is empty. Duplicate opens stop being a class of bug.

It is self-healing. A copy that failed at 02:00 is simply a difference at 03:14, and gets placed. A position opened by a retry that should not exist is a difference in the other direction, and gets closed or flagged.

It has no memory of what it meant to do. There is no queue of pending intentions to lose, corrupt or replay out of order. The truth is always the accounts themselves.

Events and polling are not a choice

People frame this as event-driven versus polling. In practice a serious engine uses both, because they buy different things.

The pattern that works is: the event stream triggers a reconcile; the timer ensures one happens anyway. A trade opening on the provider does not itself send an order — it wakes the loop up early. If the event never arrives, the next scheduled pass finds the difference and acts. Latency degrades from milliseconds to the interval; correctness does not degrade at all.

This also removes a whole category of ordering bugs. Because every pass recomputes from current state, an event that arrives late or out of order cannot cause the wrong action — by the time it is processed, the state it describes is either already reflected or already superseded.

Attribution: which copy belongs to which original

The diff only works if the engine can look at a follower position and say which provider position it represents. That link is attribution, and it needs to exist in two places.

On the broker. MetaTrader gives you a magic number and a comment field; cTrader gives you a label. Writing an identifier there means the link survives even if the engine loses its own records, and it lets a human looking at the terminal see which positions are copies.

On the server. A mapping table holding provider position id, follower account, follower ticket, the policy version that produced it, and the time. This is what survives the broker side being unreliable — and it is unreliable. Comment fields get truncated to a length that varies by broker, and some brokers overwrite comments entirely on partial close or on stop-out, which is precisely when you most need the link.

Attribution is also what makes three other things possible:

• Leaving manual trades alone. A position with no attribution was placed by a human. The engine's desired state should not include it and the engine should never close it.
• Multiple providers into one account. Without a tag, an exit from provider B can close provider A's position.
• Readable history. After the fact, every follower trade points at the provider trade that caused it and the policy that sized it. Without that, reconstructing what happened is guesswork.

Netting and hedging change what "desired state" means

This is where the abstraction has to bend to the platform.

On a hedging account — MT4 always, MT5 optionally, cTrader — the desired state is a set of positions. A provider holding a 1.0 lot long and a 0.4 lot short in EURUSD can be represented faithfully: two positions, two tickets, two attributions.

On a netting account — MT5 optionally, and most futures platforms — an account can hold exactly one net position per symbol. The desired state is a net volume per symbol, and the provider's two positions collapse into a 0.6 lot long. That collapse is lossy and the consequences are real:

• Attribution becomes many-to-one. Two provider positions map to one follower position, so "close the short" means reducing the net position rather than closing anything.
• A partial close on the provider is a volume delta on the follower, computed from the new net, not from the closed ticket.
• Stops and targets cannot be per-original-position, because there is only one position to attach them to. Either the engine manages them itself in software, or the follower runs without the provider's exact levels.

An engine that does not model this explicitly will appear to work on matched demo accounts and then behave inexplicably the first time a netting follower is added. The honest handling is to detect the account's mode at connect time, compute desired state in the right shape for it, and say plainly in the interface which shape each follower is in.

Loop protection: when an account is both

Now the interesting case. An account can legitimately be a provider for one relationship and a follower in another — you copy a colleague's account into yours, and your own account feeds three prop accounts. The routing graph is no longer a star.

Without protection, the engine copies a position into an account, then observes that account as a provider, sees a new position, and copies it onward. Round and round, or outward forever. Three mechanisms stop it, and a serious engine uses all three.

1. Attribution as the primary defence. A position that carries a copy tag is not an origination. When reading an account in its provider role, the engine considers only untagged positions — the ones a human or an expert advisor placed directly. A copy can therefore never be a source. This alone prevents the simple two-account loop.

2. A hop counter as the backstop. Each replicated position carries a depth. An origination is depth 0; a copy of it is depth 1. Copies beyond a configured maximum depth are refused and logged. This catches the cases attribution misses — notably when a broker strips the comment field and the tag is lost on the way in.

3. Cycle detection at configuration time. The routing graph is known before any trade happens. Adding a relationship that creates a cycle should be rejected in the interface, with the cycle named, rather than discovered at 03:00. This is a graph traversal over a handful of nodes and there is no excuse for not doing it.

  legal                          illegal (rejected at save time)

  colleague ──▶ you              you ──▶ prop A
        you ──▶ prop A           prop A ──▶ you
        you ──▶ prop B
        you ──▶ prop C           depth: you(0) ▶ propA(1) ▶ you(2) ▶ ...

Failure handling, ordering and clocks

Three smaller things that separate a reliable engine from a demo.

Partial application. An action set of four orders can succeed twice and fail twice. The engine records each outcome individually and lets the next pass retry the failures. It does not roll back the successes, because in a market there is no rollback — there is only a new position and a new decision.

Backoff and poison handling. A copy that fails for a permanent reason — the symbol does not exist, the account is read-only, the volume is below minimum — must not be retried every 200 milliseconds forever. It gets a small number of attempts and is then parked as a skip with a reason, visible in the interface, until a human or a configuration change clears it.

Sequence, not timestamps. Broker clocks drift, and server time zones differ by hours. Ordering decisions taken on wall-clock time across two brokers will eventually be wrong. Ordering belongs to a monotonic sequence the engine controls.

What the latency budget actually contains

Because the loop is triggered by events, the time between a provider fill and a follower fill is a chain: the provider's terminal or API noticing, the event reaching the engine, the policy computing, the order reaching the follower's broker, and the follower's broker filling it. Only the middle of that chain belongs to the copier. The two ends belong to brokers, and on most retail setups they are the larger share.

That breakdown deserves its own treatment, because "how fast is your copier" is almost always the wrong question and the right one is "how much slippage does the delay actually cost on the instruments I trade".

Where this fits

The latency chain above, measured end to end and with the parts named, is in /blog/copy-trading-data-flow-execution-speed. For the specific ways this goes wrong between two live accounts — suffixes, unequal balances, partial closes, reconnects and orphaned pendings — see /blog/master-slave-copying-failure-modes.