Draw an LLM between the user and an API
This skips prompt versioning, context selection, schemas, memory, permissions, invalid output, and the observation loop.
AI agent system design · production case study
Aurora is the NexusTrade agent: a router, a durable ReAct loop, and a host that owns tools, approvals, and recovery. Use the real system as the junior-to-senior case study.
Open the final system map ↓Follow the real boundary between request routing, ReAct decisions, and the executors that turn JSON into work.
Router V5 reads the hydrated conversation and decides whether to run one catalog prompt, ask for clarification, create a durable agent, or continue an existing agent.
The 60-second answer
Aurora's Router V5 chooses Call Tool, Ask Clarity, Create
Agent, or Continue Agent. Call Tool runs one catalog
prompt and commits. A durable agent returns schema-valid ReAct JSON.
The stepper dispatches actions, commands, approval waits, or
completion. Every observation is persisted before the next turn.
Independent actions may run concurrently and still commit in order.
Interview mode
Scope one task and draw one model turn. Add context, authority, ownership, and financial reconciliation as the workload demands them. Test one failure at each boundary and explain the trade-off.
The diagrams below are the answer. Use this clock to scope the product, draw the controller, introduce long-running work, and reserve time for failures and trade-offs.
This skips prompt versioning, context selection, schemas, memory, permissions, invalid output, and the observation loop.
The model remains replaceable. Prompt, schema, memory, tool, and authority contracts define the agent product.
Interview trade-off = context → choice → reason → consequence → upgrade trigger
Junior interview · MVP
Design an assistant that answers a trading question, reads one user's portfolio, runs one paper backtest, and proposes a paper order for manual approval.
Junior · 1 of 4
A language model call starts with an array of messages. Append the assistant response and send the expanded array on the next request. That transcript creates the back-and-forth experience.
messages = [
{ role: "user", content: "Explain RSI" },
{ role: "assistant", content: "RSI measures recent momentum..." },
{ role: "user", content: "Now apply that to AAPL" }
]
response = model.generate({ messages })
This version can explain a trading concept. It has no current market data, portfolio state, backtesting engine, scheduler, or brokerage adapter. Controlled context and tools add those capabilities.
Junior · 2 of 4
Do not paste every available fact into one giant prompt. Build the model input from layers with different owners and lifetimes. Stable instructions come first; current goals, state, and retrieved evidence are added for this turn.
Assemble four layers: the versioned system contract, the task procedure, current turn state, and a small relevant evidence slice. Heavy tool results remain in durable storage and re-enter later turns as compact summaries or owner-scoped references.
REACT DECISION CONTRACT
Return JSON with "thought" and exactly one of:
"actions" independent product-tool calls
"command" agent-control work such as askUser or createSubagents
"finalAnswer" the completed response
SIMPLIFIED BACKTEST TOOL PROCEDURE
Use "Backtest Portfolios" to launch a native backtest.
Pass the resolved portfolio ID and an explicit date window.
After launch, use the returned backtest ID with "Read Backtest".
Do not launch the same portfolio and window again.
Boundary 1 · The decision model chooses a tool
{
"thought": "The portfolio exists and the requested window is explicit.",
"actions": [
{
"tool": "Backtest Portfolios",
"input": "Backtest PORTFOLIO_ID from 2021-01-01 through 2025-12-31."
}
]
}
Boundary 2 · The tool prompt produces its schema
{
"backtestConfigs": [
{
"portfolioId": "PORTFOLIO_ID",
"startDate": "2021-01-01",
"endDate": "2025-12-31"
}
]
}
These are two separate model boundaries. The decision model returns an
outer JSON decision, but each actions[].input value is
plain text. The server appends that text as the user message for the
selected tool prompt. The Backtest Portfolios prompt then returns the
tool-specific backtestConfigs JSON shown above. Server
code resolves the portfolio for the authenticated user, validates the
dates and product constraints, launches the native backtest, and
returns its backtest ID.
Putting escaped backtestConfigs inside
actions[].input would only encode JSON as text for the
second model to interpret. It would not provide typed arguments
directly to the backtest implementation.
Reject blank inputs and unknown tools. Treat an object emitted as
input as serialized text, never trusted arguments.
Server code rejects invalid configurations, unauthorized portfolios,
restricted jobs, and invalid dates before computation starts.
Junior · 3 of 4
Give the decision model a catalog of supported tools. It selects a tool and writes a bounded text instruction for it. The server resolves the selected tool. Some tools handle a complete machine request directly; Backtest Portfolios invokes its own force-JSON prompt to produce the tool-specific schema.
{
"thought": "I need one current quote and one portfolio read before sizing.",
"actions": [
{ "tool": "Stock Screener", "input": "Read the current quote for SPY." },
{ "tool": "Read Portfolio", "input": "Read portfolio portfolio_123." }
]
}
That JSON is a proposal. The model has not executed a tool, granted itself authority, or written an observation. Host validation is the next step.
Junior · 4 of 4
The host application owns parsing, validation, permissions, execution, and error handling. Reject unknown tools and invalid JSON. Persist a semi-automated proposal instead of running it.
const state = validateAgentState(modelJson)
const actions = getStateActions(state)
if (agent.automationMode === "semi-automated") {
await persistPendingActionApproval(agent, state)
} else {
const results = await executeActionBatch(agent, actions)
await commitObservationsInActionOrder(agent, results)
}
This is the host-side contract, not model freedom. The stepper
enforces exactly one decision branch and at most ten actions. Parallel
actions are on by default and can be disabled with
AGENT_MULTI_ACTIONS_ENABLED=false. Each named tool is
resolved by the server before execution. Semi-automated runs persist
the exact proposal and wait for approval.
Mid-level interview · product scale
Evolve the assistant into a multi-user product with request routing, ReAct, typed tools, scheduled runs, asynchronous backtests, progress updates, and automation-mode approvals.
Mid-level · 1 of 5
Many users and concurrent turns make a durable agent per message the wrong default. Classify the request first. NexusTrade gives Router V5 the hydrated conversation. The router chooses the smallest supported path and, when durable work is needed, authors the initial plan and title.
Routing plays automatically when this figure enters view. Use the button to replay it.
type RouterDecision =
| { action: "Call Tool"; toolName: CatalogPromptName; input: { text?: string } }
| { action: "Create Agent"; input: { plan: string; title?: string } }
| { action: "Ask Clarity"; input: { questions: string } }
| { action: "Continue Agent"; input: {} }
The action set is closed. Catalog names go in toolName,
never in action. Ask Clarity questions is
one markdown string, not an array. The schema asks for a Create Agent
title; the runtime still accepts a plan if the title is missing.
Call Tool is one-shot catalog fulfillment. Create Agent
and Continue Agent hand the durable loop to ReAct and the stepper. Ask
Clarity and Call Tool also return suggested next messages.
Mid-level · 2 of 5
Each ReAct decision is forced to JSON and contains exactly one of
actions, command, or
finalAnswer. The stepper validates that
contract, checks whether approval is required, and sends the decision
to the matching execution path.
Suppose an agent needs the user's portfolios, watchlists, and scheduled agents. None of these reads depends on another, so the model can request all three in one decision. Every action sees the same conversation snapshot and remains valid even if either sibling fails.
{
"thought": "These product reads are independent.",
"actions": [
{ "tool": "Fetch User Portfolios", "input": "Load my portfolios" },
{ "tool": "Fetch User Watchlists", "input": "Load my watchlists" },
{ "tool": "List Scheduled Agents", "input": "Load my schedules" }
]
}
Every action reads the same conversation snapshot and receives a
stable batch ID plus action index. The runtime waits for every
sibling with Promise.allSettled, sorts results by
action index, and commits observations in that original order. A
handled tool failure becomes a typed error observation; an
unexpected rejection remains visible and cannot silently erase
successful siblings. Recovery reuses the stable execution IDs so
completed actions are not mistaken for new work.
Run the batch to see concurrent settlement, a typed tool failure, and deterministic commit order.
claim = Agent.claimNextAgent(workerId)
renewClaimEvery(claim, 30_seconds)
messages = hydrateConversationTail(agent)
prompt = buildReActPrompt({
entitlements,
originalRequest,
attachedPortfolio,
currentPlan,
currentIteration,
recentCommittedActions,
toolAllowlist
})
decision = model.respond(prompt, forceJSON = true)
state = validateExactlyOneOf(decision, ["actions", "command", "finalAnswer"])
if (state.actions) {
snapshot = messages
results = settleAllConcurrently(
state.actions.map((action, actionIndex) =>
execute(action, snapshot, stableExecutionId(actionIndex))
)
)
persistObservations(results.sortByActionIndex())
}
if (state.command) executeAgentCommand(state.command)
if (state.finalAnswer) completeAgent(state.finalAnswer)
publishAgentUpdateAfterCommit()
Backtest Portfolios, optimizers, sandboxes, and ingestion
tools can yield with durable computation IDs. The stepper persists
those IDs, moves to waiting_for_computation, and releases
the worker slot. Subagent joins use
waiting_for_subagents; clarification uses
awaiting_user_input.
A process crash may lose the current in-flight operation. It must preserve every completed observation and prevent duplicate committed side effects.
Mid-level · 3 of 5
The model proposes an action. It does not grant itself authority. Aurora uses an automation mode, not an LLM risk classifier. Automated runs execute after product checks. Semi-automated runs persist the exact proposal and wait for a person.
Agent-created orders are staged proposals in product state. The model has no brokerage credentials. A dedicated execution path owns submission and reconciliation.
Mid-level · 4 of 5
A NexusTrade conversation already has a persisted
Agent document in chat status. Router V5 can
answer or ask for clarification while that document remains the
conversation container. When it chooses Create Agent, the
server upgrades the same _id in place with the plan,
title, execution settings, and a runnable status.
{
"_id": "agent-id",
"userId": "authenticated-user-id",
"conversationId": "conversation-id",
"title": "Portfolio Change Review",
"initialPrompt": "Review my portfolio and summarize material changes.",
"plan": "Inspect the portfolio, gather evidence, and summarize material changes.",
"automationMode": "semi-automated",
"origin": { "type": "manual" },
"status": "pending_plan_approval",
"maxIterations": 40,
"currentIteration": 0,
"runIterations": 0
}
These are representative fields after a semi-automated chat is
promoted. _id remains the run identity.
title and plan come from the Router.
initialPrompt keeps the original request. Router and
ReAct prompts are snapshotted in AgentPrompts. Chat
Aurora uses Router V5. Strategy-triggered LaunchAgent still has a
Planner V4 init path. Automated runs enter running
instead of waiting for plan approval.
Mid-level · 5 of 5
The current request determines which product data belongs in the next model call. This portfolio-creation request only needs its new specification, current plan, entitlements, and available tools. Full results from earlier research add no decision input.
The model parses positions, historical trades, price arrays, and news that cannot change how this new portfolio is created.
Product and backtest data stay in their existing durable records. Selected oversized discovery, screening, and sandbox outputs move to private Tigris objects. The next turn receives compact observations and owner-scoped references instead of copying every payload again.
Mid-level · prompt operations
Once more than one person edits prompts, a save is not enough. Shared prompts need versioning. Promotion quality is a separate question from save.
A prompt is executable application configuration. Changing its system
instructions, examples, model, JSON setting, or referenced schema can
change agent behavior. NexusGenAI is the registry. A save snapshots
the configuration and increments currentVersion. Restore
is a version read, not a bakeoff. The referenced schema is a separate
dependency, so compatibility belongs to the prompt version that uses
it.
{
"name": "Agent decision prompt",
"currentVersion": 42,
"model": "selected-model",
"forceJSON": true,
"schemaId": "agent-decision-schema-id",
"prompt": "Return exactly one ReAct decision branch...",
"examples": ["..."]
}
Select “Run evaluation method” to follow one bakeoff candidate.
Calibrate a judge on reviewed good and broken examples before using its score. Compare versions on the same held-out cases, then change the component that owns the failure. Permission errors belong to product checks; stale evidence belongs to retrieval. Runtime code can restore a prior NexusGenAI version without changing the stepper.
Senior interview · fault-tolerant platform
Scale the product across stateless web instances and agent workers. Support parallel research, long computations, bounded costs, and multiple brokerage adapters while preserving recoverable state and user authority.
maxIterations and a tree-level
LLM cost ceiling (default $50 stop, $20 ops alert).
Senior · 1 of 2
Agent {
title, initialPrompt, lastUserRequest, plan, conversationId
status, previousStatus, automationMode
currentIteration, totalIterations, runIterations, maxIterations
currentState: {
thought, actionBatchId,
actions[] | command | finalAnswer,
commandExecution,
pendingComputations[]
}
parentAgentId, childAgentIds[], subagentType
claimedBy, claimedAt, claimToken, claimGeneration
claimHeartbeatAt, claimExpiresAt, stateVersion
origin, portfolioContext, costCeilingUsd, treeCostUsd
updatedAt, lastProgressAt, completedAt, stoppedAt, error
}
Aurora stores the run as a durable agent document. A worker atomically claims a runnable agent, performs a bounded step, renews ownership while active, and releases the claim when the agent finishes or waits. Another worker can recover an abandoned run from the last committed observation.
Long work owns its compute lifecycle. The agent owns the goal, durable dependency, result, and next decision.
Senior · 2 of 2
| Pressure | Design response | Failure question |
|---|---|---|
| Many concurrent agents | Cap each worker at 100 active executions, poll every 500 ms, and claim agents with renewable leases. | Can two workers execute the same step? |
| Expensive prompts | Use compact observations, a code-driven tool allowlist, and a tree-level cost ceiling. | Can one user exhaust shared model capacity? |
| Slow tools | Persist a job ID, enter a waiting state, release the worker, and wake from completion. | What happens if completion arrives twice? |
| Parallel research | Launch bounded subagents with narrower context and join their durable results. | How do partial failures affect synthesis? |
| Rapid UI progress | Commit canonical state first, then publish low-latency events to connected clients. | Can a reconnect reconstruct missed updates? |
| External side effects | Use provider-supported idempotency where available, dedicated adapters, and venue-specific reconciliation before retrying. | Did a timeout occur before or after acceptance? |
Each trace should identify the model, system prompt version, injected context, action schema, proposed arguments, permission result, tool observation, latency, token usage, cost, retries, and terminal state. Evaluators can grade task completion, unsupported claims, permission compliance, efficiency, and the final explanation.
Aurora also writes SystemAlert records for worker errors,
stalled agents, queue backlog, and cost thresholds. Those alerts carry
operational evidence for the responder; the user-facing agent trace
carries the decision history. Both are required because a green HTTP
route says nothing about a stuck claim or an unresolved order.
A system design interview answer should close this loop. Reliability comes from replayable state and idempotent boundaries. Agent quality improves when traces produce specific changes to prompts, tools, routing, or policy.
Each turn commits one recoverable state transition. Independent actions may run concurrently, but their observations retain stable identities and commit in action order before the next model decision.
Play the lifecycle to see where durable state replaces a blocked worker.
Read Backtest yields when the returned backtest ID is
still processing. The wait stores the computation type, backtest ID,
message ID, and action execution identity. A wake check adopts the
terminal result and transitions the agent back to
running, even after a process restart.
For each failure, identify the surviving record and explain how the system prevents a duplicate financial effect.
AnswerThe agent stores its owner, lease, generation, and last committed state. Worker B resumes from that state with generation 8. A late generation 7 write that goes through the execution fence is rejected. Tree-cost rollups and some HTTP handlers are unfenced by design.
AnswerCreate the action identity before execution. At-least-once delivery is safe only when the tool can find or safely recreate its prior result and the agent can adopt the persisted observation.
AnswerThe computation has its own job ID. The agent persists that ID, releases the worker, and lets the same waiting action adopt one terminal result.
AnswerAgent and chat records are canonical.
WebSockets are the fast path. A status poll or reconnect
reloads persisted state, and
stateVersion prevents an older snapshot from
replacing it.
AnswerA timeout leaves an unknown outcome, so a blind retry can duplicate a position. Alpaca and Public can carry stable client order identity. TradeStation returns its order ID in the response, so its unknown outcome requires venue-specific reconciliation.
Research ends at evidence or order intent. Crossing the second gate requires current product authority. A profitable backtest does not grant brokerage permission.
This is a required design boundary for any research agent. Text retrieved from outside the product cannot change system prompts, tool allowlists, permissions, or approval state. Preserve its source, limit what enters context, and route every suggested action through the same schema and product-policy checks.
active concurrency ≈ arrival rate × active step time
Five turns per second with four seconds of active work needs about 20 execution slots before headroom. A suspended backtest consumes no slot while it waits.
Redis handles low-latency delivery. MongoDB stores recoverable agent and product state. Compute workers and broker adapters keep long work and financial credentials outside model execution.
Practical implementation paths
Claude Code, Cursor, and Codex demonstrate the shell: instructions, tools, context, and approvals. Aurora is the NexusTrade implementation: portfolio context, durable research, staged orders, and financial authority outside the model. Any shell still depends on server-side validation.
The model proposes the next move. The surrounding system controls context, capabilities, permissions, persistence, recovery, cost, and financial side effects. That controller is the actual AI trading bot.
Keep reading
Free forever. Read every article, ask Aurora about any strategy in it, and backtest the idea against market history.
No credit card required. Already have an account?
Create a free account to backtest ideas against market history, inspect the risk, and deploy to paper or live markets when you're ready.
Free to browse. No credit card required.
Free workspace
Create your account and take your first strategy from question to backtest.
Free to explore No credit card
Secure authentication. Review our Privacy Policy.
You're on the list
Finish creating your free account to ask Aurora, backtest strategies, and paper trade when you're ready.
Free to explore No credit card
No comments yet.