The orchestration layer

One prompt in. The right model, on the right miner, every time.

Elis AI is the layer between your request and the model that answers it. Every prompt is broken into work, matched to a reusable expert, right-sized to the smallest capable model, dispatched to a miner your policy allows, redacted, metered, traced, and graded — so the next request routes smarter than the last. No model is ever hardcoded.

Why it matters
90%
token reduction in internal tests using right-sized models
10–100×
cost spread between a frontier model and a right-sized SLM
Never
a hardcoded model — selection is earned per request
The problem

Teams are forced to choose between cost, control, and clarity.

The all-you-can-eat era of AI just ended — every major vendor is repricing toward consumption billing at exactly the moment usage becomes uncontrollable. The orchestration layer exists to end the trade-offs below.

Runaway frontier-only bills

Sending every prompt — trivial or hard — to one premium model means you pay frontier prices for out-of-office emails. Consumption-based pricing punishes success: the more your team uses AI, the more it hurts.

The false “cost or compliance” choice

Cheap cost routers send your prompts to third-party clouds your security team blocks. Heavy governance platforms add oversight but never lower daily spend. Teams are told to pick one.

Vendor lock-in from hardcoded models

Pin a model id into a route and you inherit its price, its outages, and its ceiling. When a better or cheaper model ships, nothing routes to it without an engineer rewriting code.

Black-box routing

When you can't see why a model, tool, or agent was chosen, you can't debug a bad answer, prove a decision to an auditor, or trust the system with regulated work.

How it works

Four owned layers. One prompt nests through all of them.

Every request flows through the same funnel — Agent → Template → Task → Step — and each layer is selected dynamically. The concrete model and miner are bound only at the very bottom, at runtime.

  1. 1AgentA reusable, persistent expert

    A prompt is matched to the best existing expert — not built from scratch each time. The agent owns its rules, its workflows, and its general knowledge. A compound prompt fans out to one agent per facet.

    Matched by similarity + earned quality + live feasibility
  2. 2TemplateThe agent's prompt protocol

    The agent's versioned recipe: which capabilities and phases run, in what order. It declares a workflow — a list of tasks — and is improved over time from graded outcomes and user feedback, behind a canary gate.

    Declares capabilities, never a model
  3. 3TaskOne unit of work

    A node in the run's execution graph. It carries capability requirements and policy — what kind of work this is and what it's allowed to use — not a model id. Tasks run in dependency order, in parallel where they can.

    Carries capability requirements, not a model
  4. 4StepThe executable atom

    A tool call, a model call, or a delegation to another agent. This is the only place a concrete model and a concrete miner are bound — at runtime, by the scorer, from what the pool can actually serve right now.

    The single place model + miner are bound
Never hardcoded

Selection is dynamic at every layer — scored on lexical + semantic similarity, earned quality (a rolling EMA from real graded outcomes), and whether a capable miner is actually reachable right now. Nothing is pinned. When a cheaper or better model becomes available, the next request can route to it with no code change.

Good to know

Containment order ≠ selection order

The layers above read Agent ⊃ Template ⊃ Task ⊃ Step — that's what contains what. But for a compound prompt the run decomposes into the task graph first, then picks the best agent + template per facet. What nests inside what and the order decisions get made are two different orderings.

Good to know

“Step” means two things

In orchestration a step is one executable atom — a tool call, a model call, or a delegation to another agent. In automation (desktop or browser) a step is a fine-grained UI action — a click, a keystroke, a screenshot. Same word, different altitude: one orchestration task can expand into many automation steps.

What it does

Everything the layer handles so your code doesn't have to.

🎯

Right-sizing

When you let Elis choose (elis/auto or elis/agent), each request is scored against every capable model and bound to the smallest that clears the bar — a one-liner never costs frontier pricing. Name a tier yourself and Elis honors your choice instead.

📐

Adaptive context sizing

Elis fits the context window to the problem instead of filling it. Each step gets only the history, recalled memory, and retrieved chunks it actually needs, sized against the resolved model's real token budget. When the input still runs long, Elis summarizes and compacts it before dispatch — never slicing a message or silently truncating — so accuracy holds while wasted tokens don't.

♻️

Recall & reuse

Beyond right-sizing, Elis recalls stored knowledge, prior findings, and context by similarity (RAG) and reuses cached embeddings, tool plans, and transformations — so it avoids re-fetching data or re-running a tool call it has already done. Exact-key caching ships today; semantic answer caching is on the roadmap.

⏱️

Expected wait by path

Picking the agent costs different amounts of time depending on the path: naming an expert yourself skips the search and routes in about 0.3s; recalling the best existing expert by similarity takes about 2s; and building a brand-new specialist on the first call takes about 5.5s — that one-time authoring cost is amortized on every recall after. (Selection time only; answer generation is extra and depends on the bound model.)

🔀

Pre-first-token failover

If the chosen miner is unreachable, Elis reroutes to the next-best capable model in the same tier before a single token streams — the user never sees the miss.

🔎

End-to-end tracing

Every tool, model, agent, and router decision is serialized and traced: which candidates were considered, what scored them, which miner ran the work, and why. Debuggable and audit-ready.

🛡️

PII redaction that fails closed

Sensitive data — emails, SSNs, cards, keys, DSNs — is detected and redacted before any model call. Originals live in a request-scoped vault, never persisted, and are re-hydrated only at final delivery.

⛏️

A tiered miner marketplace

Model execution runs on miners in a scoped pool: shared community miners reachable by everyone, org-scoped miners only your organization can use, and BYOK miners bound to your own keys. Your policy decides what's in reach.

🚑

Miner-scoped failure handling

A failure quarantines the one miner that failed — not the model, not the provider. Auth errors deactivate only the dead token's models; rate limits back off only the miners sharing that token. One bad node never cascades.

🧾

Metered & billed correctly

Every token-consuming step is metered: the requester is charged and the owning miner is paid out, tracked per user and per organization. Settlement is idempotent — no double-charges, no unmetered paths.

📈

Learns from every run

Graded outcomes and 👍/👎 feedback update per-layer quality signals, so routing, templates, and model choice improve on their own — proven on canary traffic before they ship to everyone.

Built for both sides

One layer, two audiences it was made for.

For clients

Governance, control, and a bill that scales with sense.

Deploy where compliance approves

Run Elis in your own VPC, a dedicated cluster, or fully on-prem / air-gapped. Sensitive prompts need never leave your boundary.

Prove every decision

A complete, exportable trace of every model, tool, and routing choice — for troubleshooting, review, and regulated audit.

Predictable, right-sized cost

Right-sizing plus caching and reuse push spend down without hand-tuning; on-site deployments drop to energy cost with no token meter.

Tenant isolation by default

Per-organization scoping at the app and data layer. One tenant's data, miners, and feedback never touch another's.

For developers

An OpenAI-compatible gateway with no lock-in.

Point your SDK at one endpoint

An OpenAI-compatible /api/v1 gateway — keep your existing client and tooling; swap the base URL and go.

Route by intent, not by model id

Ask for a tier — elis/auto, elis/small, elis/frontier, elis/agent — and let the scorer bind the concrete model. Or pin an agent when you want a specific expert.

Bring your own keys & models

Use your OpenAI, Azure, Ollama, or any custom REST provider. Elis orchestrates the models your fleet already serves — it never injects its own.

Inspect the whole decision

The Decision Trace shows the candidates, scores, miner, and per-stage timings behind every answer — no black box.

Developer API

Every option, reachable from your existing SDK.

Everything the orchestration layer does is reachable through two OpenAI-compatible endpoints. Point your existing SDK at the gateway, pick how much state you want us to hold, and opt into agents, memory, tools, and reasoning depth per request.

Chat Completions

POST /api/v1/chat/completions

Stateless. Drop-in OpenAI compatibility.

  • You send the full messages array each call
  • Sync or streamed (SSE)
  • openai.chat.completions namespace works unchanged
  • Best for IDE/editor integration and simple prompt → response

Responses

POST /api/v1/responses

Stateful. We hold the turn history.

  • Pass previous_response_id instead of replaying history
  • reasoning.effort control (low / medium / high)
  • openai.responses namespace works unchanged
  • Best for multi-turn agents with server-owned memory
🧠

Use our agents

Send model: elis/agent and the engine matches the best expert to your prompt — recalling memory and running its own server-side tools where relevant — then returns a normal completion.

📌

Pin or bring your own

Pin a specific expert with agent (or agent_id) and a recipe with template — scope-enforced to global or your org's own agents that you build in the console. A pin is honored no matter how simple the prompt.

🧵

Session memory

On the Responses API, thread turns with previous_response_id and let us own the history. Or set store: true on an elis/agent call to index an exchange for later semantic recall.

🔧

Tools

Direct tiers speak OpenAI function/tool calling — your code executes the call and sends the result back. elis/agent instead runs the engine's own server-side tool loop (web search, retrieval, code, data, and more).

🔌

Connect your tools & MCP

Install external and MCP-style tools at the org level. The planner discovers them per run and mixes them into the agent's capability set — so your own integrations become first-class steps.

🔑

Bring your own providers & models

Already have OpenAI, Anthropic, Azure, or any REST provider? Add it by name + URL + key and Elis orchestrates over it. Running your own Ollama models? Register them as miners — the pool routes to your fleet, and Elis never injects a model you didn't bring.

🎚️

Tiers & speed-mode tags

Route by intent: elis/auto, elis/nano, elis/small, elis/frontier, elis/agent. Add a speed-mode tag — elis/agent-instant | -balanced | -deep (the same names the chat uses) — to steer depth without pinning a model. The older tier-name tags (-nano | -small | -frontier) still work as aliases.

🧩

Reasoning effort

On the Responses API, reasoning.effort of low / medium / high maps to instant / fast / deep speed modes — more deliberation only when the task earns it.

📡

Streaming & measured usage

Stream tokens over SSE on both endpoints. The final frame carries a usage object of actually-billed tokens — miner-reported, not estimated — so total_tokens equals what was charged.

🔬

Inspect every decision

Every orchestrated response includes an elis object: run_id (links to its Decision Trace), agent_id, template_id, and resolved_model — so you can confirm exactly which expert and model ran.

📇

Discover models

GET /api/v1/models returns every tier with live context window, input/output limits, and capabilities — OpenAI-compatible, so client.models.list() just works.

Naming a tier is an override — here's what changes

One important nuance: choosing a direct tier by name is a deliberate override. When you name a direct tier, you've made the sizing decision yourself — so Elis honors it and dispatches straight to a miner. Only elis/agent runs the full orchestration engine, and only elis/auto re-sizes the tier for you. On the agent engine, a speed-mode tag (elis/agent-deep etc.) sets a band, not a pin: it fixes the floor and ceiling, and inside that band each task is still right-sized — easy sub-tasks bind smaller models; only the steps that need it take the top of the band.

Model you sendRight-sizing?Task splitting?What happens
elis/nano · small · frontierNo — you fixed the tierNo — single direct dispatchThe pool still binds the best model within your chosen tier.
elis/autoYes — tier picked per promptNo — still direct dispatchPrompt-aware tier router, then straight to a miner. No orchestration.
elis/agentYes — sized per stepYes — decomposes & fans outThe full engine: memory, tools, agent graph, right-sizing per step. A speed-mode tag (agent-instant | -balanced | -deep) bounds the band it may bind within — never a single pinned model.

Expected latency & cost — pick a model

Every option trades latency for depth differently. Select one to see its warm floor (simple prompt) vs a complex prompt, what runs, and where it sits on cost. Model your own dollar figures in the live preview.

Direct tiers — you pick the model
Auto — right-size fitter
Agent — full orchestration
elis/auto
Right-sizesSplits tasks

Router hop (~1 s) + tier dispatch; complexity routes the prompt up (nano on easy → frontier on hard).

Simple prompt
~2 s
Complex prompt
~2.5–3 s
Relative cost

You pay only the tier each prompt needs. No task splitting.

Warm-path measured response times — July 2026 prod benchmark. Two variables move latency: the mode (how much machinery runs) and the task complexity. Actual numbers depend on which miners serve the pool at dispatch time, and the first elis/agent call in a fresh session pays a one-time warm-up.

Use an agent, with memory & tools

Orchestrated run: auto-matched or pinned expert, server-side tools, indexed for recall.

curl -X POST https://YOUR_DOMAIN/api/v1/chat/completions \
  -H "Authorization: Bearer elis_xxxxxxxxxxxxxxxx" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "elis/agent",
    "agent": "return_a_json_array_1",
    "store": true,
    "messages": [{"role": "user", "content": "Give three prime numbers."}]
  }'
Server-owned session memory

Stateful Responses API: thread turns with previous_response_id, control reasoning depth.

# First turn — no previous_response_id
curl -X POST https://YOUR_DOMAIN/api/v1/responses \
  -H "Authorization: Bearer elis_xxxxxxxxxxxxxxxx" \
  -d '{"model": "elis/agent", "input": "Our launch date is 2026-09-01."}'

# Next turn — thread it; the server holds the context
curl -X POST https://YOUR_DOMAIN/api/v1/responses \
  -H "Authorization: Bearer elis_xxxxxxxxxxxxxxxx" \
  -d '{
    "model": "elis/agent",
    "input": "How many days until launch?",
    "previous_response_id": "resp_abc123def456"
  }'
The self-learning loop

Selection that gets better on its own — and proves it.

The layer doesn't just route — it learns which routes were good and adjusts. The loop is closed end-to-end and isolated per tenant.

  1. 1

    Run & trace

    Every run persists a full trace — router selection, chain steps, tool I/O, model dispatch, miner choice.

  2. 2

    Grade & collect feedback

    An automatic grader scores the output; the user's 👍/👎 and facet checkboxes (incomplete, too slow, off-topic…) route to specific signals.

  3. 3

    Update per-layer quality

    Those signals move earned-quality EMAs on the agent, template, tool bundle, and model profile — the exact layers responsible.

  4. 4

    Promote on merit

    A challenger template that beats its champion on real graded samples is promoted through a canary gate — never straight to full traffic.

  5. 5

    The next request benefits

    Similar prompts now route to the improved expert and better-fit models. The loop is tenant-isolated: your feedback only ever tunes your routing.

It gets cheaper the more you use it

As Elis learns your recurring work, it solidifies the winning multi-step plan into a reusable task list — so similar requests reuse a proven chain instead of planning from scratch. Fewer redundant planning passes and tool calls means the cost keeps dropping the longer you run it. Reusing learned plans is live for automations today and rolling out across the platform.

Deployment options

Run it where your workload — and your auditors — need it.

Distributed edge

Community Pool

For non-regulated workloads that want the lowest cost and latency. Runs across a distributed edge network close to your users.

  • Lowest cost + lowest latency
  • Shared and BYOK models
  • No data-egress fees
Fastest to live

Shared (managed)

Fully managed by Elis with per-organization logical isolation. The quickest path from trial to production.

  • Managed for you
  • Per-org isolation
  • Shared + BYOK miners
AirShield

Dedicated / On-prem

Your own isolated stack — VPC, dedicated cluster, or fully air-gapped — for regulated workloads. Data stays inside your boundary.

  • On-prem / air-gapped option
  • Required for regulated frameworks
  • Energy cost, no token meter
Get started

See the orchestration layer make a decision.

The live preview runs a real prompt through the funnel — fan-out, right-sizing, miner selection, and the token savings — without touching a bill. Then wire your SDK to the gateway and ship.