hongkyukim/context-quest-foundry-elastic-workshop

A companion repo for the in-person Microsoft x Elastic build lab. You connect a
Microsoft Foundry agent to Elasticsearch over the Model Context Protocol (MCP),
measure how it does, then build an AI index of Knowledge Indicators (KIs) with
Elastic Context Engine and measure again. The comparison covers token usage, answer
accuracy, latency and discovery calls.
> Speakers: Peter Laudati (Principal Partner Solution Architect, Microsoft) and
> Jeff Vestal (Senior Principal AI Architect, Elastic).
> Bring a laptop. Azure and Elastic lab environments are provided on site.
Foundry agent ──MCP──▶ Kibana /api/agentbuilder/mcp ──▶ Elasticsearch
│ │ ct-orders / ct-support-tickets / ct-products
│ └─ platform.contextengine. ──▶ AI index "trailworks-commerce" (KIs)
└── run usage, tool calls, latency ──▶ context-quest bench ──▶ reports/foundry.md
## Agenda
| Time | |
| 5:30 – 6:00 pm | Check-in, food and networking |
| 6:00 – 8:00 pm | Hands-on workshop (labs 00–06 below) |
| 8:00 pm | Open discussion and mixer |
Facilitator timings are in docs/facilitator-guide.md.
## The scenario
Contoso Trailworks is an outdoor retailer with three Elasticsearch indices: orders, support
tickets and products. Two business rules matter, and you can't see either one in the mappings:
1. Revenue counts completed orders only. Refunded and cancelled lines keep their total.
2. An at-risk customer has 2+ unresolved (open or pending) tickets with priority P1 or P2.
An agent that only does discovery (list indices, read mappings, sample documents) spends tokens
working out the schema and then writes a plausible but wrong query. An agent that pulls KIs gets
the orientation, the rules and verified ES|QL in one retrieval, so it makes fewer calls and gets
the answer right.
## Quick start (offline, no credentials)
bash
git clone https://github.com/hongkyukim/context-quest-foundry-elastic-workshop
cd context-quest-foundry-elastic-workshop
make setup # uv venv + editable install with dev deps
make test # contract tests, no network
make simulate # baseline vs Context Engine, simulated
You can also run it without make: uv venv && uv pip install -e '.[dev]' && .venv/bin/context-quest simulate --trace.
Sample simulated output (chars/4 token estimate, modelled latency; the live numbers come from Foundry):
| mode | tool calls | discovery calls | total tokens | latency | mean accuracy |
| baseline | 33 | 28 | 30,899 | 36.1 s | 0.69 |
| context | 17 | 8 | 17,271 | 22.3 s | 1.00 |
## Labs
| # | Lab | You will |
| 00 | Setup | Claim lab environments, configure .env, run the simulator |
| 01 | Load data | Load the Trailworks indices, explore them with ES\|QL |
| 02 | Connect Foundry to Elastic over MCP | Probe the Agent Builder MCP server and attach it to a Foundry agent (portal + SDK) |
| 03 | Baseline | Measure tokens, tool calls, latency and accuracy without context |
| 04 | Build an AI index with KIs | Create the AI index, add sources, generate and review KIs |
| 05 | Context-optimized agent | Give the agent the Context Engine tools and rerun the benchmark |
| 06 | Evaluate, explain, secure | Fix KIs from the evidence, trace answers back to sources, lock down keys |