Context

Voice-to-action matters: people speak intent and expect execution. Safe execution needs predictable tools—catalogs, carts, calendars, payments, tickets—with hard guardrails and logs. Grocery example: "Add 2L oat milk, 6 bananas, a dozen eggs, whole-grain bread, delivery 7–9pm." A reliable agent must match items, check budget and preferences, reserve a slot, pay, and confirm—no improvisation.

Wizard's Chess is the demo sandbox that proves this pipeline under strict rules before wiring it into real commerce.

Demo

Approach

Input → rules check → safe action or escalation → audit trail.

1. Voice note intake: user speaks a move or goal; audio is transcribed.
2. Ruleset comparison: the transcript is mapped to a finite set of allowed actions (legal chess moves, game commands).
3. Decision:
   • If legal and safe, execute.
   • If ambiguous, illegal, or unsafe, reject, explain, and await further input (human-in-the-loop).
4. Execution: apply the move on the board; log state change with timestamp and rationale.
5. Autonomous opponent: on Black's turn, an agent makes a move. It is intentionally weak. This illustrates the gap between model specialization (strong chess engines) and a general LLM (OpenAI gpt-4.1) used here for understanding and orchestration, not elite gameplay.

Operators: Voice Command • Rule & Constraint Adherence • Tool Calling & Action Graph • Needle-in-Haystack Search

Outcome

Example applications

• •Voice-to-cart (grocery): build and place a budget-bound order with allowed substitutions and a booked delivery window.
• •Voice-to-ticket: open a bug with owners and context; notify the channel.
• •Voice-to-dispatch: fill a shift gap against availability and policy.

Stack

Role & Collaborators

Role: goal-to-plan logic, rules and constraints, tool-call graph, confirmations, exception playbooks.

Collaborators: team leads, schedulers, operations owners.