Direct voice conversation
Mira uses OpenAI Realtime over WebRTC for direct speech interaction, with semantic turn detection and interruption support. Separate transcription events populate the readable conversation.
Yash Jobalia / Project case study
Clinic Assistant Turn conversation into reviewed application actions.
I built Clinic Assistant to connect natural conversation to a working application. Mira brings direct voice interaction and typed chat to appointment and account workflows. Users explore current availability, review proposed changes, confirm actions, and see the resulting records in patient and doctor workspaces.
Built by Yash Jobalia. A fictional clinic demonstrating conversational AI for administrative workflows. Explore with fictional details.
Mira
Voice & conversational assistant
You
Help me book a dermatologist. I’ve had an itchy rash for three days.
Mira
Do you have a preferred doctor, or would you like the earliest available appointment?
You
The earliest available. I’d rather skip extra questions.
Checking availability → preparing appointment and intake summary
Appointment proposalReview your appointment
Concern: itchy rash · Duration: three days. Review the proposed doctor, time, and shared details before confirming.
Awaiting your confirmation01 / The engineering problem
Make conversation useful beyond the reply.
I connected conversational intent to current records, explicit permissions, and reviewable workspace changes. A shared action layer gives voice, typed chat, and manual appointment controls consistent action meanings, while recent context supports follow-up requests and continuity through account access.
02 / Voice first
Speak naturally. Keep the context.
Independent reply language
Input-language detection remains automatic. A searchable selector controls Mira's spoken and written reply language, defaults to English, and can update an active call.
Flexible intake and reviewed actions
Mira is instructed to reuse volunteered details, ask relevant follow-ups one at a time, and stop optional intake when requested. Booking details and patient-reported notes are reviewed before a workspace change is confirmed.
Visible progress and bounded context
The workspace shows transcripts, processing status, tool activity, and completion receipts. It stores up to 200 recent transcript messages per identity; new voice sessions restore the latest 30, while typed requests use the latest 40.
Typed conversation and private account forms
OpenAI Responses powers typed chat through the same workspace action dispatcher. Users can switch to typed interaction after ending a voice call. Existing-account passwords and recovery completion stay in private forms.
Call lifecycle handling
Call controls support explicit hangup, idle check-ins, session cleanup, and a brief connection-recovery grace period. Processing states and tool activity keep users informed throughout the conversation.
Account assistance that carries the task forward
Mira supports exact email or full-phone account lookup, private sign-in handoff, assisted registration, and password recovery. Guest tasks can resume after sign-in. Country-aware phone normalization and optional gender support flexible account setup, with credentials handled through dedicated account flows.
03 / From intent to appointment
A conversation with a reviewable outcome.
- 01
Ask
A signed-in patient asks, “Help me book a dermatologist.”
- 02
Gather
Mira retrieves current choices, clarifies the preferred time, and gathers relevant intake details the patient chooses to share.
- 03
Review & confirm
The patient reviews the doctor, time, and summarized notes, makes corrections, and confirms by voice, text, or button.
- 04
See the result
The server rechecks availability and saves the confirmed appointment. A completion receipt accompanies the updated list and calendar.
Guest discovery
Explore specialists and availability, ask about Clinic Assistant, and begin account access. Mira can look up an exact contact match, open private sign-in, or prepare registration.
Patient workflows
Book, cancel, and reschedule owned appointments; read visit notes; and save reviewed messages in existing appointment notes. Patients can update supported profile fields and navigate appointment, calendar, and account views.
Doctor workflows
Protected doctor accounts can search and read clinic-wide appointments, cancel upcoming visits, and request patient rescheduling while retaining the original slot. Doctors can also book their own care with another doctor.
A personalized workspace
Manual system, light, and dark appearance controls persist in the browser and sync with signed-in accounts. Mira can guide users to settings. The installable PWA includes a reconnect page for returning to connected workflows.
04 / Architecture
Direct audio. Validated actions.
Browser microphone & speaker↔OpenAI RealtimeWebRTC audio connection · speech in and speech out
Typed input → OpenAI Responses API
Voice tools, typed tools, and manual appointment/profile proposals share the workspace dispatcher:
- Validate action & permissions
- Review signed draft
- Confirm & recheck current facts
- Execute & return receipt
- Interface
- Next.js App Router, React, TypeScript, and Tailwind CSS; responsive patient and doctor workspaces.
- Voice & text
- OpenAI Realtime, WebRTC, semantic turn detection, and parallel transcription events. OpenAI Responses API for typed tool orchestration.
- Semantic contracts
- A versioned TypeScript layer defines entities, relationships, action meanings, appointment states, and role/access rules. Zod-derived schemas and shared result semantics support AI instructions and server policies.
- Action execution
- Next.js route handlers connect voice tool events and the typed tool loop to the workspace dispatcher. Signed drafts bind normalized arguments, actor, and semantic version. Confirmation revalidates current facts and claims a persistent single-use token before the business operation.
- Identity & data
- Supabase Auth, protected doctor assignments, PostgreSQL row-level policies, column grants, scoped SQL functions, and a unique confirmed-slot index enforce access and scheduling rules.
05 / Engineering decisions
Trust comes from the action boundary.
Executable domain contracts
A TypeScript semantic layer defines entities, relationships, action meanings, role rules, and appointment states. Shared contracts and Zod schemas feed AI instructions, server policies, and result labels.
Review before workspace changes
Workspace mutations use signed, identity-scoped drafts that expire after ten minutes and bind the proposed arguments and ontology version. Confirmation rechecks current facts and claims a single-use token before execution.
Authorization beyond the prompt
Server session and ownership checks work with PostgreSQL row-level policies, protected doctor assignments, column grants, and scoped database functions. Doctor accounts can inspect clinic appointments; patient access remains owner-scoped.
Preserve the original appointment
Rescheduling uses a database function and slot uniqueness constraint to move an owned appointment atomically while retaining its reference and notes. A doctor's rescheduling request is a separate flag and keeps the existing slot reserved.
Make outcomes visible
Structured receipts identify proposals and completed changes. Typed orchestration retains returned action effects throughout the conversation. Reviewed patient messages are saved in the notes of their existing appointment.
Purposeful persistence and installation
Recent transcripts support continuity across visits. Generated credentials are delivered separately from model tool results. The installable PWA caches public icons and a reconnect page, with connected AI and account workflows.
06 / Engineering quality & demonstrated skills
Test the workflow behind the conversation.
The September 18, 2026 source audit at commit 3f2044c confirmed passing TypeScript checks, a successful production build, and all 54 unit tests passing. Unit coverage exercises role and ownership policies, current-fact validation, replay rejection, action schemas, and phone normalization.
This project brings together real-time browser audio, conversational state, tool orchestration, typed API contracts, authentication, database permissions, and accessible full-stack interfaces.
Workflow-focused browser coverage
Local Playwright tests use mocked application and Realtime interactions to exercise language selection, guest-task continuity, doctor workflows, action receipts, processing states, appearance preferences, and PWA caching.