Embedded Files
AI-MINDS explores how Agentic XR systems can evolve beyond static models into self-learning, domain-aware ecosystems.
These systems use large language models (LLMs) as cognitive cores, combined with extended-reality (XR) sensors and domain-specific logic (DSL) as knowledge.
Through continuous fine-tuning, expert feedback, and ethical monitoring, the system becomes context-adaptive, capable of reasoning, explaining, and safely acting within specialized environments such as education and healthcare.
Agentic XR
Instead of a static chatbot, the project builds immersive, human-like mentors and coaches that interact naturally through speech, gesture, gaze, and emotion in XR environments. These agents can explain, assess, and adapt, much like a teacher or therapist, while remaining safe, explicable, and human in loop.
Framwork
At runtime, the avatar coach runs on the edge (headset or nearby device) for low latency while keeping raw data local. Multimodal signals feed a MAPE-K control loop, Monitor, Analyze, Plan, Execute over Knowledge, grounded in a DSML (domain-specific modelling language) and system models. This enables safe, explainable real-time adaptation.
Monitor (M) tracks KPIs (key performance indicators) such as accuracy, range-of-motion, tempo, balance, and adherence.
Analyze (A) detects trends, errors, plateaus, or fatigue using DSML rules and model estimates.
Plan (P) selects the next intervention (cue, difficulty, pacing, rest) with a clinician/teacher-readable rationale.
Execute (E) delivers actions through the avatar (speech, gesture, XR overlay) under safety constraints.
Knowledge (K) stores goals, history, policies, model parameters, and provenance used by the loop.
Safety & governance
A Digital Twin runs in parallel for anomaly detection and policy pre-checks before actions are applied. It can pause, flag, or request expert-in-the-loop sign-off. All events are logged for audit and explainability.
Learning without moving raw data
When local performance drifts or stalls, Meta-adaptation proposes model/policy updates. Devices send encrypted updates to a Federated Learning Aggregator, which improves the global model and then rolls it back to devices, raw streams never leave the edge.
Google Sites
Report abuse