Predictive Geometry of LLM Representations

This is the direction I am most fascinated by right now: whether the geometry of LLM representations can predict behavior before the model commits to an output. I study internal signals such as attention sinks, first-token dominance, semantic trajectories, hidden-state geometry, and competing hypothesis traces as early indicators of hallucination, instability, or reliable reasoning.

The long-term goal is to turn representation geometry into a practical predictive layer: a way to understand, monitor, and potentially intervene in model behavior before the final response is generated.

Relevant Papers and Projects

GhostTrack
Mechanistic hypothesis and semantic tracking for LLMs.

Framework for tracing competing thought trajectories and identifying hallucination risks before final output.

Attention Sink Analysis
Transformer attention analysis.

Tools for analyzing first-token dominance and attention sink behavior in long-context language models.

Semantic Field Transform
Continuous semantic fields for language representation.

Neural-Pulse
Runtime verification of LLM behavior using internal signals.

Workshop Papers

Ghost Tracks: Hypothesis Competition in Sparse Feature Space as a Preliminary Signal of LLM Hallucinations
K. Khalil
Mechanistic Interpretability Workshop at ICML 2026. Poster.

The Singular Anchor: First Token Dominance in Large Language Model Attention Sinks
K. Khalil
5th Muslims in ML Workshop at NeurIPS 2025. Accepted.