Predictive Geometry of LLM Representations

Home Research Publications Projects Service Updates Blog CV

This is the direction I am most fascinated by right now: whether the geometry of LLM representations can predict behavior before the model commits to an output. I study internal signals such as attention sinks, first-token dominance, semantic trajectories, hidden-state geometry, and competing hypothesis traces as early indicators of hallucination, instability, or reliable reasoning.

The long-term goal is to turn representation geometry into a practical predictive layer: a way to understand, monitor, and potentially intervene in model behavior before the final response is generated.

Relevant Papers and Projects

The Singular Anchor: First Token Dominance in Large Language Model Attention Sinks
K. Khalil
5th Muslims in ML Workshop at NeurIPS 2025. Accepted.
GhostTrack
Mechanistic hypothesis and semantic tracking for LLMs.

Framework for tracing competing thought trajectories and identifying hallucination risks before final output.

Attention Sink Analysis
Transformer attention analysis.

Tools for analyzing first-token dominance and attention sink behavior in long-context language models.

Semantic Field Transform
Continuous semantic fields for language representation.
Neural-Pulse
Runtime verification of LLM behavior using internal signals.