Physics-grounded AI research lab

Preventing AI from wiping out humanity.

We study what drives AI behavior so we can steer it as it grows more powerful.

YAT response surface
2 of 5 neurons Add a neuron to see how it reshapes the surface.

What makes the model say “Mexico City”?

Try replacing an activation from the Mexico prompt with one from the Peru prompt. Compare the targeted swap and the control to see what would support the hypothesis that the path carries country information.

Illustrative experiment

Prompt “The capital of Mexico is”

Choose an intervention
Candidate path
Mexico activation
Unrelated path
Mexico activation

Hypothetical result Mexico City stays favored

Read the experimental protocol
01 / Trace Follow the signal
Compare the Mexico prompt with the same prompt about Peru. Look for a path whose activity tracks the country as the rest of the sentence stays fixed.
02 / Intervene Swap the activation
Run the Mexico prompt again and patch the candidate component with its activation from the Peru run. Compare how the scores for Mexico City and Lima shift.
03 / Validate Check what travels
Compare with a patch on an unrelated component, then repeat across countries and phrasings. A repeatable, selective shift would support the circuit hypothesis.

What this opens up. The same experiment can test whether a shortcut drives a failure, whether a circuit survives fine-tuning, or whether a targeted edit changes the intended behavior without disturbing nearby capabilities.

Our research

A kernel theory of how models compute

What makes a neuron respond? What can a layer represent? What information can survive through depth? YAT gives us a mathematical way to ask these questions.

YAT kernel

ⵟ(x, w) = (x·w + b)² / (‖x − w‖² + ε)
Alignment
Which directions agree?
Distance
How close are the vectors?

YAT scores an input against a learned center using both alignment and distance. At a fixed alignment, greater distance weakens the response. Along some directions, alignment can sustain a response even far from the center. The gravity well gives a visual intuition for this geometry.

01 / Neuron Explain the response
The kernel separates the role of alignment from the role of distance. That gives us a mathematical reason why inputs can activate the same unit, even when they are far apart.
02 / Layer Describe what it can represent
With shared positive bias and regularization, a YAT layer is a weighted sum of kernel responses. The theory gives that layer an exact function-space norm and characterizes what the kernel can approximate.
03 / Depth Explain what can carry forward
Once earlier layers are fixed, they induce a kernel on the original inputs. If those layers map two inputs to the same representation, the next layer cannot distinguish them.

Research papers

From the mathematics of YAT to faster attention and kernel approximations.

  1. 01 / Foundations

    A Universal Reproducing Kernel Hilbert Space from Polynomial Alignment and IMQ Distance

    Taha Bouhsine

    Introduces the YAT kernel and shows what changes when a model uses both alignment and distance to compare inputs.

  2. 02 / Attention

    SLAY: Geometry-Aware Spherical Linearized Attention with Yat-Kernel

    Jose Miguel Luna, Taha Bouhsine, Krzysztof Choromanski

    Uses YAT geometry to build attention that scales linearly with sequence length.

  3. 03 / Approximation

    Bernstein–Schur Kernels: Random Features by Sketched Modulation and Radial Randomization

    Taha Bouhsine

    Approximates kernel computations with compact random features so they can be used on larger datasets.

The same answer can hide different computations

The two examples use the same prompt. One shows the prediction. The other illustrates a path we could test.

See the answer

The answer tells us what the model predicted. It leaves open which features the model used and how their signals reached the output.

Black box Output only
Prompt"the capital of Mexico is"
Output▋
· · · more layers above · · · · · · more layers below · · · BLACK BOX illustrative network INPUT HIDDEN LAYERS OUTPUT
Why tracing is hard: A neuron can respond to several features, and one feature can be spread across many neurons. An activation alone gives us a clue about the computation.

Trace a candidate path

A trace through a small illustrative network shows what a candidate circuit could look like. A real circuit explanation also needs an intervention test.

Circuit trace Tracing the network
Prompt"the capital of Mexico is"
Output▋
· · · illustrative network · · · INPUT HIDDEN LAYERS OUTPUT

How we test a circuit

Locate
Find components whose responses are relevant to the behavior.
Trace
Follow how their responses combine across layers.
Test
Perturb a component and compare the result with a baseline.
Design
Use those findings to build models that are easier to inspect.

Founding team

Co-founder & CEO

Taha Bouhsine

Studies the geometry of neural networks and develops methods for interpreting how models compute.

LinkedIn ↗

Co-founder & CPO

Jose Miguel L.

Brings AI product experience from Apple and co-authors Azetta’s research on geometry-aware attention.

LinkedIn ↗

Hugging Face

Models and checkpoints

Browse language and embedding models from mlnomad, or pick a checkpoint to reproduce a paper run.

Multilingual encoders

GOAT and YAT mmBERT releases for masked language modeling, text embeddings, and retrieval.

YAT paper checkpoints

Matched 261M language models, seed 0. Each link opens the exact revision cited in the paper.

Appendix G of the YAT paper links all 15 runs, with checkpoint revisions and training and evaluation records.