# Compare language-model experiments

Drowse's Loom records a conversation as a branching tree. Keep a baseline, change a setting, and compare continuations from a shared point. Combine those branches with token inspection and completion scoring to examine what changed under a particular intervention.

Resource: https://drowse.ai/learn/experiments.md

## Branch from a turn or a token

Each authored or generated turn becomes a tree node. Navigate to an earlier turn, create a branch, regenerate a continuation, or compare nodes. Stars, notes, edge labels, and filters help organize a growing experiment. Saved chats preserve the tree so you can return to alternatives.

A token fork begins at a selected generation position. Choose a captured alternative token or supply replacement text, then continue from that prefix. This makes a local change inspectable alongside its original branch.

- [Conversation and token workflows](https://github.com/stdlogits/drowse/blob/main/README.md#the-dashboard)
- [Loom implementation](https://github.com/stdlogits/drowse.ai/blob/main/webui/src/hosted/runtime/browserLoom.ts)

## Change one control at a time

Keep the model, prompt, parent branch, seed, and sampling settings fixed when testing a steering coefficient. Compare an unsteered baseline with the intervention, then inspect both text quality and the relevant instrument readings. Save the recipe and annotate anything else that changed.

Temperature, top-p, top-k, and repetition penalties affect token selection. System instructions affect chat context. Steering affects internal activations. Seeds help comparisons within a fixed environment; identical behavior across devices, model revisions, or runtime builds is not guaranteed. One successful example is limited evidence of a general effect.

- [Activation steering guide](https://drowse.ai/learn/activation-steering.md)
- [Generation and comparison semantics](https://github.com/stdlogits/drowse.ai/blob/main/webui/docs/webmcp.md#choosing-controls)

## Score completion choices

The template lab defines a slot, candidate values, and one or more contexts. Restricted-choice scoring compares the model's likelihoods for those candidates, optionally under steering. Multi-token choices expose both summed log probability and a length-normalized view, with separate probabilities across the candidate set.

Use this when you want to compare a distribution rather than only the sampled completion. The result is conditional on the chosen candidates and contexts. A template can also supply the examples for a fitted manifold, connecting authoring and evaluation.

- [Browser template scoring source](https://github.com/stdlogits/drowse.ai/blob/main/webui/src/hosted/runtime/browserManifoldFitting.ts)
- [Template format and validation](https://github.com/stdlogits/drowse/blob/main/drowse/store/templates.py)

## Record and share the evidence

Export conversations, transcripts, or compatible manifold archives for reuse. Correlation and pairwise geometry tools help compare supported fitted directions across layers. Record the model, artifact source, prompt, settings, and comparison method with your result. Exported files may contain your experiment text, so review their contents before sharing.

- [Available analysis and export actions](https://github.com/stdlogits/drowse/blob/main/webui/src/lib/webmcp/catalogue.ts)
- [Python and agent workflows](https://drowse.ai/developers.md)

[Documentation index](https://drowse.ai/llms.txt) · [Open Drowse](https://drowse.ai/app) · [Source code](https://github.com/stdlogits/drowse)