Evaluation

Bergson supports validating model attributions using the linear datamodeling score (LDS), a widely used metric introduced in TRAK: Attributing Model Behavior at Scale, via bergson validate. This method compares summed attribution scores to ground-truth results over hundreds of re-training runs.

To evaluate attributions of large models that cannot be re-trained many times, we provide a synthetic factual dataset and its generator, and a proxy evaluation of how well attribution can be used to retrieve logically entailing data via bergson recall. The evaluation reports MRR and Recall@k.