Retrieval demo

These are real rankings, not illustrations. Each query is encoded once by the frozen CLAP text encoder and scored against all 400 held-out test recordings by cosine similarity. The only difference between the two columns is whether the audio embedding is frozen CLAP or ResiCLAP.

Each result is a real recording of an object being struck. The name is the struck object — the thing making the sound, not a target. The test pool holds 400 recordings of just 5 objects (Mug (ceramic), Bin (plastic), Flowerpot (small ceramic), Vase (wood), Spatula (metal)), none of which the model saw during training, so the same name recurs often. Individual recordings differ by where the object was struck (one of 5 strike points) and by which microphone recorded it — the array spans 4 distance rings and a range of angles, both shown beneath each name. Two recordings of the same object can therefore sound quite different, which is exactly the distinction these queries are asking the model to make.

Be warned that the differences are meant to be subtle. Every clip in a given comparison is the same kind of event — a ceramic object being struck — and only the brightness, attack or decay varies. That is precisely why the task is hard for a general audio–language model, and why we measure the property relative to its own material rather than in absolute terms. Each query below therefore begins with two calibration clips that bracket the property, so you can hear the axis before judging a ranking. Headphones help considerably.

Try a query

All 24 evaluation queries are available. The shortcuts below show the three largest improvements and, for balance, the query where ResiCLAP helps least.

A result counts as relevant when it matches the queried material and falls on the correct side of that property's distribution among sounds of the same material — the same category-relative criterion used in the paper. Improvements are not uniform: some queries gain a great deal, others not at all. Rankings come from the seed-7 checkpoint, while the tables elsewhere on this site report means over seeds 7, 17 and 27, so a single query here may differ from the averaged figure.