Reading the ingest surface…
ChoonCatalogAdd references
demoThis build is a demo. The catalog and detections are a fixed sample.Scopereading…Sound is used for one thing: a short tone when a long scan finishes while this tab is in the background. The A/B listening check plays audio either way, because there the sound is the evidence.Reading the ingest surface…
Of the rows I submitted, which are now in the catalog, which are findable — and exactly which are neither, and why?
This is the sample shape. The form is hidden here on purpose: ?sample=1 exists so the response can be read when the service is down, and a live write launched from a page labelled sample is the confusion §13.3 exists to prevent. Read the live surface to run one.
334 rows accounted for at ingest, of 334 submitted — every row is in exactly one group.
Added is not the same as findable, and this run has not been verified.
every accepted row is pending QC; a reference is not known to be retrievable until it has been indexed and queried back. The catalog shows those rows as qc_status: pending with indexed: null — unknown, not false. Nothing follows from a clean scan on them until QC has run.
Sample data — A completed run in the endpoint’s own vocabulary, with a sample of each group’s rows. Source: src/lib/fixtures/ingest.ts. Nothing below was measured from your catalog.
Written to the catalog at qc_status: pending. In the catalog — and not yet known to be findable, which is a different claim and is made separately.
| Row | Recording | ISRC | Title | Artist | Reason |
|---|---|---|---|---|---|
| 0 | rec_00777 | USUM72100884 | Bright Room | Marguerite Osei | — |
| 1 | rec_00778 | USUM72100885 | Bright Room (radio edit) | Marguerite Osei | — |
| 2 | rec_01180 | FRZ109800142 | Cendre | Atelier Nord | — |
Showing 3 of 312 rows in this group.
The recording_id was already in this scope, so the row was upserted rather than inserted. Not an error and not a skip — re-running an ingest is expected to land here.
| Row | Recording | ISRC | Title | Artist | Reason |
|---|---|---|---|---|---|
| 40 | rec_00219 | GBAYE0601498 | Nocturne in Amber | Hélène Vaux | already in scope |
| 41 | rec_00447 | USUM71703861 | Slow Tide | Marguerite Osei | already in scope |
Showing 2 of 14 rows in this group.
Nothing was written for these rows. The endpoint reports one of three reasons — invalid (the row could not be built), rejected (the store refused it), write_failed (the write threw) — and they are three different problems.
| Row | Recording | ISRC | Title | Artist | Reason |
|---|---|---|---|---|---|
| 117 | rec_00551 | — | Ferrous | Dag Solheim | invalid KeyError: 'track_id' — the row carries no track id to index against |
| 203 | rec_01905 | NO-G38-22-0001 | Untitled Sketch 7 | Dag Solheim | invalid isrc malformed — 14 characters with separators, an ISRC is 12 |
| 259 | rec_00733 | NLA320900871 | Quiet Machines | Voorde | write_failed MonitorStoreError: could not reach the store — this row can be retried unchanged |
Showing 3 of 8 rows in this group. Fix the rows and re-submit only these. The rows that landed do not need re-sending.
Written to the catalog, then checked. QC could not read or could not retrieve the audio, so the recording is present in the catalog and absent from the index — the state that makes a clean scan meaningless.
| Row | Recording | ISRC | Title | Artist | Reason |
|---|---|---|---|---|---|
| 260 | rec_00992 | SEBTO2000455 | Vinter (live) | Åke Lindqvist | failed_unreadable FLAC stream ends mid-frame at 00:02:11 of a declared 00:04:52 |
| 301 | rec_00551 | — | Ferrous | Dag Solheim | failed_not_retrievable indexed 0 hashes — the reference was queried back and did not return itself |
Showing 2 of 2 rows in this group. Re-upload the audio and re-run QC. The catalog rows are already correct.
Rejected at ingest and failed QC after ingest are separate groups on purpose. A rejected row was never written and is fixed in the metadata; a QC failure was written, is in the catalog, and is fixed in the audio. Collapsing them into “10 errors” would send both fixes to the wrong person and would claim the catalog is missing rows it actually holds.
§13.6’s example puts a fourth group — 2 failed (unreadable audio) — beside the ingest counts, and POST /api/catalog/ingest does not report one. It writes rows at qc_status: pending and says why: “a reference is not known to be retrievable until it has been indexed and queried back”. Unreadable audio is found later, by QC, and lands on the catalog row as failed_unreadable. So a live run here shows only ingest-stage groups; the sample shows the QC stage as well, labelled after ingest, because the design and the code disagree about this and both readings are worth carrying. L5.
curl -s -X POST "$CHOON/api/catalog/ingest" \
-H 'content-type: application/json' \
-d '{"recordings":[{"recording_id":"rec_00777","track_id":777,
"audio_uri":"gs://audiofp-audio/references/rec_00777.wav",
"isrc":"USUM72100884","title":"Bright Room",
"main_artist":"Marguerite Osei","territories":["WW"]}]}'Identical to what the form sends, minus the proxy hop: from this origin the request goes through /api/catalog/ingest, which signs it with your gcloud identity — no API key is involved on this page or in the curl.
There is no file upload, and no CSV parser.
§13.6 describes ingest as “audio plus a metadata CSV”. The endpoint takes neither: it takes a JSON array of recordings that already point at audio in the bucket (audio_uri). A CSV control here would have to parse and map columns in the browser and would then post the same JSON — so the parser, not the ingest, is what is missing, and it is missing on both sides of the proxy.
Progress while a run is in flight is not modelled, because the endpoint is synchronous.
catalog_router.py says so in its cost note: at a few thousand rows an ingest is one multi-row insert, not a pipeline, and a queue would buy a standing resource to serve a request that takes milliseconds. §10.2’s counted-progress rendering — 312 of 4,000 rows, never a percentage — arrives if that ever stops being true. Until then the in-flight state above says the outcome is unknown rather than drawing a bar that is not measuring anything.