2 ADR 0114 representative thumbnail is manual pin or medoid of largest cluster
Forgejo Actions edited this page 2026-10-17 07:47:45 +00:00

ADR 0114: Representative thumbnail of a class instance is the manual pin or the medoid of its largest cluster

Status: proposed Date: 2026-10-16

Context

Whenever a class instance is referenced or displayed — instance cards, people browse, person detail headers, related-person chips, merge cards, suggestion carousels, collaborative person views, self-person avatar fallback, ranking preview URLs — it must show ONE stable, canonical thumbnail. Today the codebase contradicts this in two ways. (1) The fallback selection is duplicated in at least five divergent fragments that have already produced a recurring bug class (ISSUE-095, ISSUE-140): ClassInstanceDiscoveryService.instances() uses newest accepted/tentative occurrence and IGNORES the manual pin; ClassInstanceService.representativeEnrichmentForPeople uses an accepted -> face-crop -> bbox-band -> area -> confidence -> recency ladder; PersonRelationService carries an admitted copy of that ladder; CollaborativeClassInstancePeopleBridge has a third variant with a different ordering; SelfPersonService and ClassInstanceFaceThumbnailService each carry their own recency fallback; REPRESENTATIVE_FACE_THUMBNAIL_SUBQUERY is a sixth SQL copy used by ranking endpoints. The observable effect is that a person card can show a different, effectively arbitrary (newest or last-write-wins) photo on every code path and can silently change as the library grows. (2) No medoid exists anywhere in production: the people worker (ADR 0108/0109) decomposes each person's faces into subclusters and represents each cluster by its unit mean CENTROID, ships only (repIndex, clusterSize, embedding), and the subcluster membership is recomputed per run and never persisted — so the member face behind a cluster cannot be recovered server-side today. The worker's medoid machinery (experiments/assignment_cv/medoid.py) is CV-experiment-only. ADR 0094 specifies the CROP GEOMETRY of the instance thumbnail but not WHICH occurrence is selected; ADR 0037/0038 govern class-instance governance; specs 026/031 currently document the rule as 'manual override else most recent accepted', which is exactly the unstable behavior being replaced.

Decision

One canonical representative-thumbnail rule for ALL class-instance references, resolved by ONE shared backend entry point (no per-service copies): (1) MANUAL PIN WINS: if the instance carries a USER-SELECTED representative (representative_occurrence_id / representative_media_item_id with provenance manual_selection, V077 semantics, mirrored on collaborative_class_instances), that occurrence and its crop (ADR 0094) is the thumbnail. Pins with any other provenance (auto-pinned first-accepted rows from the removed auto-pin path, migration backfill without manual_selection) are NOT manual and fall through to rule 2 — existing non-manual pins therefore change resolution once, deliberately. (2) OTHERWISE: the thumbnail is the MEDOID OF THE LARGEST CLUSTER of the instance's usable occurrence embeddings, largest = MOST MEMBER EMBEDDINGS. Medoid = the actual member occurrence (a real, displayable face/occurrence) minimizing the sum of cosine distances to the other members of its cluster — a member, never a synthetic centroid; a singleton cluster's single member is its own medoid; equal medoid sums are broken by smallest member face annotation id. Persons may hold SEVERAL medoids: their clusters are exactly the ADR 0108 average-linkage subclusters of the assigned usable faces at the job-snapshotted representative_split_distance, computed in the same decomposition pass that produces the identity representatives, for every clustering method version (v1: one cluster, i.e. one medoid over all usable faces; v2/v3: one medoid per subcluster; largest subcluster wins, size ties by smallest member face annotation id). Non-person domains (and any instance without a decomposition) treat the instance's usable class_occurrence_embeddings (ADR 0093) as exactly one cluster and select its single, unique medoid by the same rule — the rule is uniform, only the cluster count differs. (3) Deterministic terminal fallback when no usable embeddings exist (faceless person, occurrence without embedding yet, worker data not yet persisted): accepted-first, then unified quality score (ADR 0103) descending where persisted, else confidence descending, then newest, then id descending — ONE shared fragment, identical in SQL and Java paths. (4) The people worker's personRepresentatives payload gains an ADDITIVE field per representative: the medoid member face annotation id (additive alongside ADR 0113's fingerprint field — both may ship in the same envelope). The API persists it as an additive column on person_instance_representatives (V156 lineage) in the same transaction as candidate ingestion. This is deterministic derived evidence over owner-accepted faces only — it is NOT a new acceptance, label, or membership decision and stays inside ADR 0038's worker-evidence boundary; until a run ships it, persons resolve via rule 3. (5) Identity matching, merge ranking and graph construction (ADR 0108/0109/0113, pessimistic per-representative scoring, representative graph nodes) KEEP unit-mean centroids unchanged — the medoid is a DISPLAY selection only and never enters identity scoring or graph nodes. (6) Instance-crop generation (generate_discovery_thumbnail -> class_instances.thumbnail_derived_asset_id, V099) must be enqueued from the canonical representative occurrence only; the current last-write-wins overwrite from any accepted occurrence is removed, and re-pinning re-enqueues the crop job. (7) The auto-pin of the first accepted occurrence when representative_occurrence_id is null is REMOVED: it fabricates a pseudo-manual pin and would freeze an arbitrary photo as 'manual'. Only explicit user selection yields manual_selection provenance. (8) The frontend consumes representativeThumbnailUrl / cropped variants only; no client-side image picking, no per-component fallback chains. (9) CONFIRMED IMAGES ONLY: only accepted (confirmed/labeled) occurrences may serve as a representative — the manual-pin write path enforces state accepted, and the medoid operates exclusively over embeddings derived from owner-accepted/assigned faces (tentative occurrences are unlabeled pool members per ADR 0102 and never enter the decomposition). The SOLE exception is the terminal fallback for UNNAMED candidate instances, which may show a tentative crop when no accepted occurrence exists (otherwise new suggestions would render without any figure); NAMED instances never serve a tentative occurrence — a named instance with only tentative occurrences has no representative at all.

Consequences

Positive: a class instance shows the SAME thumbnail in every UI surface, and it only changes when the user re-pins or when clustering genuinely recomputes — the 'random/newest photo' symptom class (ISSUE-095/140 family) is eliminated structurally by the single entry point; the five-plus divergent SQL/Java fragments collapse into one shared service + one shared SQL subquery; cards show a representative MEMBER (real face, recognizable) rather than an arbitrary newest photo; manual pins are honored on every path including listings that previously ignored them. Supersedes at spec level: this ADR replaces the 'manual override else most recent accepted occurrence' rule documented in specs/026-discovery-domain-instance-browse (AC6, T001) and docs/superpowers/specs/2026-06-25-discovery-domain-instance-browse-design.md — those artifacts are amended to reference this ADR when the implementation lands; no ADR is superseded. Costs / risks: worker change is required (medoid member id in personRepresentatives) plus a schema extension to persist it — until the first post-deploy worker run, person instances fall back to the terminal ladder, so person cards may change once when the run lands (accepted, one-time); existing non-manual pins (auto-pin legacy, backfill) stop winning and resolve via medoid (one-time); non-people domains need the same medoid derivation from class_occurrence_embeddings (V101) server-side since their workers ship no decomposition — implemented once in the shared entry point; clustering recompute can change the largest cluster and thus the thumbnail (inherent to the rule, deterministic per run); removal of the auto-pin changes behavior for instances that relied on it (their representative column becomes null and resolves via medoid — which is the intended semantics); thumbnail_derived_asset_id becomes derived-from-representative, so re-pinning must re-enqueue the crop job. Non-goals: albums have no cover field and are out of scope for this ADR (separate decision if wanted); map-grid representativeMarker is a different concept and untouched; ADR 0094 crop geometry is unchanged.