fix(media): decode URL-encoded filenames and guard capture-year bounds #1028
Loading…
Reference in a new issue
No description provided.
Delete branch "fix/capture-year-guard"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Root cause
Ingested original filenames are stored percent-encoded in
original_assets.original_filename(e.g.video_87%4001-03-2019_10-15-42.mp4for real namevideo_87@01-03-2019_10-15-42.mp4).MediaJobHandlers.timestampFromFilenamematched its YYYY-MM-DD patterns against the encoded string, so%40+01read as year 4001 (greedy(\\d{4})), shifting month/day/hour too:capture_timestamp = 4001-03-20 19:10:15with confidenceinferred. No year sanity check existed anywhere on the write path, so the start page rendered timeline sections like "March 4001" (21 active media_items in prod).Fix (write side only; read queries and frontend untouched)
%HHonly, literal+preserved, malformed escapes fall back to the raw name) before applying the filename-date patterns — used for date extraction only, never for storage/echo.video_87%4001-...now decodes tovideo_87@01-03-2019_...and matches no pattern → item stays undated (correct outcome).captureTimestampinextractMetadatafor ALL sources (EXIF DateTimeOriginal, DateTimeDigitized, filename-derived): year < 1800 or > current year + 1 (worker zone) → value omitted from the emitted metadata map.WorkerResultService.acceptMetadataCandidatetreats an incomingcaptureTimestampwith an implausible year as absent — timestamp, timezone and confidence are not bound, soCOALESCEpreserves the stored values and the row never claims a timestamp it did not write.Tiny
isPlausibleCaptureYearhelper duplicated per module (no shared module exists; no new module wiring).Tests
MediaJobHandlersTest: encodedvideo_87%4001-03-2019_10-15-42.mp4→ no capture timestamp (regression);clip%402019-03-01T10-15-42.mp4→ decodes and yields2019-03-01T10:15:42Z;video_4001-01-01_10-00-00.mp4→ rejected by guard; existing sane-date tests unchanged.MLResultAcceptanceTest: candidate withcaptureTimestampyear 4001 leaves an existingcapture_timestamp/capture_timezone/timestamp_confidenceuntouched, and leaves a fresh item undated; sane timestamps still write (existing tests).rtk ./gradlew :apps:worker-media:test→ 29 tests, 0 failures.rtk ./gradlew :apps:api:test→ 1546 tests, 0 failures, 9 skipped (pre-existing assumptions).Refs: ISSUE-098 (KNOWN_ISSUES.md). Existing prod rows are repaired separately.
Ingested original filenames are stored percent-encoded in original_assets.original_filename (e.g. 'video_87%4001-03-2019_10-15-42.mp4' for the real name 'video_87@01-03-2019_10-15-42.mp4'). The media worker's timestampFromFilename matched its YYYY-MM-DD patterns against that encoded string, so the '%40'+'01' sequence read as year 4001 (greedy (\d{4})), shifting month/day/hour as well: capture_timestamp 4001-03-20 19:10:15 with confidence 'inferred'. No year sanity check existed anywhere on the write path, so the web start page rendered timeline sections titled 'March 4001' (21 active media_items affected in prod). Three-part write-side fix: - worker: percent-decode the filename (UTF-8 %HH only, literal '+' preserved, malformed escapes fall back to the raw name) before applying the filename-date patterns; the decoded value is used only for date extraction, so 'video_87%4001-...' now stays undated instead of yielding year 4001. - worker: guard the final captureTimestamp in extractMetadata for ALL sources (EXIF DateTimeOriginal, DateTimeDigitized, filename-derived); values with year < 1800 or > current year + 1 (worker zone) are omitted from the emitted metadata map. - api: WorkerResultService.acceptMetadataCandidate treats an incoming captureTimestamp with an implausible year as absent (timestamp, timezone and confidence are not bound), so COALESCE preserves the stored values and the row never claims a timestamp it did not write. Read side (TimelineQueryService, frontend) is untouched; timeline sections faithfully rendered the stored value. Existing rows are repaired separately.