feat(places): local gazetteer place search API #1190

Merged
chris merged 1 commit from feat/place-search-api into main 2026-10-13 10:16:30 +02:00
Owner

Adds the forward-geocoding counterpart to the existing reverse place lookup: an authenticated (standard user, no admin) GET /api/places/search?q=<string>&limit=<int> endpoint that searches the already-imported local GeoNames gazetteer by name and alternative names.

Contract

  • q: required, 2–100 chars after trim, else 400 (IllegalArgumentException → existing ApiExceptionHandler).
  • limit: optional, default 10, allowed 1–50, else 400.
  • 200 response: {"results": [{providerNativeId, name, level, latitude, longitude, countryCode, admin1Code, featureCode}]} — camelCase, consistent with all other API responses (no global naming strategy is configured). No match or no imported dataset → 200 with empty results.

SQL approach
Search runs over name_variants (main names + imported alternates) with a single parameterized ILIKE-style LIKE (LOWER(name) LIKE '%' || :escaped || '%', user wildcards escaped, LIMIT enforced, no dynamic SQL). Match classes (exact/prefix/substring) are computed in SQL via a CASE expression; DISTINCT ON (provider_native_id) deduplicates to the best variant per place; outer ranking: match class → prominence_score DESC NULLS LAST (the import stores log10(population), so this is the data-driven population surrogate — place_name_candidates has no population column) → level weight (city/locality/venue before admin_region before geographic_area before country) → display name as stable tiebreaker. Scope mirrors PlaceLookupService.lookup(): only enabled providers joined with completed datasets; the synthetic fallback level is excluded. No results also for schemas without any import.

Migration V163
pg_trgm was not enabled anywhere, so V163 adds CREATE EXTENSION IF NOT EXISTS pg_trgm plus a GIN trigram index on lower(name_variants.name) so '%q%' substring search is index-backed instead of seq-scanning the full gazetteer, and a (dataset_id, provider_native_id) index on place_name_candidates for the join path. A prefix-only plan without trigram would leave substring search quadratic on the variants table, which is the dominant query shape here.

Tests

  • PlaceSearchServiceTest (15, Testcontainers + PostGIS, reuses importFixture): exact/prefix/substring, case-insensitivity, ranking within classes (prominence, level weight, NULLS LAST), German alternate-name hit returning the display name, dedup across multiple variants, disabled-provider/draft-dataset exclusion, fallback exclusion, literal wildcard handling, empty/no-data → empty list, q/limit validation, and an EXPLAIN check that the trigram index is a usable access path.
  • PlaceSearchControllerTest (5): auth required, standard user without admin role, validation propagation, default limit, camelCase serialization.
  • Full :apps:api suite: 1990 tests, 0 failures (includes all 34 existing places tests).
Adds the forward-geocoding counterpart to the existing reverse place lookup: an authenticated (standard user, no admin) `GET /api/places/search?q=<string>&limit=<int>` endpoint that searches the already-imported local GeoNames gazetteer by name and alternative names. **Contract** - `q`: required, 2–100 chars after trim, else 400 (`IllegalArgumentException` → existing `ApiExceptionHandler`). - `limit`: optional, default 10, allowed 1–50, else 400. - 200 response: `{"results": [{providerNativeId, name, level, latitude, longitude, countryCode, admin1Code, featureCode}]}` — camelCase, consistent with all other API responses (no global naming strategy is configured). No match or no imported dataset → 200 with empty `results`. **SQL approach** Search runs over `name_variants` (main names + imported alternates) with a single parameterized ILIKE-style `LIKE` (`LOWER(name) LIKE '%' || :escaped || '%'`, user wildcards escaped, LIMIT enforced, no dynamic SQL). Match classes (exact/prefix/substring) are computed in SQL via a CASE expression; `DISTINCT ON (provider_native_id)` deduplicates to the best variant per place; outer ranking: match class → `prominence_score DESC NULLS LAST` (the import stores log10(population), so this is the data-driven population surrogate — `place_name_candidates` has no population column) → level weight (city/locality/venue before admin_region before geographic_area before country) → display name as stable tiebreaker. Scope mirrors `PlaceLookupService.lookup()`: only enabled providers joined with `completed` datasets; the synthetic `fallback` level is excluded. No results also for schemas without any import. **Migration V163** `pg_trgm` was not enabled anywhere, so V163 adds `CREATE EXTENSION IF NOT EXISTS pg_trgm` plus a GIN trigram index on `lower(name_variants.name)` so '%q%' substring search is index-backed instead of seq-scanning the full gazetteer, and a `(dataset_id, provider_native_id)` index on `place_name_candidates` for the join path. A prefix-only plan without trigram would leave substring search quadratic on the variants table, which is the dominant query shape here. **Tests** - `PlaceSearchServiceTest` (15, Testcontainers + PostGIS, reuses `importFixture`): exact/prefix/substring, case-insensitivity, ranking within classes (prominence, level weight, NULLS LAST), German alternate-name hit returning the display name, dedup across multiple variants, disabled-provider/draft-dataset exclusion, fallback exclusion, literal wildcard handling, empty/no-data → empty list, q/limit validation, and an EXPLAIN check that the trigram index is a usable access path. - `PlaceSearchControllerTest` (5): auth required, standard user without admin role, validation propagation, default limit, camelCase serialization. - Full `:apps:api` suite: 1990 tests, 0 failures (includes all 34 existing places tests).
feat(places): local gazetteer forward place search API
All checks were successful
Wiki Sync / Sync wiki from docs (push) Has been skipped
Build and Deploy / Detect changed services and test scopes (push) Successful in 3s
Build and Deploy / Android unit tests and debug APK (push) Has been skipped
Build and Deploy / Smoke API and media worker (push) Has been skipped
Build and Deploy / Smoke web (push) Has been skipped
Build and Deploy / Smoke ML worker (push) Has been skipped
Build and Deploy / Smoke people worker (push) Has been skipped
Build and Deploy / Build postgres image (push) Has been skipped
Build and Deploy / Build plants worker image (push) Has been skipped
Build and Deploy / Build converter worker image (push) Has been skipped
Build and Deploy / Deploy postgres (push) Has been skipped
Build and Deploy / Build API image (push) Has been skipped
Build and Deploy / Build media worker image (push) Has been skipped
Build and Deploy / Build ML worker image (push) Has been skipped
Build and Deploy / Build people worker image (push) Has been skipped
Build and Deploy / Build web image (push) Has been skipped
Build and Deploy / Deploy API (push) Has been skipped
Build and Deploy / Deploy web (push) Has been skipped
Build and Deploy / Deploy media worker (push) Has been skipped
Build and Deploy / Deploy ML worker (push) Has been skipped
Build and Deploy / Deploy plants worker (push) Has been skipped
Build and Deploy / Deploy people worker (push) Has been skipped
Build and Deploy / Deploy converter worker (push) Has been skipped
Build and Deploy / Clean Docker build artifacts (push) Has been skipped
Build and Deploy / Run scoped PR tests (push) Successful in 2m15s
38ebffa82a
Add GET /api/places/search (standard user auth, no admin) that matches
the query against name_variants of enabled providers with completed
datasets and returns place data from place_name_candidates.

- Matching: case-insensitive exact, then prefix, then substring over all
  imported name variants (incl. alternates); LIKE wildcards in user
  input are escaped, everything parameterized, LIMIT enforced.
- Ranking: match class, then prominence_score DESC NULLS LAST (imported
  as log10(population)), then level weight (city/locality/venue <
  admin_region < geographic_area < country), then display name as stable
  tiebreaker. Deduplicated to one row per provider_native_id; the
  synthetic 'fallback' level is not searchable.
- Validation: q required, 2..100 chars after trim; limit optional,
  default 10, 1..50; violations map to 400 via IllegalArgumentException.
  No match or no imported data returns 200 with empty results.
- Migration V163: pg_trgm extension + GIN trigram index on
  lower(name_variants.name) and (dataset_id, provider_native_id) index
  on place_name_candidates for the join path.

Tests: 15 service (Testcontainers+PostGIS, exact/prefix/substring,
ranking incl. NULL prominence, alternates, dedup, provider/dataset
status filters, index-plan check) + 5 controller (auth, validation,
camelCase serialization). Full :apps:api suite green (1990 tests).
chris merged commit 88fd92a42f into main 2026-10-13 10:16:30 +02:00
Sign in to join this conversation.
No reviewers
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
chris/memories!1190
No description provided.