ADR 0043: Parametrized Worker Capabilities
Status: proposed Date: 2026-06-04
Context
The current worker capability model is flat strings: a worker advertises ["face_detection", "semantic_embedding", "discovery_plants_image"] and Core dispatches jobs with matching job_type. This is a 1:1 mapping between capability and job type.
This works for simple cases but breaks down when workers need to express what they can do in more detail:
- Model conversion: A worker can convert
pytorch→onnxbut nottensorflow→onnx. Currently there's no way to express this — the worker would need separate capabilities for each conversion pair. - Thumbnail generation: A worker can produce JPEG and WebP thumbnails but not AVIF. No way to express format support.
- Embedding models: A worker can produce 512-dim CLIP embeddings but not 768-dim E5 embeddings. No way to express model/dimension compatibility.
- Inference strategies: A worker supports
whole_image_classifierbut notregion_detector. No way to express which strategies a worker implements.
The result is either too-generic capabilities (worker claims model_conversion but can only do one pair) or an explosion of capabilities (model_conversion_pytorch_onnx, model_conversion_tensorflow_onnx, etc.).
Decision
Extend the worker capability model with annotations — optional key-value pairs that describe the parameters of a capability.
Current Model
{
"workerId": "ml-worker-01",
"capabilities": ["face_detection", "semantic_embedding"]
}
Extended Model
{
"workerId": "ml-worker-01",
"capabilities": [
"face_detection",
{
"capability": "model_conversion",
"annotations": {
"sourceFormat": "pytorch",
"targetFormat": "onnx"
}
},
{
"capability": "generate_thumbnail",
"annotations": {
"outputFormats": ["jpeg", "webp"]
}
}
]
}
Rules
- Backward compatible: Plain string capabilities continue to work as before. Annotations are optional.
- Core matching: When a job requires specific annotations (e.g.,
sourceFormat=pytorch), Core matches against workers whose capability annotations satisfy the requirement. When a job has no annotation requirements, any worker with the base capability matches. - Worker declares, Core enforces: Workers declare what they can do. Core validates that dispatched jobs match the worker's declared capabilities. Workers must not claim jobs outside their declared capabilities.
- Annotations are opaque to Core: Core stores and matches annotations but does not interpret their meaning. The semantics of
sourceFormat,targetFormat,outputFormats, etc. are domain-specific and defined by the job type. - Discovery job annotations: For
discovery_*jobs, the annotations come from the domain's manifest (e.g.,inferenceStrategy: "whole_image_classifier"). Core matches workers that declare the same strategy in their capability annotations.
Matched Job Types
The first consumers of parametrized capabilities:
| Capability | Annotations | Use Case |
|---|---|---|
model_conversion |
sourceFormat, targetFormat |
PyTorch→ONNX, ONNX→Quantized |
generate_thumbnail |
outputFormats |
JPEG, WebP, AVIF thumbnails |
semantic_embedding |
modelId, dimensions |
CLIP 512-dim, E5 768-dim |
discovery |
domainKey, inferenceStrategy |
Domain-specific discovery |
Database Changes
The worker_capabilities or claim request currently stores capabilities as TEXT[]. The extended model stores capabilities as JSONB:
[
"face_detection",
{"capability": "model_conversion", "annotations": {"sourceFormat": "pytorch", "targetFormat": "onnx"}}
]
Workers continue to claim with a flat list of capability strings (for backward compatibility). Core enriches the claim with annotation matching internally.
Consequences
- Workers gain expressiveness without breaking existing deployments.
- Core's job dispatch becomes more nuanced — annotation-aware matching for jobs that carry annotation requirements.
- The
model_conversioncapability (ADR 0044) anddiscoverywithinferenceStrategyare the first consumers. - Future capabilities (thumbnail formats, embedding models) can use the same pattern without Core code changes.
Memories Wiki
Using Memories
- Accounts, the Library Hub, and Data Export
- Albums and Automatic Suggestions
- Explore and Discovery Domains
- The Media Viewer
- People and Faces
- Places and Map
- Plants (In Development)
- Datenschutzerklärung (Privacy Policy)
- Fast, Progressive Image Loading
- Search
- Sharing and Households
- Timeline and Photos
- Uploading and Import
Self-Hosting
- Backup and Restore
- CLI Password Reset
- Deployment Architecture
- Known Issues and Operator Diagnostics
- Map Tiles and Place Data Privacy
- Storage Layout
- Upgrades and Versioning
- Workers
Reference