translations to-spreadsheet: export only the changed objects of a metadata file (--metadata-file, --only-changed, --data-set-ids)

README.md

@@ -282,12 +282,32 @@ $ yarn start translations to-spreadsheet \
   translations.xlsx
 ```
 
+Export only the objects of a metadata package that are new or renamed with respect to the instance, restricted to a translated data set (the typical "what do translators need for this feature" sheet):
+
+```shell
+$ yarn start translations to-spreadsheet \
+  --url='http://USER:PASSWORD@HOST:PORT' \
+  --metadata-file=feature-metadata.json \
+  --only-changed \
+  --default-locale=en \
+  --exclude-names='^\[DEPRECATED\]' \
+  --data-set-ids=NQOwInnRDNL \
+  --models='dataElements[formName],indicators[name]' \
+  --locales='Spanish,French' \
+  --include-data \
+  translations.xlsx
+```
+
 Notes:
 
 -   `--models`: comma-separated list of models to export. Each model must specify its translatable fields with `[field1,field2]` (e.g. `indicators[name,shortName]`); a model without fields raises an error.
 -   `--locales`: comma-separated list of locale names to include as columns, in the order given. The match ignores any ` (...)` suffix, so `Spanish` matches a `Spanish (Spain)` locale.
 -   `--include-data`: write one row per object with the source values and the existing translations. When omitted, only the header row is written (a column template).
--   `--program-id=ID` / `--data-set-id=ID` (exclusive): scope the export to the objects in that program's or data set's metadata dependency export (`/api/programs/{id}/metadata`, `/api/dataSets/{id}/metadata`) instead of the whole instance. Only the requested `--models` are kept from the export.
+-   `--program-ids=ID1,ID2` / `--data-set-ids=ID1,ID2` (exclusive): scope the export to the objects in those programs' or data sets' metadata dependency exports (`/api/programs/{id}/metadata`, `/api/dataSets/{id}/metadata`) instead of the whole instance. Only the requested `--models` are kept from the exports; an object shared by several parents appears once. With `--metadata-file`, `--data-set-ids` keeps the objects of the file belonging to those data sets (their data elements, indicators, sections and the options of those data elements).
+-   `--metadata-file`: read the objects from a DHIS2 metadata JSON export (`{"dataElements": [...], ...}`) instead of the instance. Useful when the metadata to translate is not yet deployed to a trusted instance. `--url` is still used to get the locales and as the reference for `--only-changed`.
+-   `--only-changed`: export only the objects of `--metadata-file` that need (re)translation: those not existing in the instance, or whose selected fields differ, or whose `--default-locale` translation of a selected field differs (a label can be changed only through that translation). Unselected fields (like a `name` prefix) are ignored.
+-   `--default-locale`: locale code of the default (DB) language, e.g. `en`. Matched by language, so `en` also matches `en_GB`.
+-   `--exclude-names`: regex; objects whose `name` matches are skipped. Example: `'^\[DEPRECATED\]'`.
 -   One sheet (tab) is generated per model type.
 -   Columns: `Type`, `UID`, then a group per field: the base source column `<field>` followed by one `<field>: <LocaleName>` column per selected locale.
 -   Each field group is color-coded (bold colored header, source column highlighted, translation cells lightly tinted) so the grid is easy to scan. The header row and the first three columns (`Type`, `UID` and the first source column) are frozen, and columns within a field group share the same width.

openspec/changes/export-translations-only-changed/.openspec.yaml

@@ -0,0 +1,2 @@
+schema: spec-driven
+created: 2026-09-18

openspec/changes/export-translations-only-changed/design.md

@@ -0,0 +1,68 @@
+## Context
+
+`translations to-spreadsheet` (`ExportTranslationsUseCase`) fetches all objects of each model
+through `MetadataRepository.getAllWithTranslations` and hands one `ModelTranslationsExport` per
+model to `ExportTranslationsSpreadsheetRepository`. The source of the objects and the reference
+instance were the same thing.
+
+## Decisions
+
+### Decision: Separate "source of objects" from "reference instance"
+
+A narrow `MetadataSourceRepository` interface (`getAllWithTranslations` only) is the source of the
+objects to export; `MetadataRepository` extends it, so the instance keeps being the default source.
+`MetadataJsonFileRepository` (data layer) implements the source from a metadata JSON export: it
+pluralizes the requested models, tags each object with its `model`, defaults `translations` to `[]`
+and keeps every other field so source columns (`formName`, ...) can be filled.
+
+The use case receives an optional `metadataSource`; the command handler builds it from
+`--metadata-file`. Locales and the reference objects always come from the instance (`--url`).
+
+### Decision: Reference lookup by id, chunked
+
+`MetadataRepository.getByIdsWithTranslations(model, ids)` fetches only the file's ids from the
+instance (`/api/metadata?model:fields=:owner&model:filter=id:in:[...]`, chunks of 100) instead of
+downloading the whole model. Objects missing from the response are "new".
+
+### Decision: Change detection is a pure domain function
+
+`isChanged(object, reference, fields, defaultLocale)` compares, for the selected fields only:
+the trimmed field value and (when a default locale is given) the trimmed translation of that field
+in the default locale. Non-selected fields are ignored on purpose: a `[DEPRECATED]` prefix on
+`name` does not require re-translating `formName`. The default-locale comparison covers labels
+that are changed only through the `en` translation, which is how some projects keep a long English
+label separate from the short `formName`.
+
+### Decision: Data set scope of a file resolved by membership
+
+`--data-set-ids` is one option with one meaning ("the objects of these data sets") and two
+implementations behind `MetadataSourceRepository.getAllWithTranslations(models, { dataSetIds })`:
+the instance merges the dependency exports of each id (#106, generalized to a list, deduplicated
+by model+id); the file, which has no such export, builds a `DataSetScope` (domain entity) from
+its own data sets and data elements: `dataSetElements` give
+the data elements, the data set `indicators` give the indicators, the scoped data elements'
+`optionSet` refs give the option sets; sections match through their `dataSet` ref.
+`isInDataSetScope` dispatches on the object's model and rejects models with no data set relation,
+so a wrong `--models` fails fast. `--program-ids` with a file is rejected.
+
+### Decision: Java legacy locale codes
+
+DHIS2 stores Indonesian translations with locale `in` while `/api/locales/db` reports `id`
+(same for `iw`/`he`, `ji`/`yi`). `normalizeLocaleCode` in the `Locale` entity maps legacy codes
+and `isSameLocale`/`haveSameLanguage` use it; the spreadsheet repository and the change detection
+match translations through it, so existing Indonesian translations appear in their column.
+
+## Data flow
+
+CLI args → `MetadataJsonFileRepository` (file, scoped by `dataSetIds`) + `MetadataD2Repository`/
+`LocalesD2Repository` (instance) → `ExportTranslationsUseCase.getObjects` (exclude by name →
+fetch reference by ids → `isChanged`) → `ModelTranslationsExport[]` →
+`ExportTranslationsSpreadsheetRepository.save`.
+
+## Risks / Trade-offs
+
+-   The file's own translations are what the sheet shows as "existing" translations. They are
+    expected to have been exported from the reference instance; otherwise translators see stale
+    values (they are still re-translating the row anyway).
+-   `--only-changed` without `--metadata-file` is rejected: comparing the instance with itself
+    yields an empty sheet.

openspec/changes/export-translations-only-changed/proposal.md

@@ -0,0 +1,49 @@
+## Why
+
+When a feature ships new or renamed metadata (e.g. the CPR "Flex" package), translators need a
+spreadsheet with just those objects, not the thousands already translated. Today
+`translations to-spreadsheet` exports every object of a model from an instance, and the new
+metadata may only exist as a JSON package (or in an instance that also carries unrelated work),
+so the delta had to be computed by hand with ad-hoc scripts.
+
+## What Changes
+
+```sh
+yarn start translations to-spreadsheet \
+  --url=REFERENCE_INSTANCE \
+  --metadata-file=feature-metadata.json \
+  --only-changed \
+  --default-locale=en \
+  --exclude-names='^\[DEPRECATED\]' \
+  --data-set-ids=DS1,DS2 \
+  --models='dataElements[formName],indicators[name]' \
+  --locales=... --include-data out.xlsx
+```
+
+-   `--metadata-file=PATH`: objects (with their translations) are read from a DHIS2 metadata JSON
+    export instead of the instance. `--url` remains required: locales come from the instance, and
+    it is the reference for `--only-changed`.
+-   `--only-changed`: keep only objects that do not exist in the instance, or whose selected fields
+    differ, or whose `--default-locale` translation of a selected field differs. Requires
+    `--metadata-file`.
+-   `--default-locale=CODE`: language matched (`en` ~ `en_GB`), same semantics as the import side.
+-   `--exclude-names=REGEX`: skip objects whose `name` matches.
+-   `--program-id`/`--data-set-id` (from #106) become `--program-ids`/`--data-set-ids`, taking
+    comma-separated IDs (plural naming as in the other commands); the dependency exports of all
+    the IDs are merged. `--data-set-ids` also applies to `--metadata-file`: the file's objects are
+    kept by membership (data elements, indicators, sections, and the options of the data
+    elements' option sets), since a file has no dependency export. `--program-ids` is not
+    supported for a file.
+-   Translations stored with a Java legacy language code (`in` for Indonesian) are now matched to
+    the DB locale (`id`) when filling the existing-translation columns.
+
+Existing invocations are unaffected: without the new options the behavior is unchanged.
+
+Builds on #106 (`--program-id`/`--data-set-id`, short locale references, `Name` column). Old vs
+new interface: `--program-id=ID` → `--program-ids=ID1,ID2`, `--data-set-id=ID` →
+`--data-set-ids=ID1,ID2` (#106 is unmerged, so no released interface changes).
+
+## Non-goals
+
+-   Diffing against a second instance (reference is always `--url`).
+-   Filtering instance exports by DHIS2 filter expressions; the delta case is served by the file.

openspec/changes/export-translations-only-changed/specs/export-translations-to-spreadsheet/spec.md

@@ -0,0 +1,91 @@
+## ADDED Requirements
+
+### Requirement: Export objects from a metadata JSON file
+
+The `translations to-spreadsheet` command SHALL accept `--metadata-file=PATH`, a DHIS2 metadata
+JSON export (`{"dataElements": [...], ...}`), and take the objects to export (with their
+`translations`) from it instead of from the instance. Models are matched by their plural key. The
+instance at `--url` SHALL still provide the locales.
+
+#### Scenario: Objects come from the file
+
+- **WHEN** the command runs with `--metadata-file=pkg.json --models='dataElements[formName]'`
+- **THEN** the `dataElements` sheet lists the file's data elements, with their `formName` and the
+  translations present in the file, and no data element is read from the instance
+
+### Requirement: Export only the objects that need translation
+
+With `--only-changed`, the command SHALL keep an object only when it does not exist in the
+instance, or when any selected field differs from the instance value, or when the
+`--default-locale` translation of a selected field differs. Values are compared trimmed; fields
+not selected in `--models` SHALL be ignored. `--only-changed` SHALL require `--metadata-file`.
+
+#### Scenario: Renamed label is exported, deprecation prefix is not
+
+- **WHEN** the file has data element A with `formName` changed and data element B whose only
+  change is a `[DEPRECATED]` prefix on `name`, with `--models='dataElements[formName]'`
+- **THEN** A is in the sheet and B is not
+
+#### Scenario: Label changed only through the default-locale translation
+
+- **WHEN** a data element keeps its `formName` but its `en` `FORM_NAME` translation differs from
+  the instance, and the command runs with `--default-locale=en`
+- **THEN** the data element is in the sheet
+
+#### Scenario: Only-changed without a file is rejected
+
+- **WHEN** the command runs with `--only-changed` and no `--metadata-file`
+- **THEN** it exits with an error naming both options
+
+### Requirement: Exclude objects by name
+
+The command SHALL accept `--exclude-names=REGEX` and skip any object whose `name` matches it,
+before change detection.
+
+#### Scenario: Deprecated indicators are skipped
+
+- **WHEN** the command runs with `--exclude-names='^\[DEPRECATED\]'` and `indicators[name]`
+- **THEN** indicators whose name starts with `[DEPRECATED]` are not in the sheet even though their
+  `name` changed
+
+### Requirement: Scope options take several ids
+
+`--program-ids` and `--data-set-ids` SHALL accept comma-separated ids. Against an instance, the
+metadata dependency exports of every id SHALL be merged, and an object present in several of
+them SHALL appear once.
+
+#### Scenario: Two data sets sharing a data element
+
+- **WHEN** the command runs with `--data-set-ids=DS1,DS2` and both data sets contain data
+  element A
+- **THEN** A appears once in the `dataElements` sheet
+
+### Requirement: Data set scope applies to a metadata file
+
+With `--metadata-file`, `--data-set-ids=ID1,ID2` SHALL keep only the file objects belonging to
+those data sets: data elements listed in their `dataSetElements`, indicators listed in the data
+sets, sections whose `dataSet` is one of them, and options of the option sets used by those data
+elements. A data set id not found in the file, a requested model with no data set relation, or
+`--program-ids` with a file SHALL abort with an error.
+
+#### Scenario: Data element of another form is not exported
+
+- **WHEN** the file has a new data element that belongs only to a data set not in
+  `--data-set-ids`
+- **THEN** it is not in the sheet, even though it is new
+
+#### Scenario: Options follow their data element
+
+- **WHEN** a scoped data element uses an option set with new options
+- **THEN** those options are in the `options` sheet
+
+### Requirement: Match translations stored with Java legacy locale codes
+
+Translation columns SHALL match an object's translations by locale ignoring the Java legacy
+language code difference (`in`/`id`, `iw`/`he`, `ji`/`yi`).
+
+#### Scenario: Indonesian translation is shown
+
+- **WHEN** an object has a `FORM_NAME` translation with locale `in` and `Indonesian` (`id`) is a
+  requested locale
+- **THEN** the `formName: Indonesian` cell holds that translation

openspec/changes/export-translations-only-changed/tasks.md

@@ -0,0 +1,36 @@
+## 1. Domain layer
+
+-   [x] `Locale`: `normalizeLocaleCode` (Java legacy codes), `isSameLocale`, `haveSameLanguage`
+-   [x] `MetadataObject`: `getMetadataObjectField`, `getMetadataObjectTranslation` (shared by the
+        spreadsheet repository and the change detection)
+-   [x] `MetadataSourceRepository` interface (with the #106 scope options); `MetadataRepository`
+        extends it and adds `getByIdsWithTranslations(model, ids)`
+-   [x] `DataSetScope`: `buildDataSetScope`, `isInDataSetScope`
+-   [x] `ExportTranslationsUseCase`: optional `metadataSource`, options `onlyChanged`,
+        `defaultLocale`, `excludeNames`; pure `isChanged`
+
+## 2. Data layer
+
+-   [x] `MetadataJsonFileRepository` reading a DHIS2 metadata JSON export, `dataSetIds` scope by
+        membership
+-   [x] `MetadataD2Repository`: merge the dependency exports of several program/data set ids
+-   [x] `MetadataD2Repository.getByIdsWithTranslations` with chunked `id:in` filter
+-   [x] `ExportTranslationsSpreadsheetRepository` matches locales through `isSameLocale`
+
+## 3. Command wiring
+
+-   [x] `--metadata-file`, `--only-changed`, `--default-locale`, `--exclude-names` on
+        `translations to-spreadsheet`; reject `--only-changed` without `--metadata-file`
+-   [x] `--program-id`/`--data-set-id` → `--program-ids`/`--data-set-ids` (comma-separated)
+
+## 4. Testing
+
+-   [x] Use case: file source, exclude by name, only-changed selection, `isChanged` cases
+-   [x] `MetadataJsonFileRepository` (incl. data set scope), `DataSetScope` and `Locale` unit
+        tests; `in`/`id` match in the sheet
+
+## 5. Verification
+
+-   [x] `yarn typecheck`, `yarn lint`, `yarn test`
+-   [x] README: document the options with a delta-export example
+-   [x] Spec under `specs/export-translations-to-spreadsheet`

src/data/ExportTranslationsSpreadsheetRepository.ts

@@ -4,9 +4,12 @@ import XLSX from "xlsx-js-style";
 import { unzipSync, zipSync } from "fflate";
 import { Async } from "domain/entities/Async";
 import { Locale } from "domain/entities/Locale";
-import { MetadataObjectWithTranslations } from "domain/entities/MetadataObject";
+import {
+    getMetadataObjectField,
+    getMetadataObjectTranslation,
+    MetadataObjectWithTranslations,
+} from "domain/entities/MetadataObject";
 import { ModelTranslationsExport } from "domain/entities/ModelTranslationsExport";
-import { translationFieldToProperty } from "domain/entities/Translation";
 import {
     ExportTranslationsOptions,
     ExportTranslationsRepository,
@@ -176,7 +179,7 @@ export class ExportTranslationsSpreadsheetRepository implements ExportTranslatio
 
     private getRow(object: MetadataObjectWithTranslations, sheet: ModelTranslationsExport): string[] {
         const fieldCells = sheet.fields.flatMap(field => [
-            getFieldValue(object, field),
+            getMetadataObjectField(object, field),
             ...sheet.locales.map(locale => this.getTranslationValue(object, field, locale)),
         ]);
 
@@ -192,21 +195,10 @@ export class ExportTranslationsSpreadsheetRepository implements ExportTranslatio
         field: string,
         locale: Locale
     ): string {
-        const property = translationFieldToProperty(field);
-        const translation = object.translations.find(
-            t => t.property === property && t.locale === locale.locale
-        );
-        return translation?.value ?? "";
+        return getMetadataObjectTranslation(object, field, locale.locale) ?? "";
     }
 }
 
-function getFieldValue(object: MetadataObjectWithTranslations, field: string): string {
-    // Objects are fetched with `:owner`, so they carry all owner fields at runtime even
-    // though the type only declares id/name/code/translations.
-    const value = (object as unknown as Record<string, unknown>)[field];
-    return typeof value === "string" ? value : "";
-}
-
 export interface SheetData {
     name: string;
     header: string[];