Skip to content

[Feature] Add start/end-date, descendants and skip-metadata to export programs script - #108

Open
nshandra wants to merge 5 commits into
developmentfrom
feature/add-options-for-program-export
Open

nshandra wants to merge 5 commits into
developmentfrom
feature/add-options-for-program-export

Conversation

@nshandra

Copy link
Copy Markdown
Contributor

📌 References

Issue: [49] Migrate event data for SKERU

📝 Implementation

  • Update export programs script to allow the data to be filtered by the lastUpdated property using the --start-date and/or --end-date options.

  • Added the --descendants option, is its used, the included orgUnits will be the specified by --orgunits-ids and its descendants.

  • The metadata export can be skipped via the --skip-metadataoption.

  • Change program import to allow importing without metadata.

…and skip-metadata. Change url to allow for a optional auth parameter. Change program import to allow importing without metadata.
@nshandra
nshandra requested a review from adrianq August 12, 2026 07:50
@nshandra nshandra self-assigned this Aug 12, 2026
@nshandra nshandra changed the title [Feature] update export programs script with [Feature] Add start/end-date, descendants and skip-metadata to export programs script Aug 12, 2026
@nshandra
nshandra marked this pull request as ready for review August 12, 2026 08:36
@adrianq
adrianq requested a review from anagperal August 12, 2026 11:09

@anagperal anagperal left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[code-only review] thanks @nshandra !! In general it seems ok, just a few comments

Comment thread README.md Outdated
Comment thread src/scripts/commands/programs.ts
Comment thread src/data/ProgramsD2Repository.ts
Comment thread src/data/D2Tracker.ts

const endpoint = {
trackedEntities: () => tracker.trackedEntities.get(apiOptions),
enrollments: () => tracker.enrollments.get(apiOptions),

@anagperal anagperal Aug 13, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updatedBefore isn't supported on the enrollments endpoint, d2-api's trackerEnrollments.d.ts:59 has updatedAfter and updatedWithin only. Nothing catches this: assigning to a variable before passing disables TypeScript's excess-property check (apiOptions)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a post-query filter in D2Tracker for enrollments that emulates this function.

Comment thread src/data/D2Tracker.ts
Comment thread README.md Outdated

The data can be filtered by:
- Organization Unit via the `--orgunits-ids` option. If `--descendants` is used, the included orgUnits will be the specified and its descendants. Note that if no `--orgunits-ids` is provided the `ouMode` will be `ALL` and the `--descendants` option will be redundant.
- The `lastUpdated` property using the `--start-date` and/or `--end-date` options.

@anagperal anagperal Aug 13, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a comment: the date window is applied independently to three related entity types (events, enrollments and tracked entities), so a filtered export can contain dangling references.

For example, with --start-date=2026-01-01: an event edited on 2026-02-14 is exported, but its enrollment (last updated 2024-06-02) and its TEI (2024-05-10) are not, so the file references two records it doesn't contain. If this is intentional like this, then it's ok.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a warning about this in the README. The task didn't warranted tackling this a the data to extract were of event type only.

The fix would be to collect the missing references in the retrieved tracker type metadata and fetch them afterwards.

Comment thread src/scripts/commands/programs.ts Outdated
const startDateArg = option({
type: optional(string),
long: "start-date",
description: "Start date",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

improve description here and in end date

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I restored startDateArg / endDateArg and created new updatedStartDateArg / updatedEndDateArg with a more detailed description and a input validation function.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants