Skip to content

feat(harvester): harvest and handle correction materials - #1042

Merged
zzacharo merged 1 commit into
CERNDocumentServer:masterfrom
liptakpanna:harvester-erratum
Nov 9, 2026
Merged

zzacharo merged 1 commit into
CERNDocumentServer:masterfrom
liptakpanna:harvester-erratum

Conversation

@liptakpanna

Copy link
Copy Markdown
Contributor

closes #1006

Open questions:

  • resource type of correction materials
  • what if only the filename says it's erratum.pdf etc?

"files": {
k: v
for k, v in metadata.get("files", {}).items()
if k != "default_preview"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why we exclude the file with the default preview from the update method?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a comment to explain: at this point the files are not uploaded yet, so setting the default_preview to a file that is not yet available raises an error. the key is ignored at this point, but used later in the FileSynchronizer

Comment on lines +15 to +16
"corrigendum": {"resource_type": "publication-other"},
"addendum": {"resource_type": "publication-other"},

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these 2 are not handled yet in the migrator-kit. Are we sure they come from INSPIRE? If so, maybe we can already add them to the list we have in the migrator-kit.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They are on INSPIRE https://github.com/inspirehep/inspire-schemas/blob/23c49f243fd9fe1e10557a5d92f22965d88b1246/inspire_schemas/records/elements/material.yml#L4

For the migrator-kit: addendum is mentioned here, but not in related identifiers.

Also, the decision log only has erratum, but the issue mentioned all three, so I'm not sure

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you confirm with the library and ammend the decision log and migrator kit if agreed please?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

return True
if not (source or material):
return False
if (not material or material == "publication") and source != "arxiv":

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this can be merged to your new correction_material branch, no?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added named conditions to make all the accepted cases clearer

@liptakpanna
liptakpanna force-pushed the harvester-erratum branch 2 times, most recently from 6bb0cd8 to fc55e11 Compare November 5, 2026 13:28
@zzacharo

zzacharo commented Nov 5, 2026

Copy link
Copy Markdown
Contributor

@liptakpanna also, can you rebase please?

@liptakpanna
liptakpanna force-pushed the harvester-erratum branch 3 times, most recently from ca864da to 40d0e51 Compare November 6, 2026 13:51
@zzacharo
zzacharo merged commit 6e90129 into CERNDocumentServer:master Nov 9, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

harvester: add erratums to record

2 participants