Research: Support non-imap Data into IMAP SDC ecosystem

[tech3371]

Investigating effort to support ingestion of non-imap data into IMAP SDC ecosystem and view data through IMAP SDC API or website.

Steps:

• Follow IMAP science filename convention FILENAME_CONVENTION = ( "<mission>_<instrument>_<datalevel>_<descriptor>_" "<startdate>(-<repointing>)_<version>.<extension>" ). The filename convention contains information such as mission, instrument name or data level or descriptor (product name) or data date or version and file type. These information are used by SDC ecosystem to validate, ingest, db indexing, support queries through all APIs. Due to that, to fit in IMAP SDC ecosystem, following IMAP science filename convention is important.
• Update ScienceFilePath to support various missions. Eg. update these functions validate_filename(), extract_filename_components() and update _dir_prefix to support various mission.
• if new instrument name, add to imap_data_access.VALID_INSTRUMENTS list
• if new data level, add to imap_data_access.VALID_DATALEVELS

This is first step of supporting non-IMAP data. After these updates are done in this library,

• make new release
• Do integration testing in sds-data-manager repo and fix additional unknown bugs
• push to production

[SavvasRaptis]

I attempted to make the files we got friendly the IMAP science filename convention. In practice we have two different CDF files, with common and native resolutions from each mission see examples attached. Do these look reasonable to you? is there anything missing?

imap_l1_examplesfiles.zip

The other mission data are embedded in the CDF files, but not through the instrument and mission tags of the cdf name, but rather through the variable names within the cdf file.