All notable changes to this project will be documented in this file.
The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.
- issues/119: As part of the operationalization of a large number of collections, added a script and instructions to partially automate dataset config creation.
- issues/164: Add
granulesto payload output for cumulus db status reporting
- issues/108: Handle case when no data is returned from a Harmony job by throwing a warning that can be tracked in CloudWatch logs.
- issues/148: Refactored message passing system after browse image generation to handle large tiled outputs (100s of output files).
- issues/148: Removed "Generate Image Metadata", "Build Image Sets", "Process Harmony Results", and "Save CNM Message" lambdas in favor of a consolidated "Process BIG Result" lambda that generates and save CNM messages for the entire result of a browse image generation workflow.
- issues/109: Handle new format of EDL token expiration datetime, also backwards compatible with old date format.
- issues/138: Fix key error when building image sets from OPERA HLS imagery.
- issues/127: Additional hotfix to allow for deployment in PODAAC environments with their version of cumulus
- features/old_null Updated null provider version to > 2.1 for PODAAC cumulus compatibility
- issues/97: Added the "bignbit" label to all
harmony-pyrequests.
- issues/127: Merged 0.4.0 changes into develop, updated harmony-py version, fixed conflict in terraform lock file.
- issues/69: Added support for other projections, with the default being EPSG:4326. The projection is read from the
outputCrskeyword in the dataset config. This likely needs refinement to account for cases when we have multiple projections per dataset. - issues/116: As part of the new projection support, it was necessary to add in a
scaleExtentparameter for polar projected browse images. The dataset configuration file now supports ascaleExtentPolarkeyword which is a list of four numbers specifying geographic bounds on the EPSG:3413 or EPSG:3031 outputs. This keyword only works if one of those projections is listed in theoutputCrskeyword. IfscaleExtentPolaris not specified, bounds are set to a default value used by GIBS ([-4194303, -4194303, 4194303, 4194303], technically this is 1m less than the bound, but this was done intentionally to avoid precision errors).
- issues/62: The retry policy for querying Harmony job status is now configurable as module parameters. By default, it will query for job status every 20 seconds for a maximum of 15 attempts.
- issues/91: Fixed bug where the CNM collection name sent to GIBS included '/' characters when dealing with certain variables. This caused processing errors in GIBS, all collection names will now replace '/' with '_' before being sent to GIBS.
- issues/89: Fixed bug where querying CMR for a collection could result in multiple results because collection version was not included in the query. Fix is to include the version in the CMR query, which will now return only one result.
- issues/96: Fixed bug causing GIBS responses to fail processing due to provider name containing an underscore
_which collided with the delimiter used in CNM identifiers. The new delimiter for CNM identifier is now an exclamation mark!. - issues/113: Height and width are now optional in the dataset configuration file. If not provided, the height and width will not be passed to harmony and will be determined automatically.
- issues/59: A new pair of keywords (
dataDayStrategyandsingleDayNumber) have been added to the DatasetConfiguration for BIG to enable proper image metadata for annual products. These keywords allow a dataset to override the umm-g date info. - issues/84: New parameter in dataset config
subdailythat sends DataDateTime to GIBS instead of DataDay. - Added optional
concept_idkeyword to dataset config to provide an override for finding the proper CMR collection concept ID when testing.
- issues/82: Fixed date parsing bug where ISO-8601 format dates, the default for UMM-G, were not handled properly.
- Update gibs_response_queue visibility timeout to match aws_lambda_function handle_gitc_response timeout
- issues/71: New module parameter
cmr_environmentis used to determine which environment to use for CMR requests when processing GIBS responses.
- issues/71: Increased timeout of the handle_gitc_response lambda function from 15 seconds to 45 seconds.
- issues/68: Fixed bug that was causing excessive size of output state object from the TransferImageSet map step.
- issues/60: Fixed bug causing GIBS responses to fail processing in OPS due to a case-sensitive comparison of environment name.
- issues/65: Fixed bug when input CMA message does not contain
cmrConceptIdby parsing the concept ID from thecmrLinkinstead.
- issues/55: Harmony client changed from per request and instead will be cached as global variable and will not validate auth credentials on initialization.
- issues/54: Fixed bug where status was not being reported to Cumulus Dashboard by adding
cumulus_metaback into the output CMA.
- Lowered required version of hashicorp/null to ~> 2.1 to be compatible with the requirements of cumulus core
- issues/50: Fixed bug where
countis unsupported for bignbit by inheriting the AWS provider config from the root module
- issues/40: New message attribute
response_topic_arnwill be added to every message sent to GIBS - issues/9: Added some documentation for installing as cumulus module
- issues/15: Change 'convertToPNG' choice to a generic send to harmony choice
- issues/16: Change apply opera treatment choice and lambda to be specific to HLS
- issues/23: Harmony requests now include
destinationUrlparameter to place output directly in s3 bucket instead of requiring data to be copied. - issues/41: Module no longer depends on deprecated hashicorp/template provider
- issues/42: Terraform version upgraded to v1.5.3
- Default values for
config_dirandbignbit_audit_pathhave changed tobig-configandbignbit-cnm-outputrespectively
- issues/7: Remove the wait for GITC response
- issues/23: Removed
lambda_rolemodule variable. The lambda role is now created as part of the module,permissions_boundary_arnis required instead.
- issues/36: Support datetimes without microseconds
- BIG terraform failing in SWOT venues due to long function(lambda) names