[BUG] Doc-level monitor upsertQueryIndex fails with HTTP 500 when source index contains alias-type fields

[thecodingshrimp]

[BUG] Doc-level monitor upsertQueryIndex fails with HTTP 500 when source index contains alias-type fields

Describe the issue

When a doc-level monitor is created (or updated) and one or more of its source indices contains fields with "type": "alias", the alerting plugin's DocLevelMonitorQueries.upsertQueryIndex method fails with an HTTP 500 error. The monitor is never created and no useful error is surfaced at production log levels.

The failure occurs in leafNodeProcessor, which traverses all field mappings from all backing indices and copies them into the percolate query index (.opensearch-alerting-queries-*). When it encounters an alias field it rewrites the alias path value by appending _<indexName>_<monitorId>. This synthesized path does not exist as a concrete field in the query index. OpenSearch rejects the subsequent PUT _mapping call because an alias must refer to an existing field in the same index's mappings.

The plugin is not idempotent with respect to its own prior output: alias fields written to backing indices during a previous monitor lifecycle are re-discovered on the next upsertQueryIndex call and trigger the same failure.

Expected behavior

upsertQueryIndex completes successfully regardless of whether source indices contain type: alias fields. Alias fields are silently skipped during mapping traversal because they are index-local contracts and cannot be transplanted to an index with a different schema. The percolate query index holds stored query_string queries, not documents; alias resolution is never used when matching percolate queries, so alias fields in the query index serve no semantic purpose.

Actual behavior

PUT _mapping on the query index fails with:

Invalid [path] value [<originalPath>_<backingIndexName>_<monitorId>] for field alias [...]:
an alias must refer to an existing field in the mappings.
[type=security_analytics_exception]

The initial failure is logged at log.debug, making it invisible at the default production log level. A deleteQueryIndexInEveryRun=true retry path is attempted but produces the same error because the same source mappings generate the same invalid synthesized path.

Steps to reproduce

1. Create an index (or use a data stream) that contains at least one field mapping with "type": "alias". Example mapping fragment:

   PUT myIndex/_mapping
   {
     "properties": {
       "myAlias": {
         "type": "alias",
         "path": "myConcreteField"
       },
       "myConcreteField": {
         "type": "keyword"
       }
     }
   }

2. Create a doc-level monitor that references myIndex as its source index.

3. Delete the monitor.

4. Re-create the doc-level monitor referencing the same index.

5. The second creation triggers upsertQueryIndex, which re-traverses the backing index. The alias field myAlias is present from step 1. The plugin produces a synthesized path myConcreteField_myIndex_<monitorId> and submits it in a PUT _mapping call that OpenSearch rejects with HTTP 500.

The bug also triggers on the first monitor creation if alias fields pre-exist in the source index for any reason.

Root cause analysis

File: alerting/src/main/kotlin/org/opensearch/alerting/util/DocLevelMonitorQueries.kt

Primary defect — leafNodeProcessor:
The function processes every leaf field in the source index mappings. When it encounters a field with "type": "alias", it falls through to the general rewrite branch, which appends _<indexName>_<monitorId> to the alias path value. The resulting path is synthetic and does not exist in the query index, causing PUT _mapping to be rejected.

Both the conflicting-name branch and the non-conflicting branch of leafNodeProcessor rewrite the alias path to a non-existent value. Neither branch produces a valid outcome for alias fields.

Secondary defect — in-place mutation:
traverseMappingsAndUpdate mutates MappingMetadata.sourceAsMap in-place. Because MappingMetadata.sourceAsMap() is documented to return a reference to the underlying map, concurrent monitor operations on the same index share and corrupt this state.

Fix (primary — two-line guard):
Add a guard at the entry of leafNodeProcessor that returns early for any field where props["type"] == "alias". The returned triple uses an empty mutableMapOf() so the field is excluded from updatedProperties and never submitted to PUT _mapping. A matching nil-check must be added in the properties.forEach accumulation loop that calls leafNodeProcessor:

// In leafNodeProcessor, before val newProps = props.toMutableMap():
if (props["type"] == "alias") {
    return Triple(fieldName, fieldName, mutableMapOf())
}

The accumulation loop should skip entries where the returned properties map is empty (i.e., do not write an empty property entry to updatedProperties).

Affected versions

Related components

• opensearch-project/alerting — primary fix location (DocLevelMonitorQueries.kt)
• opensearch-project/security-analytics — secondary issue: the SA plugin writes type: alias fields to backing indices when a detector is created and does not clean them up on deletion. This permanent residue is the most common trigger for the alerting bug. Tracked separately in opensearch-project/security-analytics.