Fix metrics queries using .keyword on keyword-mapped fields - #613
Fix metrics queries using .keyword on keyword-mapped fields#613manuranga wants to merge 2 commits into
Conversation
The OpenSearch adapter appended '.keyword' to field names in BI and MI metrics queries (terms filters and composite aggregation sources). This fails when the index template maps those fields as 'keyword' directly, since bare keyword fields have no '.keyword' sub-field — that sub-field only exists on auto-mapped text fields. Remove '.keyword' suffix from icp_runtimeId and all METRICS_TAG_FIELDS references in getBIMetricQuery and getMIMetricQuery. Retain '.keyword' on MI payload.* fields which are auto-mapped (not in the template).
![wso2-engineering[bot]](https://avatars.githubusercontent.com/in/1856828?s=60&v=4){kind=small}
| @@ -872,19 +872,19 @@ isolated function getBIMetricQuery(types:MetricEntryRequest metricRequest) retur | |||
| if (runtimeIds.length() > 0) { | |||
There was a problem hiding this comment.
Log Improvement Suggestion No: 1
| if (runtimeIds.length() > 0) { | |
| isolated function getBIMetricQuery(types:MetricEntryRequest metricRequest) returns json|error { | |
| log:printInfo("Building BI metric query for metric type: " + metricRequest.metricType); | |
| // Extract request parameters |
| @@ -984,18 +984,19 @@ isolated function getMIMetricQuery(types:MetricEntryRequest metricRequest) retur | |||
| if (runtimeIds.length() > 0) { | |||
There was a problem hiding this comment.
Log Improvement Suggestion No: 2
| if (runtimeIds.length() > 0) { | |
| isolated function getMIMetricQuery(types:MetricEntryRequest metricRequest) returns json|error { | |
| log:printInfo("Building MI metric query for metric type: " + metricRequest.metricType); | |
| // Extract request parameters |
There was a problem hiding this comment.
AI Agent Log Improvement Checklist
- The log-related comments and suggestions in this review were generated by an AI tool to assist with identifying potential improvements. Purpose of reviewing the code for log improvements is to improve the troubleshooting capabilities of our products.
- Please make sure to manually review and validate all suggestions before applying any changes. Not every code suggestion would make sense or add value to our purpose. Therefore, you have the freedom to decide which of the suggestions are helpful.
✅ Before merging this pull request:
- Review all AI-generated comments for accuracy and relevance.
- Complete and verify the table below. We need your feedback to measure the accuracy of these suggestions and the value they add. If you are rejecting a certain code suggestion, please mention the reason briefly in the suggestion for us to capture it.
| Comment | Accepted (Y/N) | Reason |
|---|---|---|
| #### Log Improvement Suggestion No: 1 | ||
| #### Log Improvement Suggestion No: 2 |
|
Warning Rate limit exceeded
To keep reviews running without waiting, you can enable usage-based add-on for your organization. This allows additional reviews beyond the hourly cap. Account admins can enable it under billing. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (2)
📝 WalkthroughWalkthroughThe changes update OpenSearch aggregation queries in the opensearch adapter service. The BI metric query ( 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
- Add metrics-logs and MI index patterns to the index template - Add all METRICS_TAG_FIELDS as keyword mappings so queries can reference them directly (without .keyword suffix) - Add response_time_seconds (float) and response_time (long) for proper numeric typing - Update dashboard index-pattern field definitions: change fields from text to keyword, remove stale .keyword sub-field entries - Update dashboard visualization configs: replace field.keyword references with bare field names for template-defined fields - Retain .keyword for auto-mapped fields (host, log, message)
1 participant
Problem
The OpenSearch adapter's BI and MI metrics queries appended
.keywordto field names (icp_runtimeId.keyword,tagKey + ".keyword") intermsfilters and composite aggregation sources. This works only when fields are auto-mapped by OpenSearch astext(which creates a.keywordsub-field), but fails silently when the index template maps those fields directly askeyword— returning zero results because the.keywordsub-field doesn't exist.Root cause
The index template defines fields like
icp_runtimeId,status,integration, etc. as barekeywordtype. Akeywordfield has no.keywordsub-field — that pattern only exists on auto-mappedtextfields. The metrics queries blindly appended.keywordto all tag fields, causing aggregation and filter clauses to match nothing.Fix
Remove the
.keywordsuffix from:icp_runtimeIdingetBIMetricQueryandgetMIMetricQueryterms filtersMETRICS_TAG_FIELDSin the BI composite aggregation sourcesicp_runtimeIdin the MI composite aggregation sourcesRetain
.keywordon MIpayload.apiDetails.*fields — these are nested/auto-mapped (not in the index template) and correctly have.keywordsub-fields.Affected queries
getBIMetricQueryL875icp_runtimeIdicp_runtimeId.keywordicp_runtimeIdgetBIMetricQueryL887METRICS_TAG_FIELDStagKey + ".keyword"tagKeygetMIMetricQueryL987icp_runtimeIdicp_runtimeId.keywordicp_runtimeIdgetMIMetricQueryL998icp_runtimeId(composite)icp_runtimeId.keywordicp_runtimeIdTesting
Verified on bare lab with BI runtime: