[Playground] OTLP-Mapper - instana to otlp span mapper by abhilash-sivan · Pull Request #2534 · instana/nodejs

abhilash-sivan · 2026-05-04T17:37:09Z

UI: https://instana.io/s/B2R0pfHbSpaV5jXmQNDd7w

This is just a playground where we convert Instana spans to Otel spans and send them to the Otel backend.

INSTANA_OTLP_FORMAT=true SERVICE_NAME=test-nodejs-service INSTANA_DEBUG=true node index.js

handle multiple keys in span.data
approve mapping design
approve port problem
approve agent announcement cycle disable use case
test: send traces via otlp port and metrics via standard port - Idenfies 2 diff apps 1. Otel and 2. Node.js. We have to add metrics mapping. Discussed/Agreed with community.
figure out: node.http.server - what is the official otel name/value so that the backend is labeling everything correctly in the UI? (prio 1)
Performance check classes vs functions

https://instana.io/s/Ehe2Zhb7QyKyxrmQg0fK8Q The BE uses path_tpl or url values ot show in the UI, so we mapped the otel span name to the same data from Instana span.

After conversion using the script:

Instana kafka trace to otlp

{
  "attributes": [
    {
      "key": "messaging.system",
      "value": {
        "stringValue": "kafka"
      }
    },
    {
      "key": "messaging.destination.name",
      "value": {
        "stringValue": "order-events"
      }
    },
    {
      "key": "messaging.operation.type",
      "value": {
        "stringValue": "send"
      }
    },
    {
      "key": "messaging.kafka.destination.partition",
      "value": {
        "intValue": 0
      }
    },
    {
      "key": "messaging.kafka.message.offset",
      "value": {
        "intValue": 12345
      }
    },
    {
      "key": "messaging.kafka.message.key",
      "value": {
        "stringValue": "order-123"
      }
    },
    {
      "key": "service",
      "value": {
        "stringValue": "order-service"
      }
    }
  ],
  "events": [],
  "links": [],
  "resource": {
    "attributes": [
      {
        "key": "service.name",
        "value": {
          "stringValue": "order-service"
        }
      },
      {
        "key": "telemetry.sdk.language",
        "value": {
          "stringValue": "nodejs"
        }
      },
      {
        "key": "telemetry.sdk.name",
        "value": {
          "stringValue": "@instana/collector"
        }
      },
      {
        "key": "telemetry.sdk.version",
        "value": {
          "stringValue": "3.0.0"
        }
      }
    ]
  },
  "traceId": "00000000000000000000xyz789abc123",
  "spanId": "000000000span002",
  "parentSpanId": "0000000parent002",
  "startTimeUnixNano": "1716019300000000000",
  "endTimeUnixNano": "1716019300050000000",
  "status": {
    "code": 1
  },
  "name": "send order-events"
}

abhilash-sivan · 2026-05-07T14:36:53Z

  spans = [];
  batchingBuckets.clear();

+  const processedSpans =


This is where the conversion is happening.

This seems to be the optimal place because if we convert early, then in some places we talk Instana format and it could raise unexpected issues. So before transmission seems simple and maintanable

Should we do this here or just after the current BE transformation,
https://github.com/instana/nodejs/pull/2534/changes?utm_source=chatgpt.com#diff-462ace70d7ac6eb4cd6f82756059c88629bfad1463fde0498d89106d7ccc8104L193

Right now we’re applying the transformation after collecting all spans. IMO it would be cleaner to do it before calling transmitSpans, so the transmit layer only deals with already-transformed spans.

TBD

Yeah, I agree. This would be a more cleaner way

// Transform internal span data format into external (backend) readable format.
span = applySpanTransformation(span);

Just after this transformation, we can call the Otel transformation.

One thing we have to consider is that currently all the spans intended for transmission — i.e., the array of spans — are converted together. If we do it in the above-mentioned area, we will have to process the spans individually.

Note: TBD — Transform each span to OTLP format immediately in addSpan and store them in OTLP format in the buffer if batching is enabled. This would require changes to the batching logic.

How do you want to organize the agent ports etc?

OTLP mode is on -> port will change.
There is something missing.

Also:
How can we skip the agent announcement cycle when connecting to an otel collector later?
Did you draw that path as well to confirm the architecture?

No, I haven’t worked much on the architecture yet. I can work on the dynamic port connections part, and I’ll also think through the agent announcement cycle and how we can skip it when connecting to an OTel collector

kirrg001 · 2026-05-12T09:05:34Z

test: send traces via otlp port and metrics via standard port - Idenfies 2 diff apps 1. Otel and 2. Node.js

This needs to be communicated in the next WG. Traces & Metrics mapping is required.

abhilash-sivan · 2026-05-20T09:03:51Z

+   * RabbitMQ-specific mappings
+   * Extends base messaging with RabbitMQ-specific fields
+   */
+  rabbitmq: {


Not verified rabbitmq per se. It's just to show how we differentiate children under messaging or any other group.

But for RabbitMQ, there’s nothing really common with BASE_MAPPINGS.messaging. I don’t think any of the others share much with it either, apart from Kafka.

kirrg001 · 2026-05-20T09:09:15Z

+      mappings: SPAN_ATTRIBUTE_MAPPINGS.kafka,
+      prefix: 'messaging.kafka',
+      additionalAttributes: {
+        'messaging.system': 'kafka'


Why do we need that here? If we have already a custom kafka mapping object?

messaging.system is an additional attribute that we append to the span. For Kafka, the system name also happens to be kafka, so it may seem redundant at first glance.

However, additionalAttributes is intended for constant or computed attributes that do not directly map from a single field inside span.data.kafka.

messaging.system falls into that category since it is effectively a constant defined per transformer, rather than something extracted from the incoming span payload itself.

I have changed this to: 'messaging.system': this.systemName

which will fetch the appropriate value in this case

aryamohanan · 2026-05-26T12:07:17Z

+      scopeSpans: [
+        {
+          scope: {
+            name: '@instana/collector',


nitpick: we might need to update the package name later. Since this transformer lives in core, it could also be used by the serverless package in the future. It would be better to dynamically fetch the package name instead of hardcoding it.

aryamohanan · 2026-05-26T12:11:48Z

+   *
+   * @see https://opentelemetry.io/docs/specs/semconv/database/
+   */
+  database: {


It should be databases since that is the naming pattern we already use. You can also see this in the IBM documentation and in the directory structure under core/src/tracing/instrumentation

aryamohanan · 2026-05-26T12:21:10Z

+
+'use strict';
+
+const { getOtlpAttributeMappings } = require('./otlp_mapper/mapper');


this file still needed?

yes, metrics are handled here

aryamohanan · 2026-05-27T09:43:17Z

+
+  attributes.push({
+    key: 'telemetry.sdk.version',
+    value: { stringValue: '3.0.0' }


nit pick: This value is currently hard coded, can we source this from package.json? or add todo comment for fixing this later

aryamohanan · 2026-05-27T09:44:48Z

+
+  // Service name from span data
+  const serviceName =
+    instanaSpan.data?.service || process.env.OTEL_SERVICE_NAME || process.env.SERVICE_NAME || 'unknown-service';


What is process.env.SERVICE_NAME? Did you mean process.env.INSTANA_SERVICE_NAME?
May be here, we can use config.serviceName as well.

instanaSpan.data?.service || process.env.OTEL_SERVICE_NAME || config.serviceName;

aryamohanan · 2026-05-27T10:41:34Z

+   * RabbitMQ-specific mappings
+   * Extends base messaging with RabbitMQ-specific fields
+   */
+  rabbitmq: {


But for RabbitMQ, there’s nothing really common with BASE_MAPPINGS.messaging. I don’t think any of the others share much with it either, apart from Kafka.

aryamohanan · 2026-05-27T10:46:23Z

+   * @see https://opentelemetry.io/docs/specs/semconv/general/attributes/#server-and-client-attributes
+   */
+  peer: {
+    hostname: { key: 'net.peer.name' },


net.peer.name and net.peer.port are depreacted.

net.peer.name => server.address
net.peer.port => server.port

https://opentelemetry.io/blog/2023/http-conventions-declared-stable/#http-client-span-attributes

aryamohanan · 2026-05-27T10:52:05Z

+/**
+ * Protocol configuration for transformer instantiation and semantic system names
+ */
+const PROTOCOL_CONFIG = {


PROTOCOL_CONFIG feels slightly misleading because this object is not really defining protocols, it’s mapping Instana span types to OTEL semantic convention system names.

We can reduce duplication in PROTOCOL_CONFIG by treating systemName as defaulting to spanType.

Maybe simplify to something like:

const OTEL_SYSTEM_NAMES = { http: 'http', kafka: 'kafka', rabbitmq: 'rabbitmq', mongo: 'mongodb' };

Then:

function getProtocolConfig(spanType) { return { spanType: spanType, systemName: OTEL_SYSTEM_NAMES[spanType] || spanType }; }

aryamohanan · 2026-05-27T11:12:03Z

+ *
+ * When a span contains multiple data keys (e.g., both 'mongo' and 'peer'),
+ * this array determines which transformer to use. Primary span types are
+ * checked first, auxiliary data types last.


Why this required?

aryamohanan · 2026-05-27T11:16:02Z

+ * @returns {string} OTLP trace ID (32-character hex)
+ */
+function convertTraceId(instanaTraceId) {
+  if (!instanaTraceId) return '00000000000000000000000000000000';


IMO this may not be a valid case, but if instanaTraceId is falsy we silently convert it to an all-zero trace ID. How is this expected to be handled? Do we have any known scenarios where this can happen? 🤔

kirrg001 · 2026-05-29T09:36:58Z

+  http: HttpTransformer,
+
+  // Messaging protocols
+  kafka: KafkaTransformer,


This is wrong IMO

Either we instantiate all of them here or none.
So we have to go with "all".

rabbitmq: span =>
new MessagingTransformer

kirrg001 added 2 commits April 9, 2026 12:21

chore: playground done

7a5da5a

chore: infra cor

a046abc

abhilash-sivan force-pushed the chore-otel-span-mapper branch from 4490ab0 to f1c0d6f Compare May 4, 2026 18:07

abhilash-sivan changed the base branch from main to chore-otlp-playground May 7, 2026 06:42

abhilash-sivan added 3 commits May 7, 2026 18:08

chore: phase 1

378f117

chore: phase 2

c1ef77d

chore: phase 3

d261bc5

abhilash-sivan force-pushed the chore-otel-span-mapper branch from 53e1fcc to d261bc5 Compare May 7, 2026 12:45

abhilash-sivan added 5 commits May 7, 2026 18:20

chore: cleanup

17710d0

chore: rewrite

0a6ec17

chore: test

8a89eef

chore: a

c78331c

chore: new fields

bcb034e

abhilash-sivan commented May 7, 2026

View reviewed changes

chore: test

980a1c0

abhilash-sivan changed the title ~~Chore otel span mapper~~ chore: instana-otlp span mapper May 7, 2026

abhilash-sivan changed the title ~~chore: instana-otlp span mapper~~ [Playground] OTLP-Mapper - instana to otlp span mapper May 8, 2026

abhilash-sivan added 2 commits May 10, 2026 17:17

test: fixed failing test

5aebc15

chore: cleanup

8c6cc88

kirrg001 reviewed May 11, 2026

View reviewed changes

Comment thread packages/collector/src/agentConnection.js

kirrg001 reviewed May 11, 2026

View reviewed changes

Comment thread packages/core/src/tracing/backend_mappers/index.js Outdated

kirrg001 reviewed May 12, 2026

View reviewed changes

Comment thread packages/core/src/tracing/otlp_mapper/mapper.js Outdated

abhilash-sivan added 4 commits May 13, 2026 15:39

chore: ui corrected

c3179eb

chore: added kafka and pg, generalize

21c211d

chore: update

fcd7622

chore: fix

c1253b0

abhilash-sivan force-pushed the chore-otel-span-mapper branch from 08a2757 to e827e9c Compare May 15, 2026 05:57

chore: clean

d6c01a6

abhilash-sivan force-pushed the chore-otel-span-mapper branch from e827e9c to d6c01a6 Compare May 15, 2026 05:58

kirrg001 reviewed May 18, 2026

View reviewed changes

Comment thread scripts/instana-to-otel-converter.js Outdated

kirrg001 reviewed May 18, 2026

View reviewed changes

Comment thread scripts/instana-to-otel-converter.js Outdated

abhilash-sivan added 8 commits May 19, 2026 12:03

chore: update logic

52be4b2

chore: spanBuffer uses temporary converter

b8a95a1

chore: move files for using inside spanBuffer

402b4d3

chore: update

3df77e8

chore: rearrange

e8b37d9

chore: added separate transformer

059c844

chore: added example otel spans

c197345

chore: make getSpanType dynamic

0935eda

abhilash-sivan commented May 20, 2026

View reviewed changes

kirrg001 reviewed May 20, 2026

View reviewed changes

Comment thread packages/core/src/tracing/converters/transformers.js Outdated

kirrg001 reviewed May 20, 2026

View reviewed changes

Comment thread packages/core/src/tracing/converters/instana-to-otel-converter-utils.js Outdated

abhilash-sivan added 5 commits May 20, 2026 15:06

chore: fix status

d805750

chore: kafka systemName

0a099fd

chore: v1 of multiple data key logic

3d4ae91

chore: refactor

90954b2

chore: optimize additioanl attributes

d5575d7

kirrg001 reviewed May 22, 2026

View reviewed changes

Comment thread packages/core/src/tracing/converters/instana-to-otel-converter.js Outdated

kirrg001 reviewed May 22, 2026

View reviewed changes

Comment thread packages/core/src/tracing/converters/instana-to-otel-converter.js

kirrg001 reviewed May 22, 2026

View reviewed changes

Comment thread packages/core/src/tracing/converters/transformers.js Outdated

abhilash-sivan added 6 commits May 22, 2026 16:45

chore: wip more dynamic

4bdb790

chore: refatcor transformers

13eccd6

chore: refactor main file

891b4d8

chore: new otel span type file

38b41de

chore: cleanup otlp transformer

94922bd

chore: minor refactor for pr comment

61a1afd

aryamohanan reviewed May 27, 2026

View reviewed changes

kirrg001 reviewed May 29, 2026

View reviewed changes


		'use strict';

		const { getOtlpAttributeMappings } = require('./otlp_mapper/mapper');

Conversation

abhilash-sivan commented May 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kirrg001 commented May 12, 2026

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

abhilash-sivan commented May 4, 2026 •

edited

Loading