Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions en/docs/develop/copilot/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ Type `/` in the Copilot input bar to invoke a command for a specific task.
| `/doc` | Generate documentation for your integration. |
| `/openapi` | Import OpenAPI specifications. |
| `/typecreator` | Create custom types. |
| `/datamap` | [Generate data mappings](../integration-artifacts/supporting/data-mapper/data-mapper.md). |
| `/datamap` | [Generate data mappings](../integration-artifacts/supporting/data-mapper/ai-mapping.md). |
| `/natural-programming` | Experimental. Generate code from requirements and check for drift. |

:::note
Expand All @@ -89,6 +89,6 @@ Type `/` in the Copilot input bar to invoke a command for a specific task.

- [Getting started](getting-started.md) — Sign in to WSO2 Integrator Copilot.
- [Generate tests with AI](../test/ai-generated-cases.md) — Use Copilot to generate test cases.
- [AI data mapper](../integration-artifacts/supporting/data-mapper/data-mapper.md) — Generate data mappings using AI.
- [AI data mapper](../integration-artifacts/supporting/data-mapper/ai-mapping.md) — Generate data mappings using AI.
- [Try-It tool](../test/built-in-try-it-tool.md) — Test services without leaving the IDE.
- [AI usage and data handling guidelines](../../reference/ai-usage-and-data-handling-guidelines.md) — How Copilot handles your data.
Original file line number Diff line number Diff line change
@@ -0,0 +1,164 @@
---
sidebar_position: 9
title: AI Data Mapping
description: Generate transformation code between data structures with AI-powered field mapping.
keywords: [wso2 integrator, data mapper, ai data mapping, auto map, field mapping, transformation]
---
Comment thread
coderabbitai[bot] marked this conversation as resolved.

import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';

# AI Data Mapping

The AI Data Mapper uses AI to generate mappings between data structures, without manual field-by-field matching. It is especially useful for large or complex schemas with hundreds of fields, deeply nested records, or domain-specific formats.

For the Data Mapper editor and manual mapping, see [Data Mapper editor](../../../understand-ide/editors/datamapper-editor.md).

## How to use


<Tabs>
<TabItem value="ui" label="Visual designer" default>

1. Define the input and output record types in the **Types** panel. Define inner records first, then compose them into the parent input and output records. For details on creating records, see [Type editor](../../../understand-ide/editors/type-editor.md).

![Types canvas with child record cards visible, and the New Type panel open for defining a parent record that references them](/img/develop/integration-artifacts/supporting/data-mapper/ai-data-mapper-define-personal-profile.png)

2. Create the data mapper. Under **Data Mappers** in the project explorer, select **+**. Set a name, add the input type with a parameter name, set the output type, and select **Create**.

![Create New Data Mapper form with fields for data mapper name, input type with parameter name, and output type](/img/develop/integration-artifacts/supporting/data-mapper/ai-data-mapper-create-form.png)

3. The Data Mapper editor opens with the input schema on the left and the output schema on the right.

![Data Mapper canvas with the input schema on the left and the output schema on the right with no mappings](/img/develop/integration-artifacts/supporting/data-mapper/ai-data-mapper-canvas.png)

4. In the top-right corner of the canvas, select **Auto Map**. The WSO2 Integrator Copilot panel opens alongside the canvas with the `/datamap` command preloaded.

![WSO2 Integrator Copilot panel open alongside the Data Mapper canvas with the /datamap command preloaded in the input field](/img/develop/integration-artifacts/supporting/data-mapper/ai-data-mapper-copilot-command.png)

5. Submit the command. The Copilot reads the project files, generates field mappings based on the input and output types, and integrates them into your workspace. When complete, mapping lines appear between the matched fields on the canvas.

![Data Mapper canvas with generated field mapping lines connecting the input and output fields, with the completion message in the Copilot panel](/img/develop/integration-artifacts/supporting/data-mapper/ai-data-mapper-result.png)

</TabItem>
<TabItem value="code" label="Ballerina code">

<h2>Define data types</h2>

Define the input and output record types in `types.bal`:

```ballerina
type Student record {
int id;
string studentName;
int age;
string gender;
string[] semesterGPA;
string academicMajor;
Student[] roommates;
string address;
};

type PersonalProfile record {
int id;
Bio bio;
AcademicRecord academicRecord;
Accommodation accommodationDetails;
};

type Bio record {
string name;
string gender;
int age;
};

type AcademicRecord record {
string major;
string[] semesterGPA;
};

type Accommodation record {
int numberOfRoommates;
string address;
};
Comment thread
VellummyilumVinoth marked this conversation as resolved.
```

<h2>Define the data mapper function</h2>

Define the `transform` function stub in `data_mappings.bal`:

```ballerina
function transform(Student student) returns PersonalProfile => {};
```

Select **Visualize** above the `transform` function to open it in the Data Mapper editor. In the editor, select **Auto Map** to generate the field mappings. The function is updated with the generated implementation:

```ballerina
function transform(Student student) returns PersonalProfile => {
id: student.id,
bio: {name: student.studentName, gender: student.gender, age: student.age},
academicRecord: {major: student.academicMajor, semesterGPA: student.semesterGPA},
accommodationDetails: {numberOfRoommates: student.roommates.length(), address: student.address}
};
```

</TabItem>
</Tabs>

## Features

### Automated mapping generation

Mapping generation takes into account:

- Field names and naming conventions
- Semantic relationships between fields
- Nested data structures
- Array types and cardinality
- Optional and nullable fields
- Domain-specific patterns such as those common in healthcare contexts

### Advanced expression generation

The AI Data Mapper handles complex transformation scenarios:

- **Parsing and conversion**: Parsing such as parsing a string as an integer, and converting a value of one type to another.
- **Optional field handling**: Handling fields that may or may not be present.
- **Nested record transformation**: Deep structure mapping with proper path navigation.
- **Array-to-array mappings**: Member-wise transformations with appropriate iteration logic.

### Supporting documentation to improve accuracy

Upload reference materials to improve mapping accuracy. While schema-only analysis is supported, providing additional documentation helps the system understand field relationships and business rules.

Supported formats:

- PDF documents
- Images (JPEG, JPG, PNG)
- CSV files
- Text files

For complex mapping scenarios involving large schemas or domain-specific requirements captured in multiple documents, upload all of them. The system analyzes all the documentation to generate accurate, context-aware transformations.

### Sub-mapping reuse

The AI Data Mapper detects existing data mappers in your codebase and reuses them where applicable. This reduces code duplication, ensures consistent transformation logic, and keeps the codebase compact.

### Function extraction for large schemas

For mappings with a large number of fields (hundreds to thousands of fields), Copilot extracts helper functions to maintain code readability and comply with language server constraints. Complex transformations involving union types, deeply nested structures, and array-to-array operations are automatically decomposed into reusable functions.

## Responsible use

Large language models can produce unexpected results when processing highly domain-specific or atypical schema patterns. Follow these practices to ensure accuracy:

- Review all generated mappings before deploying to production.
- Test with representative data samples that reflect actual use cases.
- Verify that the generated transformation logic aligns with your business requirements.
- Provide feedback on incorrect or incomplete mappings to support continuous improvement.

## What's next

- [Data Mapper editor](../../../understand-ide/editors/datamapper-editor.md) — Open, configure, and work with the visual mapping canvas.
- [Data mapper](./data-mapper.md) — End-to-end guide to creating and using data mappers.
- [Expression editor](../../../understand-ide/editors/expression-editor.md) — Write custom expressions for individual field mappings.
Original file line number Diff line number Diff line change
Expand Up @@ -47,16 +47,13 @@ When you need a focused, visual mapping rather than a free-form function, use **

## Map with AI

Inside any data mapper view, select **Auto Map** in the top-right corner of the canvas to generate mappings automatically using the WSO2 Integrator Copilot.
Select **Auto Map** in the top-right corner of the data mapper canvas. The WSO2 Integrator Copilot panel opens with the `/datamap` command preloaded. Submit it to generate mappings between your types automatically.

![Data mapper toolbar with the Auto Map button in the top-right corner](/img/develop/integration-artifacts/supporting/data-mapper/ai-map-option.png)

The Copilot panel opens with a `/datamap` command preloaded. Submit it to generate field mappings based on the input and output types, then review and adjust the result on the canvas.

![Copilot panel with the /datamap command ready to generate mappings](/img/develop/integration-artifacts/supporting/data-mapper/ai-map-view.png)
For a full walkthrough with examples, see [AI data mapping](./ai-mapping.md).

## What's next

- [Array mappings](./array-mappings/array-mappings.md) — Map between arrays using iteration, joins, and aggregation.
- [Generic type mappings](./generic-type-mappings.md) — Generate types from a sample JSON or XML payload.
- [Sub Mappings](./submappings.md) — Reuse mapping logic across multiple output fields.
- [AI data mapping](./ai-mapping.md) — Generate mapping logic using AI.
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,9 @@ The mapping area is the central region between the input and output sides. Links

## Auto Map

**Auto Map** in the header attempts to wire every output field from the available inputs in one action. It matches fields by name and compatible type, and falls back to an AI-assisted suggestion when no exact match is found. Review the generated mappings on the canvas before saving, because **Auto Map** can produce best-effort suggestions that may need adjustment.
**Auto Map** in the header opens the WSO2 Integrator Copilot panel alongside the canvas with the `/datamap` command preloaded. Submit the command to let the Copilot read the project files and generate field mappings based on the input and output types. When complete, mapping lines appear on the canvas.

For more, see [AI data mapping](/docs/develop/integration-artifacts/supporting/data-mapper/ai-mapping).

{/* ![Auto Map suggestions on the mapping canvas](/img/develop/understand-ide/editors/datamapper-editor/auto-map.png) */}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ Calls a function defined in the project, an imported library, or the Ballerina s

![Call Function button in the Statement section](/img/develop/flow-design-elements/call-function-node.png)

The function picker lists functions in three sections: `Within Project`, `Imported Functions` (functions from imported libraries such as `log`), and `Standard Library`. Select **Create Function** under `Within Project` to define a new function inline.
The function picker lists functions in three sections: **Within Project**, **Imported Functions** (functions from imported libraries such as `log`), and **Standard Library**. Select **Create Function** under **Within Project** to define a new function inline.

![Function picker showing Within Project, Imported Functions, and Standard Library entries](/img/develop/flow-design-elements/call-function-options.png)

Expand All @@ -53,7 +53,7 @@ Adds a data mapper call that transforms data from one record shape to another. U

![Map Data button in the Statement section](/img/develop/flow-design-elements/map-data-node.png)

The picker lists every data mapper in the project under `Within Project`. Select an existing mapper to invoke it from the flow, or select **Create Data Mapper** to define a new one. New and existing mappers open in the [Data Mapper editor](../datamapper-editor.md), where you draw connections between source and target fields, write inline expressions, or use **Auto Map**.
The picker lists every data mapper in the project under **Within Project**. Select an existing mapper to invoke it from the flow, or select **Create Data Mapper** to define a new one. New and existing mappers open in the [Data Mapper editor](../datamapper-editor.md), where you draw connections between source and target fields, write inline expressions, or use [**Auto Map**](/docs/develop/integration-artifacts/supporting/data-mapper/ai-mapping).

![Data Mappers picker with Create Data Mapper action and existing mappers](/img/develop/flow-design-elements/map-data-view.png)

Expand Down
1 change: 1 addition & 0 deletions en/sidebars.ts
Original file line number Diff line number Diff line change
Expand Up @@ -223,6 +223,7 @@ const sidebars: SidebarsConfig = {
},
'develop/integration-artifacts/supporting/data-mapper/generic-type-mappings',
'develop/integration-artifacts/supporting/data-mapper/submappings',
'develop/integration-artifacts/supporting/data-mapper/ai-mapping',
],
},
],
Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading