diff --git a/en/docs/develop/copilot/overview.md b/en/docs/develop/copilot/overview.md index d31b2946511..8f7f351bbf9 100644 --- a/en/docs/develop/copilot/overview.md +++ b/en/docs/develop/copilot/overview.md @@ -78,7 +78,7 @@ Type `/` in the Copilot input bar to invoke a command for a specific task. | `/doc` | Generate documentation for your integration. | | `/openapi` | Import OpenAPI specifications. | | `/typecreator` | Create custom types. | -| `/datamap` | [Generate data mappings](../integration-artifacts/supporting/data-mapper/data-mapper.md). | +| `/datamap` | [Generate data mappings](../integration-artifacts/supporting/data-mapper/ai-mapping.md). | | `/natural-programming` | Experimental. Generate code from requirements and check for drift. | :::note @@ -89,6 +89,6 @@ Type `/` in the Copilot input bar to invoke a command for a specific task. - [Getting started](getting-started.md) — Sign in to WSO2 Integrator Copilot. - [Generate tests with AI](../test/ai-generated-cases.md) — Use Copilot to generate test cases. -- [AI data mapper](../integration-artifacts/supporting/data-mapper/data-mapper.md) — Generate data mappings using AI. +- [AI data mapper](../integration-artifacts/supporting/data-mapper/ai-mapping.md) — Generate data mappings using AI. - [Try-It tool](../test/built-in-try-it-tool.md) — Test services without leaving the IDE. - [AI usage and data handling guidelines](../../reference/ai-usage-and-data-handling-guidelines.md) — How Copilot handles your data. diff --git a/en/docs/develop/integration-artifacts/supporting/data-mapper/ai-mapping.md b/en/docs/develop/integration-artifacts/supporting/data-mapper/ai-mapping.md new file mode 100644 index 00000000000..08c83c4693a --- /dev/null +++ b/en/docs/develop/integration-artifacts/supporting/data-mapper/ai-mapping.md @@ -0,0 +1,164 @@ +--- +sidebar_position: 9 +title: AI Data Mapping +description: Generate transformation code between data structures with AI-powered field mapping. +keywords: [wso2 integrator, data mapper, ai data mapping, auto map, field mapping, transformation] +--- + +import Tabs from '@theme/Tabs'; +import TabItem from '@theme/TabItem'; + +# AI Data Mapping + +The AI Data Mapper uses AI to generate mappings between data structures, without manual field-by-field matching. It is especially useful for large or complex schemas with hundreds of fields, deeply nested records, or domain-specific formats. + +For the Data Mapper editor and manual mapping, see [Data Mapper editor](../../../understand-ide/editors/datamapper-editor.md). + +## How to use + + + + + +1. Define the input and output record types in the **Types** panel. Define inner records first, then compose them into the parent input and output records. For details on creating records, see [Type editor](../../../understand-ide/editors/type-editor.md). + + ![Types canvas with child record cards visible, and the New Type panel open for defining a parent record that references them](/img/develop/integration-artifacts/supporting/data-mapper/ai-data-mapper-define-personal-profile.png) + +2. Create the data mapper. Under **Data Mappers** in the project explorer, select **+**. Set a name, add the input type with a parameter name, set the output type, and select **Create**. + + ![Create New Data Mapper form with fields for data mapper name, input type with parameter name, and output type](/img/develop/integration-artifacts/supporting/data-mapper/ai-data-mapper-create-form.png) + +3. The Data Mapper editor opens with the input schema on the left and the output schema on the right. + + ![Data Mapper canvas with the input schema on the left and the output schema on the right with no mappings](/img/develop/integration-artifacts/supporting/data-mapper/ai-data-mapper-canvas.png) + +4. In the top-right corner of the canvas, select **Auto Map**. The WSO2 Integrator Copilot panel opens alongside the canvas with the `/datamap` command preloaded. + + ![WSO2 Integrator Copilot panel open alongside the Data Mapper canvas with the /datamap command preloaded in the input field](/img/develop/integration-artifacts/supporting/data-mapper/ai-data-mapper-copilot-command.png) + +5. Submit the command. The Copilot reads the project files, generates field mappings based on the input and output types, and integrates them into your workspace. When complete, mapping lines appear between the matched fields on the canvas. + + ![Data Mapper canvas with generated field mapping lines connecting the input and output fields, with the completion message in the Copilot panel](/img/develop/integration-artifacts/supporting/data-mapper/ai-data-mapper-result.png) + + + + +

Define data types

+ +Define the input and output record types in `types.bal`: + +```ballerina +type Student record { + int id; + string studentName; + int age; + string gender; + string[] semesterGPA; + string academicMajor; + Student[] roommates; + string address; +}; + +type PersonalProfile record { + int id; + Bio bio; + AcademicRecord academicRecord; + Accommodation accommodationDetails; +}; + +type Bio record { + string name; + string gender; + int age; +}; + +type AcademicRecord record { + string major; + string[] semesterGPA; +}; + +type Accommodation record { + int numberOfRoommates; + string address; +}; +``` + +

Define the data mapper function

+ +Define the `transform` function stub in `data_mappings.bal`: + +```ballerina +function transform(Student student) returns PersonalProfile => {}; +``` + +Select **Visualize** above the `transform` function to open it in the Data Mapper editor. In the editor, select **Auto Map** to generate the field mappings. The function is updated with the generated implementation: + +```ballerina +function transform(Student student) returns PersonalProfile => { + id: student.id, + bio: {name: student.studentName, gender: student.gender, age: student.age}, + academicRecord: {major: student.academicMajor, semesterGPA: student.semesterGPA}, + accommodationDetails: {numberOfRoommates: student.roommates.length(), address: student.address} +}; +``` + + + + +## Features + +### Automated mapping generation + +Mapping generation takes into account: + +- Field names and naming conventions +- Semantic relationships between fields +- Nested data structures +- Array types and cardinality +- Optional and nullable fields +- Domain-specific patterns such as those common in healthcare contexts + +### Advanced expression generation + +The AI Data Mapper handles complex transformation scenarios: + +- **Parsing and conversion**: Parsing such as parsing a string as an integer, and converting a value of one type to another. +- **Optional field handling**: Handling fields that may or may not be present. +- **Nested record transformation**: Deep structure mapping with proper path navigation. +- **Array-to-array mappings**: Member-wise transformations with appropriate iteration logic. + +### Supporting documentation to improve accuracy + +Upload reference materials to improve mapping accuracy. While schema-only analysis is supported, providing additional documentation helps the system understand field relationships and business rules. + +Supported formats: + +- PDF documents +- Images (JPEG, JPG, PNG) +- CSV files +- Text files + +For complex mapping scenarios involving large schemas or domain-specific requirements captured in multiple documents, upload all of them. The system analyzes all the documentation to generate accurate, context-aware transformations. + +### Sub-mapping reuse + +The AI Data Mapper detects existing data mappers in your codebase and reuses them where applicable. This reduces code duplication, ensures consistent transformation logic, and keeps the codebase compact. + +### Function extraction for large schemas + +For mappings with a large number of fields (hundreds to thousands of fields), Copilot extracts helper functions to maintain code readability and comply with language server constraints. Complex transformations involving union types, deeply nested structures, and array-to-array operations are automatically decomposed into reusable functions. + +## Responsible use + +Large language models can produce unexpected results when processing highly domain-specific or atypical schema patterns. Follow these practices to ensure accuracy: + +- Review all generated mappings before deploying to production. +- Test with representative data samples that reflect actual use cases. +- Verify that the generated transformation logic aligns with your business requirements. +- Provide feedback on incorrect or incomplete mappings to support continuous improvement. + +## What's next + +- [Data Mapper editor](../../../understand-ide/editors/datamapper-editor.md) — Open, configure, and work with the visual mapping canvas. +- [Data mapper](./data-mapper.md) — End-to-end guide to creating and using data mappers. +- [Expression editor](../../../understand-ide/editors/expression-editor.md) — Write custom expressions for individual field mappings. diff --git a/en/docs/develop/integration-artifacts/supporting/data-mapper/mapping-capabilities.md b/en/docs/develop/integration-artifacts/supporting/data-mapper/mapping-capabilities.md index f40ac13a418..d3510948dcc 100644 --- a/en/docs/develop/integration-artifacts/supporting/data-mapper/mapping-capabilities.md +++ b/en/docs/develop/integration-artifacts/supporting/data-mapper/mapping-capabilities.md @@ -47,16 +47,13 @@ When you need a focused, visual mapping rather than a free-form function, use ** ## Map with AI -Inside any data mapper view, select **Auto Map** in the top-right corner of the canvas to generate mappings automatically using the WSO2 Integrator Copilot. +Select **Auto Map** in the top-right corner of the data mapper canvas. The WSO2 Integrator Copilot panel opens with the `/datamap` command preloaded. Submit it to generate mappings between your types automatically. -![Data mapper toolbar with the Auto Map button in the top-right corner](/img/develop/integration-artifacts/supporting/data-mapper/ai-map-option.png) - -The Copilot panel opens with a `/datamap` command preloaded. Submit it to generate field mappings based on the input and output types, then review and adjust the result on the canvas. - -![Copilot panel with the /datamap command ready to generate mappings](/img/develop/integration-artifacts/supporting/data-mapper/ai-map-view.png) +For a full walkthrough with examples, see [AI data mapping](./ai-mapping.md). ## What's next - [Array mappings](./array-mappings/array-mappings.md) — Map between arrays using iteration, joins, and aggregation. - [Generic type mappings](./generic-type-mappings.md) — Generate types from a sample JSON or XML payload. - [Sub Mappings](./submappings.md) — Reuse mapping logic across multiple output fields. +- [AI data mapping](./ai-mapping.md) — Generate mapping logic using AI. diff --git a/en/docs/develop/understand-ide/editors/datamapper-editor.md b/en/docs/develop/understand-ide/editors/datamapper-editor.md index f23013beb0b..0084d80efde 100644 --- a/en/docs/develop/understand-ide/editors/datamapper-editor.md +++ b/en/docs/develop/understand-ide/editors/datamapper-editor.md @@ -73,7 +73,9 @@ The mapping area is the central region between the input and output sides. Links ## Auto Map -**Auto Map** in the header attempts to wire every output field from the available inputs in one action. It matches fields by name and compatible type, and falls back to an AI-assisted suggestion when no exact match is found. Review the generated mappings on the canvas before saving, because **Auto Map** can produce best-effort suggestions that may need adjustment. +**Auto Map** in the header opens the WSO2 Integrator Copilot panel alongside the canvas with the `/datamap` command preloaded. Submit the command to let the Copilot read the project files and generate field mappings based on the input and output types. When complete, mapping lines appear on the canvas. + +For more, see [AI data mapping](/docs/develop/integration-artifacts/supporting/data-mapper/ai-mapping). {/* ![Auto Map suggestions on the mapping canvas](/img/develop/understand-ide/editors/datamapper-editor/auto-map.png) */} diff --git a/en/docs/develop/understand-ide/editors/flow-diagram-editor/statement.md b/en/docs/develop/understand-ide/editors/flow-diagram-editor/statement.md index a4db331236e..8c12ea45a9f 100644 --- a/en/docs/develop/understand-ide/editors/flow-diagram-editor/statement.md +++ b/en/docs/develop/understand-ide/editors/flow-diagram-editor/statement.md @@ -41,7 +41,7 @@ Calls a function defined in the project, an imported library, or the Ballerina s ![Call Function button in the Statement section](/img/develop/flow-design-elements/call-function-node.png) -The function picker lists functions in three sections: `Within Project`, `Imported Functions` (functions from imported libraries such as `log`), and `Standard Library`. Select **Create Function** under `Within Project` to define a new function inline. +The function picker lists functions in three sections: **Within Project**, **Imported Functions** (functions from imported libraries such as `log`), and **Standard Library**. Select **Create Function** under **Within Project** to define a new function inline. ![Function picker showing Within Project, Imported Functions, and Standard Library entries](/img/develop/flow-design-elements/call-function-options.png) @@ -53,7 +53,7 @@ Adds a data mapper call that transforms data from one record shape to another. U ![Map Data button in the Statement section](/img/develop/flow-design-elements/map-data-node.png) -The picker lists every data mapper in the project under `Within Project`. Select an existing mapper to invoke it from the flow, or select **Create Data Mapper** to define a new one. New and existing mappers open in the [Data Mapper editor](../datamapper-editor.md), where you draw connections between source and target fields, write inline expressions, or use **Auto Map**. +The picker lists every data mapper in the project under **Within Project**. Select an existing mapper to invoke it from the flow, or select **Create Data Mapper** to define a new one. New and existing mappers open in the [Data Mapper editor](../datamapper-editor.md), where you draw connections between source and target fields, write inline expressions, or use [**Auto Map**](/docs/develop/integration-artifacts/supporting/data-mapper/ai-mapping). ![Data Mappers picker with Create Data Mapper action and existing mappers](/img/develop/flow-design-elements/map-data-view.png) diff --git a/en/sidebars.ts b/en/sidebars.ts index ef0cd6c784c..b37345fcaa4 100644 --- a/en/sidebars.ts +++ b/en/sidebars.ts @@ -223,6 +223,7 @@ const sidebars: SidebarsConfig = { }, 'develop/integration-artifacts/supporting/data-mapper/generic-type-mappings', 'develop/integration-artifacts/supporting/data-mapper/submappings', + 'develop/integration-artifacts/supporting/data-mapper/ai-mapping', ], }, ], diff --git a/en/static/img/develop/integration-artifacts/supporting/data-mapper/ai-data-mapper-canvas.png b/en/static/img/develop/integration-artifacts/supporting/data-mapper/ai-data-mapper-canvas.png new file mode 100644 index 00000000000..a254bda768a Binary files /dev/null and b/en/static/img/develop/integration-artifacts/supporting/data-mapper/ai-data-mapper-canvas.png differ diff --git a/en/static/img/develop/integration-artifacts/supporting/data-mapper/ai-data-mapper-copilot-command.png b/en/static/img/develop/integration-artifacts/supporting/data-mapper/ai-data-mapper-copilot-command.png new file mode 100644 index 00000000000..c1b8ecacd8b Binary files /dev/null and b/en/static/img/develop/integration-artifacts/supporting/data-mapper/ai-data-mapper-copilot-command.png differ diff --git a/en/static/img/develop/integration-artifacts/supporting/data-mapper/ai-data-mapper-create-form.png b/en/static/img/develop/integration-artifacts/supporting/data-mapper/ai-data-mapper-create-form.png new file mode 100644 index 00000000000..0ac2f5eff53 Binary files /dev/null and b/en/static/img/develop/integration-artifacts/supporting/data-mapper/ai-data-mapper-create-form.png differ diff --git a/en/static/img/develop/integration-artifacts/supporting/data-mapper/ai-data-mapper-define-personal-profile.png b/en/static/img/develop/integration-artifacts/supporting/data-mapper/ai-data-mapper-define-personal-profile.png new file mode 100644 index 00000000000..63247a6807e Binary files /dev/null and b/en/static/img/develop/integration-artifacts/supporting/data-mapper/ai-data-mapper-define-personal-profile.png differ diff --git a/en/static/img/develop/integration-artifacts/supporting/data-mapper/ai-data-mapper-result.png b/en/static/img/develop/integration-artifacts/supporting/data-mapper/ai-data-mapper-result.png new file mode 100644 index 00000000000..31378faab40 Binary files /dev/null and b/en/static/img/develop/integration-artifacts/supporting/data-mapper/ai-data-mapper-result.png differ