zBuilder Add custom dependencies task

zBuilder/AddCustomDependencies/README.md

@@ -0,0 +1,189 @@
+
+# Add Custom Dependencies
+
+## Overview
+The AddCustomDependencies sample is a custom zBuilder Groovy task that establishes custom dependencies between source files and related files such as configuration artifacts. It inspects the [`SOURCE_LIST`](https://www.ibm.com/docs/en/adffz/dbb/3.0.x?topic=reference-task-index) build context variable and adds logical dependencies based on configurable file patterns.
+
+This task is useful for impact build scenarios such as:
+- Changes to a source file trigger processing of a related configuration file, for example to include it in a package or trigger a test case.
+- Changes to a configuration file trigger processing of dependent source files.
+
+This task assumes that the basename of the source file and dependency file is identical. For each file in the source list, the task derives the member name from the basename, searches for a matching dependency file in a configured directory, and stores the resulting logical dependency in the DBB MetadataStore.
+
+## Contents
+
+| Folder/File | Description |
+| --- | --- |
+| [groovy/addCustomDependencies.groovy](groovy/addCustomDependencies.groovy) | Groovy script that scans source files and adds matching custom logical dependencies. |
+
+## Installation Instructions
+
+### Copy Files
+- Clone this [DBB Community Repository](https://github.com/IBM/dbb) to your workstation.
+- Copy [`addCustomDependencies.groovy`](groovy/addCustomDependencies.groovy) to the `$DBB_BUILD/groovy` directory.
+
+> **Note:** The Groovy script must reside in `$DBB_BUILD/groovy` to be discovered by the task configuration.
+
+### Integrate with `dbb-build.yaml`
+
+Configure one or more `AddCustomDependencies` tasks in your `dbb-build.yaml`. The task must run after analysis tasks that populate the `SOURCE_LIST` build context variable.
+
+A practical location is in the `impact` lifecycle after `ImpactAnalysis`.
+
+Example `dbb-build.yaml` configuration defining a bidirectional dependency between a package bind configuration file and its corresponding program:
+
+```yaml
+lifecycles:
+  - lifecycle: impact
+    tasks:
+      - Start
+      - ScannerInit
+      - MetadataInit
+      - ImpactAnalysis
+      - AddPkgDependencyToProgram
+      - AddPgmDependencyToPkgFile
+      - Languages
+      - Finish
+
+tasks:
+  - task: AddPkgDependencyToProgram
+    script: groovy/addCustomDependencies.groovy
+    variables:
+      - name: dependencyfileExtension
+        value: ".pkgbnd"
+      - name: dependencyFilePath
+        value: "config"
+      - name: forFilesFilter
+        value: ["**/*.cbl"]
+
+  - task: AddPgmDependencyToPkgFile
+    script: groovy/addCustomDependencies.groovy
+    variables:
+      - name: dependencyfileExtension
+        value: ".cbl"
+      - name: dependencyFilePath
+        value: "cobol"
+      - name: forFilesFilter
+        value: ["**/*.pkgbnd"]
+```
+
+> **Note:** Task variables can also be configured or overridden in `dbb-app.yaml`.
+
+## How It Works
+The [`addCustomDependencies.groovy`](groovy/addCustomDependencies.groovy) script performs the following:
+
+1. Reads the `SOURCE_LIST` build context variable populated by analysis tasks such as `ImpactAnalysis`, `FullAnalysis`, or `MergeAnalysis`.
+2. Derives the basename of each file in `SOURCE_LIST`.
+3. Searches the workspace for a matching file under the configured [`dependencyFilePath`](#required-task-variables).
+4. Retrieves the source file's `LogicalFile` object from the MetadataStore.
+5. Adds a custom `LogicalDependency` to the source file.
+
+## Required Task Variables
+
+The script supports the following task configuration variables:
+
+| Variable | Description |
+| --- | --- |
+| `dependencyfileExtension` | File extension of the dependency file to match |
+| `dependencyFilePath` | Path to dependency files relative to `APP_DIR` |
+| `forFilesFilter` | List of glob patterns for files that should receive custom dependencies |
+
+## Console Output
+When the task runs, it reports:
+- each custom dependency that was added
+- how many matching dependency files were found
+- how many custom dependencies were added
+
+In verbose mode, it also reports changed files that did not have a matching dependency file.
+
+## Task Dependencies
+This task consumes the following build context variable:
+
+- `SOURCE_LIST`: set of files changed in the current build, typically populated by analysis tasks such as `ImpactAnalysis`
+
+This task must run after analysis tasks in zBuilder because they identify files during incremental build processing.
+
+- `MetadataInit` must run before this task to establish a DBB MetadataStore connection
+
+This task is not intended for lifecycles that do not use a DBB MetadataStore connection, such as the `user` lifecycle.
+
+## Complete Working Example
+
+### Repository Structure
+```text
+myapp/
+├── cobol/
+│   ├── FILEA.cbl
+│   ├── FILEB.cbl
+│   └── FILEC.cbl
+└── config/
+    ├── FILEA.pkgbnd
+    ├── FILEB.pkgbnd
+    └── FILEC.pkgbnd
+```
+
+### Configuration in `dbb-build.yaml`
+```yaml
+lifecycles:
+  - lifecycle: impact
+    tasks:
+      - Start
+      - ScannerInit
+      - MetadataInit
+      - ImpactAnalysis
+      - AddPkgDependencyToProgram
+      - AddPgmDependencyToPkgFile
+      - Languages
+      - Finish
+
+tasks:
+  - task: AddPkgDependencyToProgram
+    script: groovy/addCustomDependencies.groovy
+    variables:
+      - name: dependencyfileExtension
+        value: ".pkgbnd"
+      - name: dependencyFilePath
+        value: "config"
+      - name: forFilesFilter
+        value: ["**/*.cbl"]
+
+  # Adds the COBOL program as a dependency to the package file
+  - task: AddPgmDependencyToPkgFile
+    script: groovy/addCustomDependencies.groovy
+    variables:
+      - name: dependencyfileExtension
+        value: ".cbl"
+      - name: dependencyFilePath
+        value: "cobol"
+      - name: forFilesFilter
+        value: ["**/*.pkgbnd"]
+```
+
+### Scenario Execution
+
+**Scenario A: Developer changes `FILEA.cbl`**
+1. `ImpactAnalysis` adds `cobol/FILEA.cbl` to `SOURCE_LIST`
+2. `AddPkgDependencyToProgram` finds `config/FILEA.pkgbnd`
+3. Dependency added: `FILEA.cbl` → `FILEA.pkgbnd`
+4. Subsequent build processing adds `FILEA.pkgbnd` to the build list when the COBOL program is modified. This is useful when the package bind file must also be processed and packaged.
+
+**Scenario B: Developer changes `FILEB.pkgbnd`**
+1. `ImpactAnalysis` adds `config/FILEB.pkgbnd` to `SOURCE_LIST`
+2. `AddPgmDependencyToPkgFile` finds `cobol/FILEB.cbl`
+3. Dependency added: `FILEB.pkgbnd` → `FILEB.cbl`
+4. The build recompiles the COBOL program to generate a new DBRM for rebinding when the package bind file is modified.
+
+## Implementation Notes
+
+- **Member Name Derivation**: The member name is extracted from the file basename by removing the file extension. For example, `cobol/FILEA.cbl` becomes `FILEA`.
+
+- **Dependency Path**: The dependency target is the repository-relative path of the matching dependency file (e.g., `config/FILEA.pkgbnd`).
+
+- **File Matching**: The task searches for exactly one matching file per source file. The file must exist at `{APP_DIR}/{dependencyFilePath}/{memberName}{dependencyfileExtension}`.
+
+- **Dependency Category**: The dependency category is derived from `dependencyfileExtension` by removing any leading dot and converting the value to uppercase.
+
+- **Glob Pattern Matching**: The `forFilesFilter` uses standard glob patterns:
+  - `**/*.cbl` - matches all `.cbl` files in any directory
+  - `**/config/*.pkgbnd` - matches `.pkgbnd` files only in `config` directories
+  - `["**/*.cbl", "**/*.CBL"]` - matches both lowercase and uppercase extensions

zBuilder/AddCustomDependencies/groovy/addCustomDependencies.groovy

@@ -0,0 +1,175 @@
+@groovy.transform.BaseScript com.ibm.dbb.groovy.TaskScript baseScript
+
+import com.ibm.dbb.dependency.LogicalDependency
+import com.ibm.dbb.dependency.LogicalFile
+import com.ibm.dbb.task.TaskConstants
+import com.ibm.dbb.metadata.*
+import com.ibm.dbb.metadata.Collection
+import com.ibm.dbb.build.*
+import java.nio.file.Path
+import java.nio.file.PathMatcher
+
+/**
+ * Add Custom Dependency
+ *
+ * For each file in CHANGED_FILES, this task looks for a matching package
+ * configuration file named config/<memberName>.pkg somewhere in the
+ * repository workspace. When found, it adds a custom logical dependency
+ * from the changed file to that config file path.
+ *
+ * Defaults:
+ * - lookup member name is derived from the changed file basename without extension
+ * - config directory name is "config"
+ * - config file extension is ".pkg"
+ *
+ * Optional task configuration variables:
+ * - customDependencyConfigDirectoryName
+ * - customDependencyConfigExtension
+ */
+
+Set<String> sourceFiles = context.getSetStringVariable(TaskConstants.SOURCE_LIST) ?: [] as Set<String>
+boolean verbose = context.getBooleanVariable(TaskConstants.IS_VERBOSE_MODE)
+
+String configDirectoryName = config.getVariable("dependencyFilePath")
+String configExtension = config.getVariable("dependencyfileExtension")
+List<String> forFilesFilterPatterns = config.getListVariable("forFilesFilter") ?: ["**/*", "**/.*"]
+workspacePath = context.getStringVariable(TaskConstants.WORKSPACE)
+
+if (!configDirectoryName) {
+	println ">> ERROR: Missing task configuration 'dependencyFilePath'. Exiting."
+	exit 1
+}
+
+if (!configExtension) {
+	println ">> ERROR: Missing task configuration 'dependencyfileExtension'. Exiting."
+	exit 1
+}
+
+// validate configuration variables of the task
+if (!configExtension.startsWith(".")) {
+	configExtension = ".${configExtension}"
+}
+
+String appDir = context.getStringVariable("APP_DIR")
+
+if (verbose) {
+	println "> Add custom dependency from matching repository config files"
+	println "> Application directory: ${appDir}"
+	println "> Config directory name: ${configDirectoryName}"
+	println "> Config extension: ${configExtension}"
+}
+
+if (sourceFiles.isEmpty()) {
+	println ">> No changed files found. No custom dependencies added."
+	return 0
+}
+
+int dependenciesAdded = 0
+int configMatchesFound = 0
+
+BuildGroup buildGroup = context.getBuildGroup(TaskConstants.BUILD_GROUP)
+Collection sourceCollection = buildGroup.getCollection("sources") // based on zBuilder defaults
+
+if (sourceCollection == null) {
+	if (verbose) {
+		println ">> The collection 'sources' was not found in BuildGroup.}"
+	}
+	return
+}
+
+List<LogicalFile> updatedLogicalFiles = new ArrayList<LogicalFile>()
+
+sourceFiles.each {
+
+	String sourceFile ->
+	if (!matchesForFilesFilter(sourceFile, forFilesFilterPatterns)) {
+		if (verbose) {
+			println ">> Skipping ${sourceFile} because it does not match forFilesFilter"
+		}
+		return
+	}
+
+	String memberName = deriveMemberName(sourceFile)
+	String dependencyTarget = findMatchingConfigPath(appDir, sourceFile, memberName, configDirectoryName, configExtension)
+
+
+	if (dependencyTarget == null) {
+		if (verbose) {
+			println ">> No matching config file found for ${sourceFile}"
+		}
+		return
+	}
+
+	if (dependencyTarget ){
+
+		configMatchesFound++
+
+		LogicalFile logicalFile = sourceCollection.getLogicalFile(sourceFile)
+		if (logicalFile == null) {
+			println ">> WARNING: Unable to resolve LogicalFile for ${sourceFile}. Skipping dependency ${dependencyTarget}"
+			return
+		}
+		String depName = CopyToPDS.createMemberName(dependencyTarget)
+		String library = "${configDirectoryName}".toUpperCase()
+		String category = configExtension?.replaceFirst(/^\./, '')?.toUpperCase()
+		LogicalDependency logicalDependency = new LogicalDependency(depName,library , category)
+		logicalFile.addLogicalDependency(logicalDependency)
+		updatedLogicalFiles.add(logicalFile)
+		dependenciesAdded++
+
+		println "> Added custom dependency to file '${sourceFile}'"
+	}
+}
+
+println "> Updating logical files: ${configMatchesFound}"
+sourceCollection.addLogicalFiles(updatedLogicalFiles)
+
+return 0
+
+String deriveMemberName(String sourceFile) {
+	String fileName = new File(sourceFile).getName()
+	int extensionIndex = fileName.lastIndexOf('.')
+	if (extensionIndex > 0) {
+		return fileName.substring(0, extensionIndex)
+	}
+	return fileName
+}
+
+String findMatchingConfigPath(String appDir, String sourceFile, String memberName, String configDirectoryName, String configExtension) {
+	if (appDir == null) {
+		println ">> WARNING: APP_DIR is not set. Unable to check matching config for ${sourceFile}"
+		return null
+	}
+
+	File configFile = new File("${appDir}/${configDirectoryName}/${memberName}${configExtension}")
+	if (!configFile.exists() || !configFile.isFile()) {
+		return null
+	}
+
+	return toRelativePath(configFile)
+}
+
+String toRelativePath(File file) {
+
+	if (!file.toURI().getPath().startsWith('/'))
+		return file.toURI().getPath()
+
+	String relPath = new File(workspacePath as String).toURI().relativize(file.toURI()).getPath()
+	// Directories have '/' added to the end.  Lets remove it.
+	if (relPath.endsWith('/'))
+		relPath = relPath.take(relPath.length()-1)
+	return relPath
+}
+
+
+boolean matchesForFilesFilter(String sourceFile, List<String> forFilesFilterPatterns) {
+	if (forFilesFilterPatterns == null || forFilesFilterPatterns.isEmpty()) {
+		return true
+	}
+
+	Path sourceFilePath = Path.of(sourceFile.replace('\\', '/'))
+	return forFilesFilterPatterns.any { String pattern ->
+		PathMatcher matcher = java.nio.file.FileSystems.getDefault().getPathMatcher("glob:${pattern}")
+		return matcher.matches(sourceFilePath)
+	}
+}

zBuilder/README.md

@@ -14,3 +14,4 @@ Sample/Extension | Type | Description
 | [Mortgage Application](MortgageApplication) | Sample | A copy of the MortgageApplication sample that ships with DBB v3.0.x provided to be easily consumable by IDz for Eclipse and IDz for VSCode IDEs to demonstrate the User Build feature. |
 | [Process Deleted Files](ProcessDeletedFiles) | Custom Groovy Task | This custom groovy task manages deleted files in incremental builds by adding deletion records to the DBB Build Report for packaging compatibility and deleting build artifacts. |
 | [Document Build Reason](ReasonToBuildDocumentation/) | Groovy Task | This custom groovy task inspects the build list to analyze and document which files were built because they were changed or built because they are an impacted file. |
+| [Add Custom Dependencies](AddCustomDependencies/) | Groovy Task | This custom Groovy task establishes _custom_ dependencies between source files and related files, such as configuration artifacts, based on matching basenames via a simple task configuration. |