EVALUATION_API.md

ClassAd Evaluation API

This document describes the public API for creating, parsing, and evaluating HTCondor ClassAds in Go.

Overview

The classad package provides a high-level API for working with ClassAds, including:

• Creating ClassAds programmatically
• Parsing ClassAd expressions
• Evaluating expressions with type safety
• Modifying ClassAd attributes

The library offers two API styles:

• Modern Generic API (recommended): Set(), GetAs[T](), GetOr[T]()
• Traditional API (still supported): InsertAttr*(), EvaluateAttr*()

Quick Start

Modern API (Recommended)

import "github.com/PelicanPlatform/classad/classad"

// Create a new ClassAd with Set()
ad := classad.New()
ad.Set("Cpus", 4)
ad.Set("Memory", 8192.0)
ad.Set("Name", "worker-01")
ad.Set("Tags", []string{"prod", "gpu"})

// Parse a ClassAd from string
jobAd, err := classad.Parse(`[
    JobId = 1001;
    Owner = "alice";
    Cpus = 2;
    Requirements = (Cpus >= 2) && (Memory >= 2048)
]`)

// Type-safe retrieval with GetAs[T]()
if cpus, ok := classad.GetAs[int](jobAd, "Cpus"); ok {
    fmt.Printf("Cpus = %d\n", cpus)
}

if owner, ok := classad.GetAs[string](jobAd, "Owner"); ok {
    fmt.Printf("Owner = %s\n", owner)
}

// Get with defaults using GetOr[T]()
priority := classad.GetOr(jobAd, "Priority", 10)
status := classad.GetOr(jobAd, "Status", "Unknown")

Traditional API (Still Supported)

// Create a new ClassAd with InsertAttr methods
ad := classad.New()
ad.InsertAttr("Cpus", 4)
ad.InsertAttrFloat("Memory", 8192.0)
ad.InsertAttrString("Name", "worker-01")

// Evaluate attributes with type-specific methods
if cpus, ok := jobAd.EvaluateAttrInt("Cpus"); ok {
    fmt.Printf("Cpus = %d\n", cpus)
}

if requirements, ok := jobAd.EvaluateAttrBool("Requirements"); ok {
    fmt.Printf("Requirements = %v\n", requirements)
}

API Reference

ClassAd Type

The ClassAd type represents a ClassAd object and provides methods for manipulating attributes.

Creation and Parsing

• New() *ClassAd - Creates a new empty ClassAd
• Parse(input string) (*ClassAd, error) - Parses a ClassAd from a string (new format)
• ParseOld(input string) (*ClassAd, error) - Parses a ClassAd from a string (old format)

Reading Multiple ClassAds

The library provides two styles of iterators for parsing multiple ClassAds from an io.Reader.

Traditional Iterator Pattern (Go 1.21+):

• NewReader(r io.Reader) *Reader - Creates a Reader for new-style ClassAds (with brackets)
• NewOldReader(r io.Reader) *Reader - Creates a Reader for old-style ClassAds (newline-delimited)
• Next() bool - Advances to the next ClassAd, returns true if one was found
• ClassAd() *ClassAd - Returns the current ClassAd (call after Next() returns true)
• Err() error - Returns any error that occurred during iteration

Go 1.23+ Range-over-Function Pattern:

• All(r io.Reader) Seq - Iterator for new-style ClassAds
• AllOld(r io.Reader) Seq - Iterator for old-style ClassAds
• AllWithIndex(r io.Reader) Seq2 - Iterator with index for new-style ClassAds
• AllOldWithIndex(r io.Reader) Seq2 - Iterator with index for old-style ClassAds
• AllWithError(r io.Reader, errPtr *error) Seq - Iterator with error capture for new-style
• AllOldWithError(r io.Reader, errPtr *error) Seq - Iterator with error capture for old-style

Example Usage (Traditional Pattern):

import (
    "os"
    "github.com/PelicanPlatform/classad/classad"
)

// Read new-style ClassAds from file
file, _ := os.Open("jobs.classads")
defer file.Close()

reader := classad.NewReader(file)
for reader.Next() {
    ad := reader.ClassAd()
    // Process ClassAd with modern API
    owner := classad.GetOr(ad, "Owner", "unknown")
    cpus := classad.GetOr(ad, "Cpus", 0)
}
if err := reader.Err(); err != nil {
    log.Fatal(err)
}

// Read old-style ClassAds
oldFile, _ := os.Open("machines.classads")
defer oldFile.Close()

oldReader := classad.NewOldReader(oldFile)
for oldReader.Next() {
    ad := oldReader.ClassAd()
    // Process ClassAd...
}
if err := oldReader.Err(); err != nil {
    log.Fatal(err)
}

Example Usage (Go 1.23+ Range-over-Function):

import (
    "os"
    "strings"
    "github.com/PelicanPlatform/classad/classad"
)

// Simple iteration with modern API
for ad := range classad.All(strings.NewReader(input)) {
    owner := classad.GetOr(ad, "Owner", "unknown")
    cpus := classad.GetOr(ad, "Cpus", 0)
    fmt.Printf("Owner: %s, Cpus: %d\n", owner, cpus)
}

// Iteration with index
for i, ad := range classad.AllWithIndex(file) {
    jobId := classad.GetOr(ad, "JobId", 0)
    fmt.Printf("ClassAd %d: JobId=%d\n", i, jobId)
}

// Iteration with error handling
var err error
for ad := range classad.AllWithError(file, &err) {
    if name, ok := classad.GetAs[string](ad, "Name"); ok {
        fmt.Printf("Name: %s\n", name)
    }
}
if err != nil {
    log.Fatal(err)
}

Attribute Manipulation

Modern API (Recommended):

• Set(name string, value any) error - Sets an attribute with any type (generic)
• GetAs[T any](ad *ClassAd, name string) (T, bool) - Type-safe generic retrieval
• GetOr[T any](ad *ClassAd, name string, defaultValue T) T - Get with default value

Traditional API (Still Supported):

• InsertAttr(name string, value interface{}) - Inserts an attribute (auto-detects type)
• InsertAttrInt(name string, value int64) - Inserts an integer attribute
• InsertAttrFloat(name string, value float64) - Inserts a float attribute
• InsertAttrString(name string, value string) - Inserts a string attribute
• InsertAttrBool(name string, value bool) - Inserts a boolean attribute

Common Methods:

• Insert(name string, expr ast.Expr) - Inserts an attribute with an AST expression
• InsertExpr(name string, expr *Expr) - Inserts an attribute with an Expr (see Expression API)
• Lookup(name string) (*Expr, bool) - Returns the unevaluated expression for an attribute
• Delete(name string) bool - Deletes an attribute
• Clear() - Removes all attributes
• Size() int - Returns the number of attributes
• GetAttributes() []string - Returns a list of all attribute names

Evaluation Methods

Modern API (Recommended):

Using the generic functions:

// Type-safe retrieval with two-value return
if cpus, ok := classad.GetAs[int](ad, "Cpus"); ok {
    fmt.Printf("Cpus: %d\n", cpus)
}

if owner, ok := classad.GetAs[string](ad, "Owner"); ok {
    fmt.Printf("Owner: %s\n", owner)
}

// Get with defaults (no error checking needed)
priority := classad.GetOr(ad, "Priority", 10)
status := classad.GetOr(ad, "Status", "Unknown")
tags := classad.GetOr(ad, "Tags", []string{"default"})

Traditional API (Still Supported):

• EvaluateAttr(name string) Value - Evaluates an attribute and returns a Value
• EvaluateAttrInt(name string) (int64, bool) - Evaluates as integer
• EvaluateAttrReal(name string) (float64, bool) - Evaluates as float
• EvaluateAttrNumber(name string) (float64, bool) - Evaluates as number (int or float)
• EvaluateAttrString(name string) (string, bool) - Evaluates as string
• EvaluateAttrBool(name string) (bool, bool) - Evaluates as boolean
• EvaluateExpr(expr ast.Expr) Value - Evaluates an AST expression
• EvaluateExprString(exprStr string) (Value, error) - Parses and evaluates an expression string
• EvaluateExprWithTarget(expr *Expr, target *ClassAd) Value - Evaluates an Expr with a target ClassAd (see Scoped Evaluation)

Expression API

The Expression API provides first-class support for working with unevaluated ClassAd expressions. This enables advanced use cases such as copying expressions between ClassAds, inspecting expressions, and evaluating expressions with explicit scope contexts.

Expr Type

The Expr type represents an unevaluated ClassAd expression. It wraps the internal AST representation and provides methods for evaluation and inspection.

Creating Expressions

ParseExpr - Parse an expression from a string:

expr, err := classad.ParseExpr("Cpus * 2 + Memory / 1024")
if err != nil {
    log.Fatal(err)
}
fmt.Println(expr.String())  // "((Cpus * 2) + (Memory / 1024))"

Lookup - Get unevaluated expressions from ClassAds:

ad, _ := classad.Parse("[x = 10; y = x * 2]")
if expr, ok := ad.Lookup("y"); ok {
    fmt.Println(expr.String())  // "(x * 2)"
}

Evaluating Expressions

Eval - Evaluate in a ClassAd context:

expr, _ := classad.ParseExpr("Cpus * 2")
ad := classad.New()
ad.Set("Cpus", 8)

result := expr.Eval(ad)
if value, ok := result.IntValue(); ok {
    fmt.Printf("Result: %d\n", value)  // Result: 16
}

EvalWithContext - Evaluate with explicit MY and TARGET scopes:

job := classad.New()
job.Set("RequestCpus", 4)

machine := classad.New()
machine.Set("Cpus", 8)

expr, _ := classad.ParseExpr("MY.RequestCpus <= TARGET.Cpus")
result := expr.EvalWithContext(job, machine)  // job=MY, machine=TARGET

if matches, ok := result.BoolValue(); ok {
    fmt.Printf("Match: %v\n", matches)  // Match: true
}

Expr Methods

• String() string - Returns the string representation of the expression
• Eval(scope *ClassAd) Value - Evaluates the expression in the given ClassAd context
• EvalWithContext(scope, target *ClassAd) Value - Evaluates with explicit MY (scope) and TARGET contexts

Copying Expressions

Expressions can be copied between ClassAds without evaluation:

// Create a template ClassAd with common expressions
template, _ := classad.Parse(`[
    StandardReq = (Cpus >= 2) && (Memory >= 4096);
    ResourceScore = Cpus * 1000 + Memory / 1024
]`)

// Create a new ClassAd and copy expressions
newAd := classad.New()
newAd.Set("Cpus", 4)
newAd.Set("Memory", 8192)

// Copy the StandardReq expression
if req, ok := template.Lookup("StandardReq"); ok {
    newAd.Set("Requirements", req)
}

// Copy the ResourceScore expression
if score, ok := template.Lookup("ResourceScore"); ok {
    newAd.Set("Score", score)
}

// Evaluate in new context with modern API
if reqVal, ok := classad.GetAs[bool](newAd, "Requirements"); ok {
    fmt.Printf("Requirements: %v\n", reqVal)  // true
}
if scoreVal, ok := classad.GetAs[int](newAd, "Score"); ok {
    fmt.Printf("Score: %d\n", scoreVal)  // 4008
}

Scoped Evaluation

The Expression API provides explicit control over MY and TARGET scopes for match-making scenarios:

job := classad.New()
job.Set("RequestCpus", 4)
job.Set("RequestMemory", 8192)

machine := classad.New()
machine.Set("Cpus", 8)
machine.Set("Memory", 16384)

// Job requirements: MY=job, TARGET=machine
jobReq, _ := classad.ParseExpr("MY.RequestCpus <= TARGET.Cpus && MY.RequestMemory <= TARGET.Memory")
jobMatches := jobReq.EvalWithContext(job, machine)

// Machine requirements: MY=machine, TARGET=job
machineReq, _ := classad.ParseExpr("TARGET.RequestCpus <= MY.Cpus")
machineAccepts := machineReq.EvalWithContext(machine, job)

// Or use the ClassAd method
jobMatches = job.EvaluateExprWithTarget(jobReq, machine)
machineAccepts = machine.EvaluateExprWithTarget(machineReq, job)

Use Cases

• Expression Templates: Define common expressions once and copy to multiple ClassAds
• Cross-ClassAd Evaluation: Evaluate expressions that reference attributes from multiple ClassAds
• Match-Making: Implement symmetric job-machine matching with explicit scopes
• Expression Libraries: Build reusable expression libraries for common requirements
• Dynamic Policies: Parse policy expressions at runtime and apply to ClassAds

See examples/expr_demo for comprehensive examples.

Value Type

The Value type represents an evaluated ClassAd value and can be one of 9 types:

• UndefinedValue - Undefined/missing value
• ErrorValue - Error during evaluation
• BooleanValue - Boolean (true/false)
• IntegerValue - 64-bit integer
• RealValue - 64-bit float
• StringValue - String
• ListValue - List of Values
• ClassAdValue - Nested ClassAd

Value Construction

• NewUndefinedValue() Value
• NewErrorValue() Value
• NewBoolValue(b bool) Value
• NewIntValue(i int64) Value
• NewRealValue(r float64) Value
• NewStringValue(s string) Value
• NewListValue(list []Value) Value
• NewClassAdValue(ad *ClassAd) Value

Value Type Checking

• Type() ValueType - Returns the type
• IsUndefined() bool
• IsError() bool
• IsBool() bool
• IsInteger() bool
• IsReal() bool
• IsNumber() bool - True for integer or real
• IsString() bool
• IsList() bool
• IsClassAd() bool

Value Extraction

• BoolValue() (bool, error)
• IntValue() (int64, error)
• RealValue() (float64, error)
• NumberValue() (float64, error) - Converts integer to float if needed
• StringValue() (string, error)
• ListValue() ([]Value, error)
• ClassAdValue() (*ClassAd, error)
• String() string - Returns string representation

Expression Evaluation

The evaluator supports:

Arithmetic Operators

• Addition: +
• Subtraction: -
• Multiplication: *
• Division: /
• Modulo: %
• Unary plus/minus: +x, -x

Comparison Operators

• Less than: <
• Greater than: >
• Less than or equal: <=
• Greater than or equal: >=
• Equal: ==
• Not equal: !=

Logical Operators

• Logical AND: &&
• Logical OR: ||
• Logical NOT: !

Conditional Operator

• Ternary: condition ? true_value : false_value
• Functional form: ifThenElse(condition, true_value, false_value) - Evaluates condition and returns appropriate branch

ad, _ := classad.Parse(`[
    x = 10;
    y = 20;

    // Ternary operator
    maxTernary = (x > y) ? x : y;

    // Functional form (useful in nested expressions)
    maxFunc = ifThenElse(x > y, x, y);

    // Can return different types
    status = ifThenElse(x > 5, "high", 0);

    // Handles undefined and error properly
    safeDiv = ifThenElse(y != 0, x / y, undefined)
]`)
// maxTernary = 20
// maxFunc = 20
// status = "high"
// safeDiv = 0.5

ifThenElse behavior:

• Evaluates first argument as condition
• If condition is true, returns second argument
• If condition is false, returns third argument
• If condition is undefined or error, returns that value
• If condition is not boolean, returns error

Attribute References

• Simple: Cpus
• In expressions: Cpus * 2 + Memory / 1024

Scoped Attribute References

ClassAds support scoped attribute references for accessing attributes in related ClassAds:

• MY.attr - References an attribute in the current ClassAd
• TARGET.attr - References an attribute in the target ClassAd (set via SetTarget())
• PARENT.attr - References an attribute in the parent ClassAd (set via SetParent())

// Create a job and machine ClassAd
job := classad.New()
job.Set("Cpus", 2)
job.Set("Memory", 2048)

// Insert Requirements as an expression, not a string
reqExpr, _ := classad.ParseExpr("TARGET.Cpus >= MY.Cpus && TARGET.Memory >= MY.Memory")
job.Set("Requirements", reqExpr)

machine := classad.New()
machine.Set("Cpus", 4)
machine.Set("Memory", 8192)

// Set target to enable TARGET.* references
job.SetTarget(machine)

// Evaluate Requirements with TARGET references
if requirements, ok := classad.GetAs[bool](job, "Requirements"); ok {
    fmt.Printf("Match: %v\n", requirements)  // true
}

Scoped Reference API:

• SetTarget(target *ClassAd) - Sets the target ClassAd for TARGET.* references
• GetTarget() *ClassAd - Returns the current target ClassAd
• SetParent(parent *ClassAd) - Sets the parent ClassAd for PARENT.* references
• GetParent() *ClassAd - Returns the current parent ClassAd

Behavior:

• MY.attr always references the current ClassAd (equivalent to attr)
• TARGET.attr evaluates to undefined if no target is set
• PARENT.attr evaluates to undefined if no parent is set
• Scoped references work in all expressions (requirements, rank, etc.)

Type Coercion

• Integer + Real → Real
• Comparisons work across numeric types
• String comparisons are lexicographic

Error Handling

• Undefined attributes evaluate to UndefinedValue
• Type mismatches return ErrorValue
• Division by zero returns ErrorValue
• Errors propagate through expressions

Examples

See examples/api_demo/main.go for comprehensive examples using the modern API:

1. Creating ClassAds programmatically with Set()
2. Parsing ClassAds from strings
3. Looking up attributes
4. Type-safe retrieval with GetAs[T]()
5. Using GetOr[T]() with defaults
6. Complex expressions
7. Arithmetic operations
8. Logical expressions
9. Conditional expressions
10. Modifying ClassAds
11. Real-world HTCondor scenarios
12. Handling undefined values

See examples/generic_api_demo/main.go for focused examples of the modern generic API.

See examples/features_demo/main.go for advanced features including:

• Scoped attribute references (MY., TARGET., PARENT.)
• ClassAd matching with MatchClassAd

Run the examples with:

go run ./examples/api_demo/main.go
go run ./examples/generic_api_demo/main.go
go run ./examples/features_demo/main.go

Testing

Run the test suite:

go test ./classad/...

The test suite includes:

• ClassAd CRUD operations
• Expression evaluation
• Type checking and coercion
• Error handling
• Value operations
• Arithmetic, comparison, and logical operations
• Unary operations
• Complex expressions
• Nested ClassAds and lists
• IS/ISNT operators
• Built-in functions
• Generic API (Set, GetAs, GetOr)

Nested ClassAds and Lists

ClassAds support nested structures:

Using Modern API

// Lists
ad, _ := classad.Parse(`[numbers = {1, 2, 3, 4, 5}]`)

// Get list with type safety
if numbers, ok := classad.GetAs[[]interface{}](ad, "numbers"); ok {
    fmt.Printf("Numbers: %v\n", numbers)
}

// Nested ClassAds
ad, _ := classad.Parse(`[
    server = [host = "example.com"; port = 8080];
    name = "web-server"
]`)

if server, ok := classad.GetAs[*classad.ClassAd](ad, "server"); ok {
    host := classad.GetOr(server, "host", "localhost")
    port := classad.GetOr(server, "port", 80)
    fmt.Printf("Server: %s:%d\n", host, port)
}

Using Traditional API

// Lists
ad, _ := classad.Parse(`[numbers = {1, 2, 3, 4, 5}]`)
numbersVal := ad.EvaluateAttr("numbers")
if numbersVal.IsList() {
    list, _ := numbersVal.ListValue()
    // Access list elements
}

// Nested ClassAds
ad, _ := classad.Parse(`[
    server = [host = "example.com"; port = 8080];
    name = "web-server"
]`)
serverVal := ad.EvaluateAttr("server")
if serverVal.IsClassAd() {
    serverAd, _ := serverVal.ClassAdValue()
    host, _ := serverAd.EvaluateAttrString("host")
    port, _ := serverAd.EvaluateAttrInt("port")
}

IS and ISNT Operators

The is and isnt operators (and their aliases =?= and =!=) provide strict identity checking (type and value):

// Unlike ==, 'is' checks type identity
ad, _ := classad.Parse(`[
    sameType = (5 is 5);              // true - same type and value
    diffType = (5 is 5.0);            // false - different types (int vs real)
    equalNotIs = (5 == 5.0);          // true - == allows type coercion
    undefCheck = (undefined is undefined);  // true
    errorCheck = (error is error);          // true

    // Meta-equal operator aliases
    metaEqual = (5 =?= 5);            // true - same as 'is'
    metaNotEqual = (5 =!= 5.0);       // true - same as 'isnt'
]`)

Operator Aliases:

• =?= is an alias for is (meta-equal operator)
• =!= is an alias for isnt (meta-not-equal operator)

Key differences from ==:

• is/=?= requires exact type match (no coercion)
• is/=?= can compare undefined and error values
• is/=?= compares list elements recursively
• isnt/=!= is the negation of is/=?=

Built-in Functions

String Functions

• strcat(str1, str2, ...) - Concatenates strings
• substr(string, offset[, length]) - Extracts substring (supports negative offsets)
• size(string_or_list) - Returns length of string or list
• toLower(string) / tolower(string) - Converts to lowercase
• toUpper(string) / toupper(string) - Converts to uppercase
• stringListMember(string, string_list[, delimiter]) - Tests if string is in a delimited list (default delimiters: comma and space; case-sensitive)
• stringListIMember(string, string_list[, delimiter]) - Case-insensitive variant of stringListMember
• regexp(pattern, target[, options]) - Tests if target matches regular expression pattern

ad, _ := classad.Parse(`[
    greeting = strcat("Hello", " ", "World");
    sub = substr("Hello World", 0, 5);
    len = size("Hello");
    lower = toLower("HELLO");
    upper = toUpper("world");

    // String list membership
    colors = "red,green,blue";
    hasRed = stringListMember("red", colors);           // true
    hasYellow = stringListMember("yellow", colors);     // false
    hasGreen = stringListIMember("GREEN", colors);      // true (case-insensitive)

    // Regular expression matching
    email = "user@example.com";
    validEmail = regexp("^[^@]+@[^@]+\\.[^@]+$", email);  // true
    startsWithUser = regexp("^user", email);              // true
    caseMatch = regexp("USER", email, "i");               // true (case-insensitive)
]`)
// greeting = "Hello World"
// sub = "Hello"
// len = 5
// lower = "hello"
// upper = "WORLD"
// hasRed = true
// hasYellow = false
// hasGreen = true
// validEmail = true
// startsWithUser = true
// caseMatch = true

stringList delimiters:

• The optional trailing argument to stringListMember/stringListIMember is the set of delimiter characters (default: comma and space), not a case option. Use stringListIMember for case-insensitive matching.

regexp options:

• "i" - Case-insensitive matching
• "m" - Multiline mode (^ and $ match line boundaries)
• "s" - Single-line mode (. matches newlines)
• Options can be combined: "im", "ims", etc.

Math Functions

• floor(number) - Returns floor as integer
• ceiling(number) / ceil(number) - Returns ceiling as integer
• round(number) - Rounds to nearest integer
• random([max]) - Returns random real 0-1 (or 0-max)
• int(value) - Converts to integer
• real(value) - Converts to real

ad, _ := classad.Parse(`[
    f = floor(3.7);       // 3
    c = ceiling(3.2);     // 4
    r = round(3.5);       // 4
    i = int(3.9);         // 3
    rl = real(5);         // 5.0
    rand = random(100)    // random float 0-100
]`)

Type Checking Functions

• isUndefined(value) - Returns true if value is undefined
• isError(value) - Returns true if value is an error
• isString(value) - Returns true if value is a string
• isInteger(value) - Returns true if value is an integer
• isReal(value) - Returns true if value is a real number
• isBoolean(value) - Returns true if value is a boolean
• isList(value) - Returns true if value is a list
• isClassAd(value) - Returns true if value is a ClassAd

ad, _ := classad.Parse(`[
    x = 42;
    checkInt = isInteger(x);      // true
    checkStr = isString(x);       // false
    checkUndef = isUndefined(y)   // true (y doesn't exist)
]`)

List Functions

• member(element, list) - Returns true if element is in list

ad, _ := classad.Parse(`[
    nums = {1, 2, 3, 4, 5};
    hasThree = member(3, nums);   // true
    hasTen = member(10, nums)     // false
]`)

Time Functions

• time() - Returns current Unix timestamp (seconds since epoch)

ad, _ := classad.Parse(`[now = time()]`)

Attribute Selection Expressions

Access nested ClassAd attributes using dot notation (record.field):

ad, _ := classad.Parse(`[
    employee = [
        name = "Alice";
        department = [
            name = "Engineering";
            location = "Building A"
        ]
    ];
    empName = employee.name;
    deptName = employee.department.name;
    deptLoc = employee.department.location
]`)

// Access values with modern API
name := classad.GetOr(ad, "empName", "")              // "Alice"
dept := classad.GetOr(ad, "deptName", "")             // "Engineering"
location := classad.GetOr(ad, "deptLoc", "")          // "Building A"

Behavior:

• Returns undefined if attribute doesn't exist
• Returns error if left side is not a ClassAd
• Can chain multiple selections: a.b.c.d

Subscript Expressions

Access list elements or ClassAd attributes using subscript notation:

List Subscripting

Use integer indices (0-based) to access list elements:

ad, _ := classad.Parse(`[
    fruits = {"apple", "banana", "cherry"};
    matrix = {{1, 2, 3}, {4, 5, 6}, {7, 8, 9}};

    first = fruits[0];
    third = fruits[2];
    element = matrix[1][2]
]`)

// Access with modern API
first := classad.GetOr(ad, "first", "")      // "apple"
third := classad.GetOr(ad, "third", "")      // "cherry"
element := classad.GetOr(ad, "element", 0)   // 6

ClassAd Subscripting

Use string keys to access ClassAd attributes:

ad, _ := classad.Parse(`[
    person = [name = "Bob"; age = 30];
    personName = person["name"];
    personAge = person["age"]
]`)

// Access with modern API
name := classad.GetOr(ad, "personName", "")  // "Bob"
age := classad.GetOr(ad, "personAge", 0)     // 30

Combined Selection and Subscripting

Mix selection and subscripting for complex data access:

ad, _ := classad.Parse(`[
    company = [
        employees = {
            [name = "Alice"; salary = 100000],
            [name = "Bob"; salary = 95000]
        }
    ];
    firstEmpName = company.employees[0].name;
    secondSalary = company.employees[1].salary
]`)

// Modern API
name := classad.GetOr(ad, "firstEmpName", "")        // "Alice"
salary := classad.GetOr(ad, "secondSalary", 0)       // 95000

Subscript Behavior:

• Lists: Index must be integer, returns undefined if out of bounds
• ClassAds: Key must be string, returns undefined if not found
• Returns error for type mismatches (e.g., string index on list)

ClassAd Matching with MatchClassAd

The MatchClassAd type provides symmetric matching between two ClassAds, inspired by the HTCondor C++ API. It automatically sets up bidirectional TARGET references to enable requirements like TARGET.Memory >= MY.Memory.

Creating a MatchClassAd

import "github.com/PelicanPlatform/classad/classad"

// Create job and machine ClassAds with modern API
job := classad.New()
job.Set("Cpus", 2)
job.Set("Memory", 2048)

// Insert Requirements as expressions
jobReq, _ := classad.ParseExpr("TARGET.Cpus >= MY.Cpus && TARGET.Memory >= MY.Memory")
job.Set("Requirements", jobReq)

machine := classad.New()
machine.Set("Cpus", 4)
machine.Set("Memory", 8192)

machineReq, _ := classad.ParseExpr("TARGET.Cpus <= MY.Cpus && TARGET.Memory <= MY.Memory")
machine.Set("Requirements", machineReq)

// Create MatchClassAd - automatically sets up TARGET references
matchAd := classad.NewMatchClassAd(job, machine)

MatchClassAd API

• NewMatchClassAd(left, right *ClassAd) *MatchClassAd - Creates a MatchClassAd with bidirectional TARGET setup
• GetLeftAd() *ClassAd - Returns the left ClassAd
• GetRightAd() *ClassAd - Returns the right ClassAd
• ReplaceLeftAd(ad *ClassAd) - Replaces the left ClassAd and updates TARGET references
• ReplaceRightAd(ad *ClassAd) - Replaces the right ClassAd and updates TARGET references

Symmetric Matching

The Symmetry() and Match() methods evaluate requirements from both sides:

// Check if both Requirements attributes evaluate to true
match := matchAd.Match()
if match {
    fmt.Println("Job and machine match!")
}

// Or use custom requirement attribute names
leftReq := "JobRequirements"
rightReq := "MachineRequirements"
customMatch := matchAd.Symmetry(leftReq, rightReq)

Match Behavior:

• Match() uses the default "Requirements" attribute
• Symmetry(leftReq, rightReq) uses custom attribute names
• Returns true only if both requirements evaluate to true
• Returns false if either requirement is false, undefined, or error

Rank Evaluation

After matching, you can evaluate rank expressions to prioritize matches:

// Evaluate rank from the left side's perspective
rankExpr, _ := classad.ParseExpr("TARGET.Memory * 2 + TARGET.Cpus")
job.Set("Rank", rankExpr)
leftRank := matchAd.EvaluateRankLeft("Rank")
if leftRank.IsReal() {
    rank, _ := leftRank.RealValue()
    fmt.Printf("Job rank: %.2f\n", rank)
}

// Evaluate rank from the right side's perspective
machineRankExpr, _ := classad.ParseExpr("1000 / TARGET.Memory")
machine.Set("Rank", machineRankExpr)
rightRank := matchAd.EvaluateRankRight("Rank")

Rank Methods:

• EvaluateRankLeft(rankName string) Value - Evaluates rank attribute from left ClassAd
• EvaluateRankRight(rankName string) Value - Evaluates rank attribute from right ClassAd
• Rank expressions can reference both MY.* and TARGET.* attributes

Complete Matching Example

// Job ClassAd with modern API
job := classad.New()
job.Set("Cpus", 2)
job.Set("Memory", 2048)
job.Set("Owner", "alice")

// Insert Requirements and Rank as expressions
jobReq, _ := classad.ParseExpr("TARGET.Cpus >= MY.Cpus && TARGET.Memory >= MY.Memory")
job.Set("Requirements", jobReq)

jobRank, _ := classad.ParseExpr("TARGET.Memory")  // Prefer more memory
job.Set("Rank", jobRank)

// Machine ClassAd with modern API
machine := classad.New()
machine.Set("Cpus", 4)
machine.Set("Memory", 8192)
machine.Set("Name", "slot1@worker1")

machineReq, _ := classad.ParseExpr("TARGET.Cpus <= MY.Cpus")
machine.Set("Requirements", machineReq)

machineRank, _ := classad.ParseExpr("1000 - TARGET.Memory")  // Prefer lighter jobs
machine.Set("Rank", machineRank)

// Create MatchClassAd and check match
matchAd := classad.NewMatchClassAd(job, machine)

if matchAd.Match() {
    fmt.Println("Match successful!")

    // Evaluate ranks
    jobRank := matchAd.EvaluateRankLeft("Rank")
    machineRank := matchAd.EvaluateRankRight("Rank")

    if jobRank.IsReal() && machineRank.IsReal() {
        jr, _ := jobRank.RealValue()
        mr, _ := machineRank.RealValue()
        fmt.Printf("Job rank: %.2f, Machine rank: %.2f\n", jr, mr)
    }
}

Dynamic Replacement

You can replace ClassAds in a MatchClassAd while preserving the bidirectional TARGET setup:

matchAd := classad.NewMatchClassAd(job1, machine1)

// Replace with new ClassAds - TARGET references automatically updated
matchAd.ReplaceLeftAd(job2)
matchAd.ReplaceRightAd(machine2)

// Check match with new ads
if matchAd.Match() {
    fmt.Println("New match successful!")
}

This is useful for:

• Reusing MatchClassAd objects in matching loops
• Testing multiple job-machine combinations
• Implementing HTCondor-style matchmaking algorithms

Error Handling

Functions properly propagate undefined and error values:

ad, _ := classad.Parse(`[
    x = undefined;
    result = size(x)  // result is undefined
]`)

ad2, _ := classad.Parse(`[
    x = error;
    result = size(x)  // result is error
]`)

Compatibility

This API is designed to mimic the C++ HTCondor ClassAd library, providing similar functionality:

• Insert*() methods for type-safe attribute insertion
• EvaluateAttr*() methods for type-safe evaluation
• Lookup() for accessing raw expressions
• Value type system matching ClassAd semantics
• Built-in functions matching HTCondor ClassAd functions
• IS/ISNT operators for strict identity checking

Implementation Status

✅ Implemented:

• Complete ClassAd CRUD API
• Expression evaluation (arithmetic, logical, comparison)
• Conditional expressions (ternary operator and ifThenElse function)
• Nested ClassAds and lists
• IS/ISNT operators (with =?= and =!= aliases)
• Attribute selection expressions (record.field)
• Subscript expressions (list[index], record["key"])
• Scoped attribute references (MY., TARGET., PARENT.)
• ClassAd matching with MatchClassAd
• Old ClassAd format support (newline-delimited, no brackets)
• Built-in functions:
  • String functions (strcat, substr, size, toLower, toUpper, stringListMember, regexp)
  • Math functions (floor, ceiling, round, random, int, real)
  • Type checking functions (isUndefined, isError, isString, etc.)
  • List functions (member)
  • Time functions (time)
  • Conditional function (ifThenElse)
• String escape sequences per HTCondor specification:
  • Standard escapes: \b, \t, \n, \f, \r, \\, \", \'
  • Octal sequences: \0-7 (3 digits for 0-3, 2 digits for 4-7)

🚧 Future Enhancements:

• Bitwise operators (&, |, ^, ~)
• Shift operators (<<, >>, >>>)
• Additional built-in functions as needed

Old ClassAd Format

The library supports both "old" and "new" ClassAd formats used by HTCondor:

New Format (Default)

ad, err := classad.Parse(`[
    Foo = 3;
    Bar = "hello";
    Moo = Foo =!= Undefined
]`)

Characteristics:

• Enclosed in square brackets [ ]
• Attributes separated by semicolons ;
• Standard in HTCondor 7.5.1 and later
• Supports all ClassAd features

Old Format

ad, err := classad.ParseOld(`Foo = 3
Bar = "hello"
Moo = Foo =!= Undefined`)

Characteristics:

• No surrounding brackets
• Attributes separated by newlines
• Used in HTCondor versions before 7.5.1
• Compatible with older HTCondor tools and output

Implementation Details

The old ClassAd parser converts the old format to new format internally by:

1. Adding surrounding brackets [ ]
2. Adding semicolons ; after each attribute assignment
3. Preserving comments and empty lines
4. Reusing the existing parser for full feature support

This ensures that old ClassAds have access to all features including:

• Nested ClassAds and lists
• Scoped attribute references
• Built-in functions
• All operators and expressions

Example: Equivalent Formats

Old Format:

MyType = "Machine"
TargetType = "Job"
Machine = "froth.cs.wisc.edu"
Arch = "INTEL"
OpSys = "LINUX"
Disk = 35882
Memory = 128
Requirements = TARGET.Owner=="smith" || LoadAvg<=0.3

New Format:

[
MyType = "Machine";
TargetType = "Job";
Machine = "froth.cs.wisc.edu";
Arch = "INTEL";
OpSys = "LINUX";
Disk = 35882;
Memory = 128;
Requirements = TARGET.Owner=="smith" || LoadAvg<=0.3
]

Both formats parse to the same internal representation and can be evaluated identically.

Expression Introspection and Utilities

The library provides powerful tools for analyzing, validating, and optimizing expressions.

String Quoting Utilities

Quote - Add ClassAd string escaping:

func Quote(s string) string

Converts a plain string to a properly quoted ClassAd string literal with escape sequences.

original := `Hello "World"\nNew Line`
quoted := classad.Quote(original)
// Returns: "Hello \"World\"\\nNew Line"

Unquote - Parse ClassAd string literals:

func Unquote(s string) (string, error)

Parses a ClassAd string literal and returns the unescaped string value.

quoted := `"Hello \"World\""`
unquoted, err := classad.Unquote(quoted)
// Returns: Hello "World"

These functions handle all ClassAd escape sequences including \n, \t, \", \\, etc.

Old Format Serialization

MarshalOld - Convert to old HTCondor format:

func (c *ClassAd) MarshalOld() string

Serializes a ClassAd to the old HTCondor format (newline-delimited, no brackets).

ad, _ := classad.Parse(`[
    JobId = 1001;
    Owner = "alice";
    Cpus = 4;
    Memory = 8192
]`)

old := ad.MarshalOld()
// Returns:
// JobId = 1001
// Owner = "alice"
// Cpus = 4
// Memory = 8192

Useful for backward compatibility with tools that expect the old format.

Dependency Analysis

ExternalRefs - Find undefined attribute references:

func (c *ClassAd) ExternalRefs(expr *Expr) []string

Returns a sorted list of attribute names referenced in the expression but not defined in the ClassAd.

expr, _ := classad.ParseExpr("RequestCpus * 1000 + Memory / 1024")
job := classad.New()
job.Set("RequestCpus", 4)

missing := job.ExternalRefs(expr)
// Returns: ["Memory"]

Use Cases:

• Validation: Check if all required attributes are present before evaluation
• Debugging: Identify why an expression evaluates to UNDEFINED
• Dependency Tracking: Determine what external data is needed
• Schema Validation: Verify ClassAds match expected structure

InternalRefs - Find defined attribute references:

func (c *ClassAd) InternalRefs(expr *Expr) []string

Returns a sorted list of attribute names referenced in the expression that are defined in the ClassAd.

defined := job.InternalRefs(expr)
// Returns: ["RequestCpus"]

Use Cases:

• Change Tracking: Know which attributes affect an expression
• Cache Invalidation: Invalidate cached results when dependencies change
• Selective Updates: Only recalculate when relevant attributes change
• Impact Analysis: Understand what expressions are affected by attribute changes

Example: Validation Workflow

// Parse a requirements expression
reqExpr, _ := classad.ParseExpr("Cpus >= RequestCpus && Memory >= RequestMemory && Arch == \"x86_64\"")

// Check what the job needs to provide
job := classad.New()
job.Set("RequestCpus", 4)
job.Set("RequestMemory", 2048)

// Find what's missing from the job
jobMissing := job.ExternalRefs(reqExpr)
// Returns: ["Cpus", "Memory", "Arch"]

// These must come from the machine ClassAd
machine := classad.New()
machine.Set("Cpus", 8)
machine.Set("Memory", 16384)
machine.Set("Arch", "x86_64")

// Validate machine has everything needed
machineMissing := machine.ExternalRefs(reqExpr)
// Returns: ["RequestCpus", "RequestMemory"]
// These come from the job, so we're good!

Partial Evaluation

Flatten - Optimize expressions by computing known values:

func (c *ClassAd) Flatten(expr *Expr) *Expr

Performs partial evaluation of an expression, replacing references to defined attributes with their literal values while preserving references to undefined attributes.

expr, _ := classad.ParseExpr("RequestCpus * 1000 + RequestMemory / 1024 + Unknown")
job := classad.New()
job.Set("RequestCpus", 4)
job.Set("RequestMemory", 8192)

flattened := job.Flatten(expr)
// Original:  (((RequestCpus * 1000) + (RequestMemory / 1024)) + Unknown)
// Flattened: (4008 + Unknown)

How It Works:

1. Recursively traverses the expression AST
2. Evaluates sub-expressions that reference only defined attributes
3. Replaces evaluated sub-expressions with literal values
4. Preserves unevaluated sub-expressions containing undefined references
5. Maintains semantic equivalence with the original expression

Use Cases:

• Query Optimization: Pre-compute constant parts of requirements expressions
• Reducing Computation: Avoid re-evaluating the same values repeatedly
• Simplification: Create more readable expressions by removing clutter
• Debugging: See which parts of an expression can be computed
• Caching: Store partially evaluated expressions for faster matching

Example: Query Optimization

// Job submitter creates a requirement
requirement, _ := classad.ParseExpr("Cpus >= RequestCpus && Memory >= RequestMemory")

job := classad.New()
job.Set("RequestCpus", 4)
job.Set("RequestMemory", 2048)

// Scheduler flattens the requirement once
flattened := job.Flatten(requirement)
// Flattened: ((Cpus >= 4) && (Memory >= 2048))

// Now this simpler expression can be evaluated against many machines
// without re-evaluating the job attributes each time
machines := []*classad.ClassAd{...}
for _, machine := range machines {
    result := machine.EvaluateExprWithTarget(flattened, job)
    if matches, ok := result.BoolValue(); ok && matches {
        // Schedule job on this machine
    }
}

Example: Conditional Flattening

expr, _ := classad.ParseExpr("x > 5 ? 100 : 200")
ad := classad.New()
ad.Set("x", 10)

flattened := ad.Flatten(expr)
// Returns: 100
// The entire conditional is evaluated and replaced with the result

Example: Preserves Undefined References

expr, _ := classad.ParseExpr("(x + y) * z")
ad := classad.New()
ad.Set("x", 10)
ad.Set("y", 20)
// z is undefined

flattened := ad.Flatten(expr)
// Returns: (30 * z)
// x and y are computed, but z is preserved

Limitations:

• Function calls are not evaluated (kept as-is)
• Side effects (if any existed) would not be triggered
• Expressions with errors preserve the error sub-expression
• Scoped references (MY., TARGET.) are preserved as-is

Complete Introspection Example

package main

import (
    "fmt"
    "github.com/PelicanPlatform/classad/classad"
)

func main() {
    // Create a job ClassAd
    job := classad.New()
    job.Set("RequestCpus", 4)
    job.Set("RequestMemory", 2048)

    // Parse a complex requirement
    req, _ := classad.ParseExpr(`
        (Cpus >= RequestCpus) &&
        (Memory >= RequestMemory) &&
        (Arch == "x86_64") &&
        (OpSys == "LINUX")
    `)

    // Analyze dependencies
    fmt.Println("=== Dependency Analysis ===")
    internal := job.InternalRefs(req)
    external := job.ExternalRefs(req)

    fmt.Printf("Job provides: %v\n", internal)
    // Prints: [RequestCpus RequestMemory]

    fmt.Printf("Machine must provide: %v\n", external)
    // Prints: [Arch Cpus Memory OpSys]

    // Validate we have everything needed
    if len(external) > 0 {
        fmt.Printf("Warning: Expression references undefined attributes: %v\n", external)
    }

    // Optimize the requirement
    fmt.Println("\n=== Query Optimization ===")
    flattened := job.Flatten(req)

    fmt.Printf("Original:  %s\n", req.String())
    fmt.Printf("Optimized: %s\n", flattened.String())
    // Optimized version has RequestCpus and RequestMemory replaced with 4 and 2048

    // Create machine ClassAd
    machine := classad.New()
    machine.Set("Cpus", 8)
    machine.Set("Memory", 16384)
    machine.Set("Arch", "x86_64")
    machine.Set("OpSys", "LINUX")

    // Evaluate optimized expression
    result := machine.EvaluateExprWithTarget(flattened, job)
    if matches, ok := result.BoolValue(); ok {
        fmt.Printf("\nMatch result: %v\n", matches)
    }

    // Convert to old format for compatibility
    fmt.Println("\n=== Old Format Export ===")
    fmt.Println(job.MarshalOld())
}

See examples/introspection_demo/main.go for more comprehensive examples.