jobctl - Waggle Edge Scheduler Job Controller

A Python package for controlling job submission and management on Waggle edge nodes.

Features

• Python API: Pythonic wrapper around sesctl for programmatic job control
• CLI: Command-line interface for quick job operations
• Auto-download: Automatically downloads sesctl binary for your platform
• Time-limited jobs: Run jobs for a specified duration, then auto-stop
• Secure: Token only read from environment variable, never stored in files

Quick Start

Step 1: Install

# Clone the repository
git clone https://github.com/waggle-sensor/jobctl.git
cd jobctl

# Install the package
pip install -e .

Or install directly from GitHub:

pip install git+https://github.com/waggle-sensor/jobctl.git

Step 2: Initialize (Download sesctl)

jobctl init

This downloads the sesctl binary and creates a config template at ~/.jobctl.yaml.

Step 3: Set Environment Variables

Required: Set your authentication token (get from portal.sagecontinuum.org):

export SES_USER_TOKEN=<your_token>

Optional: For Mobotix camera jobs:

export SES_MOBO_PASSWORD=<camera_password>

Add these to your ~/.bashrc or ~/.zshrc for persistence.

Step 4: Configure (~/.jobctl.yaml)

Edit ~/.jobctl.yaml with your settings:

# Waggle Edge Scheduler server URL
host: https://es.sagecontinuum.org

# Your pre-existing job ID (create one via Sage UI first)
job_id: "77"

# Default target nodes
default_nodes:
  - W09A

# Mobotix camera defaults (optional)
camera_ip: "130.202.23.119"
camera_username: admin
camera_password: ${SES_MOBO_PASSWORD}

Step 5: Submit a Job

Using CLI:

# Check status first
jobctl status

# Submit Mobotix camera job (runs until stopped)
jobctl mobotix --directions NEH,NEB,NEG --ip 130.202.23.119

# Submit with auto-stop after 30 minutes
jobctl mobotix --directions NEH,NEB,NEG --ip 130.202.23.119 --duration 30m

# Stop the job
jobctl stop

Using Python API:

from jobctl import JobClient, mobotix_scan_job
import os

# Create client (reads config from ~/.jobctl.yaml, token from env)
client = JobClient()

# Create a Mobotix scan job
job = mobotix_scan_job(
    directions=["NEH", "NEB", "NEG"],
    nodes=["W09A"],
    ip="130.202.23.119",
    username="admin",
    password=os.environ["SES_MOBO_PASSWORD"],
    south="15"
)

# Run for 5 minutes, then auto-stop (blocking)
client.run_for(job, duration_seconds=300)

---

How It Works

Key Concept: jobctl always edits a pre-existing job ID rather than creating new jobs.

1. You create a job once via the Sage UI or sesctl create
2. You provide this job ID in ~/.jobctl.yaml
3. jobctl edits this job with new content each time you submit
4. The workflow is: edit → submit → suspend

---

CLI Reference

jobctl - Waggle Edge Scheduler Job Controller

Commands:
  init      Download sesctl and create config template
  submit    Submit a job from YAML file
  stop      Suspend the configured job
  status    Show status of the configured job
  mobotix   Create and submit a Mobotix camera job
  version   Show version information

Global Options:
  --job-id, -j    Override configured job ID
  --config, -c    Config file path
  --verbose, -v   Verbose output

CLI Examples

# Initialize (first time setup)
jobctl init

# Check job status
jobctl status

# Submit Mobotix job
jobctl mobotix --directions NEH,NEB,NEG --ip 130.202.23.119

# Submit with time limit (auto-stops after duration)
jobctl mobotix --directions NEH,NEB --ip 130.202.23.119 --duration 30m

# Submit from YAML file
jobctl submit my_job.yaml
jobctl submit my_job.yaml --dry-run
jobctl submit my_job.yaml --duration 1h

# Stop job
jobctl stop

# Override job ID for any command
jobctl status --job-id 123

---

Python API Reference

JobClient

from jobctl import JobClient, mobotix_scan_job
import os

client = JobClient()

# Create a Mobotix job
job = mobotix_scan_job(
    directions=["NEH", "NEB", "NEG"],
    nodes=["W09A"],
    ip="130.202.23.119",
    username="admin",
    password=os.environ["SES_MOBO_PASSWORD"],
    south="15"
)

# Method 1: Run for fixed duration (RECOMMENDED for notebooks)
# Blocks until duration is complete, then auto-stops
client.run_for(job, duration_seconds=300)  # 5 minutes
client.run_for(job)  # Default: 15 minutes

# Method 2: Submit and manually stop later
client.submit(job)
# ... do other things ...
client.stop()

# Check status anytime
status = client.status()
print(status.stdout)

Custom Jobs

from jobctl import Job, PluginSpec, cron_rule

job = Job(
    name="my-custom-job",
    nodes={"W09A": True, "V032": True},
    plugins=[
        PluginSpec(
            image="registry.sagecontinuum.org/user/plugin:tag",
            args=["--arg1", "value1"],
            env={"DEBUG": "true"}
        )
    ],
    science_rules=[cron_rule("my-custom-job", "*/5 * * * *")]
)

client.run_for(job, duration_seconds=600)

---

Configuration

Environment Variables

Variable	Required	Description
SES_USER_TOKEN	Yes	Authentication token (from Sage portal)
SES_MOBO_PASSWORD	For Mobotix	Camera password
SES_HOST	No	Override scheduler URL
JOBCTL_JOB_ID	No	Override job ID

Security Note: SES_USER_TOKEN is only read from environment variable, never from config files.

Config File (~/.jobctl.yaml)

# Waggle Edge Scheduler server URL
host: https://es.sagecontinuum.org

# Pre-existing job ID to edit
job_id: "77"

# Default target nodes
default_nodes:
  - W09A

# Directory to save generated job YAML files
job_path: ./jobs

# Mobotix camera defaults
camera_ip: "130.202.23.119"
camera_username: admin
camera_password: ${SES_MOBO_PASSWORD}  # References env var

Config Priority

1. Environment variables (highest priority)
2. Config file (~/.jobctl.yaml)
3. Default values (lowest priority)

---

Troubleshooting

"SES_USER_TOKEN environment variable not set"

Set your token:

export SES_USER_TOKEN=<your_token>

Get your token from portal.sagecontinuum.org.

"sesctl not found"

Run jobctl init to download the sesctl binary.

"Invalid or expired token"

Your token may have expired. Get a new one from the Sage portal.

"Camera password not set"

For Mobotix jobs, set:

export SES_MOBO_PASSWORD=<password>

Job name validation error

Job names must use only lowercase letters, numbers, and hyphens. No underscores or special characters.

---

Examples

See the examples/ and notebooks/ directories:

• examples/jobctl_tutorial.ipynb - Complete tutorial notebook
• notebooks/mobotix_test.ipynb - Quick Mobotix test

---

Links

• Waggle Edge Scheduler
• Sage Portal
• sesctl Documentation

License

BSD 3-Clause License

Disclaimer

Created with the help of AI agentic coding models, tested by Bhupendra Raut.