User Guide - Day-to-Day Usage

This guide covers how to use the Distributed Gradle Building System effectively after initial setup.

Basic Usage

Running a Build

From your project directory on the master machine:

/path/to/Distributed-Gradle-Building/sync_and_build.sh

This will:

Sync your code to all worker machines
Run the build with optimal parallelism
Display build results

Running Specific Gradle Tasks

Pass any Gradle task as additional arguments:

# Build specific module
./sync_and_build.sh :app:assembleDebug

# Clean and build
./sync_and_build.sh clean assemble

# Run tests
./sync_and_build.sh test

# Build release bundle
./sync_and_build.sh :app:bundleRelease

Dry Run (Sync Only)

To sync code without building:

# Edit sync_and_build.sh temporarily to comment out the build command
# or create a separate sync-only script

Understanding the Build Process

Sync Phase

The rsync process excludes:

build/ - Build outputs
.gradle/ - Gradle cache and daemon files
*.iml - IntelliJ IDEA module files
.idea/ - IDE-specific files

This ensures only source code is distributed.

Build Phase

The build runs with:

--parallel - Execute tasks in parallel
--max-workers=$MAX_WORKERS - Optimal worker count
Your additional task arguments

Common Workflows

1. Development Workflow

# Make changes in your IDE
git add .
git commit -m "Feature implementation"

# Sync and build
./sync_and_build.sh assembleDebug

# Test on device/emulator
# ...

2. Release Workflow

# Ensure clean state
./sync_and_build.sh clean

# Build release
./sync_and_build.sh :app:bundleRelease

# Deploy AAB/APK
# ...

3. Testing Workflow

# Run all tests
./sync_and_build.sh test

# Run specific test class
./sync_and_build.sh test --tests "*.MyTestClass"

# Run tests with coverage
./sync_and_build.sh test jacocoTestReport

Performance Tips

1. Optimize Worker Count

The setup automatically calculates MAX_WORKERS = CPU_CORES * 2. Adjust if needed:

# Edit .gradlebuild_env
export MAX_WORKERS=12  # Custom value

2. Use Build Caching

Add to gradle.properties:

org.gradle.caching=true

3. Configure Memory

For large projects, increase JVM memory in gradle.properties:

org.gradle.jvmargs=-Xmx6g -XX:+UseG1GC

4. Exclude Large Directories

If you have large directories that don't need syncing, modify sync_and_build.sh:

rsync -a --delete -e ssh \
  --exclude='build/' --exclude='.gradle/' --exclude='*.iml' --exclude='.idea/' \
  --exclude='large-data/' --exclude='test-fixtures/' \
  "$PROJECT_DIR"/ "$ip":"$PROJECT_DIR"/

Monitoring and Debugging

Check Build Status

# Check Gradle daemon status
./gradlew --status

# View recent builds
./gradlew --continuous --info

Monitor Sync Operations

Add logging to sync_and_build.sh:

echo "$(date): Syncing to $ip"
rsync -a --delete --stats -e ssh \
  --exclude='build/' --exclude='.gradle/' --exclude='*.iml' --exclude='.idea/' \
  "$PROJECT_DIR"/ "$ip":"$PROJECT_DIR"/
echo "$(date): Sync to $ip completed"

Check Worker Status

From master, verify workers are accessible:

for ip in $WORKER_IPS; do
  echo "Checking $ip..."
  ssh $ip "df -h $PROJECT_DIR && echo 'OK'"
done

Advanced Usage

1. Selective Syncing

Create a wrapper script for specific scenarios:

#!/bin/bash
# quick_build.sh - Skip tests for faster builds
/path/to/Distributed-Gradle-Building/sync_and_build.sh assemble -x test

2. Staged Builds

For multi-stage builds:

# Stage 1: Compile only
./sync_and_build.sh compileJava compileKotlin

# Stage 2: Run tests
./sync_and_build.sh test

# Stage 3: Package
./sync_and_build.sh assemble

3. Continuous Integration

Integrate with CI/CD pipelines:

# In your CI script
export WORKER_IPS="worker1 worker2"
export MAX_WORKERS=8
export PROJECT_DIR="$CI_PROJECT_DIR"

/path/to/Distributed-Gradle-Building/sync_and_build.sh assemble test

Configuration Management

Environment-Specific Builds

Create different config files:

# For dev builds
cp .gradlebuild_env .gradlebuild_env.dev
# Edit .gradlebuild_env.dev for dev-specific settings

# For production builds
cp .gradlebuild_env .gradlebuild_env.prod
# Edit .gradlebuild_env.prod for prod settings

Create a wrapper script:

#!/bin/bash
# build_with_config.sh
ENV_TYPE=${1:-dev}
source ".gradlebuild_env.$ENV_TYPE"
/path/to/Distributed-Gradle-Building/sync_and_build.sh "${@:2}"

Branch-Specific Workers

For different branches with different resource needs:

# Edit sync_and_build.sh
case "$(git branch --show-current)" in
  "main")
    export MAX_WORKERS=16
    ;;
  "develop")
    export MAX_WORKERS=8
    ;;
  *)
    export MAX_WORKERS=4
    ;;
esac

Team Collaboration

Shared Configuration

Commit configuration templates:

# gradlebuild_env.template
export WORKER_IPS="<YOUR_WORKER_IPS>"
export MAX_WORKERS="<CALCULATED_WORKERS>"
export PROJECT_DIR="<PROJECT_PATH>"

Each developer creates their own .gradlebuild_env from template

Work-in-Progress Syncing

For sharing work between developers:

# Create a WIP sync script
#!/bin/bash
# wip_sync.sh - Sync specific branches to teammates
BRANCH=$(git branch --show-current)
for ip in TEAMMATE_IPS; do
  ssh $ip "cd $PROJECT_DIR && git fetch origin $BRANCH && git checkout $BRANCH"
done

Troubleshooting Common Issues

Sync Failures

Problem: rsync reports permission errors

# Solution: Check directory permissions
ssh worker_ip "ls -la $PROJECT_DIR"
# Fix if needed
ssh worker_ip "chmod -R u+w $PROJECT_DIR"

Problem: Sync is very slow

# Solution: Check network speed and exclude more files
# Add to rsync excludes:
--exclude='node_modules/' --exclude='npm_cache/'

Build Failures

Problem: OutOfMemoryError during build

# Solution: Increase heap size
export GRADLE_OPTS="-Xmx4g"
# Or in gradle.properties:
org.gradle.jvmargs=-Xmx4g

Problem: Worker machines not keeping up

# Solution: Reduce parallelism
export MAX_WORKERS=4
# Check CPU usage on workers
ssh worker_ip "top"

Best Practices

1. Regular Maintenance

Clean Gradle caches weekly: ./gradlew clean
Monitor disk space on workers
Update Java versions consistently

2. Version Control

Commit changes before syncing to workers
Use consistent Git versions across machines
Tag releases for reproducible builds

3. Network Optimization

Use wired connections for workers
Configure QoS to prioritize build traffic
Monitor network latency

4. Security

Use SSH keys instead of passwords
Regularly rotate SSH keys
Monitor SSH access logs

Integration with IDEs

IntelliJ IDEA

Create a custom run configuration:

Run → Edit Configurations
Add "Shell Script" configuration
Command: /path/to/sync_and_build.sh
Program arguments: assembleDebug

VS Code

Create a task in .vscode/tasks.json:

{
    "version": "2.0.0",
    "tasks": [
        {
            "label": "Distributed Build",
            "type": "shell",
            "command": "/path/to/sync_and_build.sh",
            "args": ["assembleDebug"],
            "group": "build",
            "presentation": {
                "reveal": "always",
                "panel": "new"
            }
        }
    ]
}

Performance Measurement

Time Your Builds

# Create timing script
#!/bin/bash
# timed_build.sh
START=$(date +%s)
/path/to/Distributed-Gradle-Building/sync_and_build.sh "$@"
END=$(date +%s)
echo "Build time: $((END-START)) seconds"

Compare with Local Builds

# Local build timing
START=$(date +%s)
./gradlew "$@"
END=$(date +%s)
echo "Local build time: $((END-START)) seconds"

Optimization Checklist

Emergency Procedures

Worker Failure

If a worker goes down:

Remove from worker list in .gradlebuild_env
Restart build with remaining workers
Replace worker when available

Master Failure

If master machine fails:

Promote a worker to master:
- Copy project to new master
- Update .gradlebuild_env
- Update worker configurations
Rerun setup on new master

Network Outage

For temporary network issues:

Switch to local build mode:

./gradlew assemble  # Bypass distribution

Fix network connectivity
Resume distributed builds

FilesExpand file tree

USER_GUIDE.md

Latest commit

History