diff --git a/.cognition/skills/debug-perlonjava/SKILL.md b/.cognition/skills/debug-perlonjava/SKILL.md new file mode 100644 index 000000000..e6129fc0e --- /dev/null +++ b/.cognition/skills/debug-perlonjava/SKILL.md @@ -0,0 +1,265 @@ +--- +name: debug-perlonjava +description: Debug and fix test failures and regressions in PerlOnJava +argument-hint: "[test-name, error message, or Perl construct]" +triggers: + - user + - model +--- + +# Debugging PerlOnJava + +You are debugging failures in PerlOnJava, a Perl-to-JVM compiler with a bytecode interpreter fallback. This skill covers debugging workflows for test failures, regressions, and parity issues between backends. + +## Project Layout + +- **PerlOnJava source**: `src/main/java/org/perlonjava/` (compiler, bytecode interpreter, runtime) +- **Unit tests**: `src/test/resources/unit/*.t` (156 tests, run via `mvn test`) +- **Perl5 core tests**: `perl5_t/t/` (Perl 5 compatibility suite) +- **Fat JAR**: `target/perlonjava-3.0.0.jar` +- **Launcher script**: `./jperl` + +## Building + +```bash +mvn package -q -DskipTests # Build JAR (required after any Java change) +mvn test # Run unit tests (156 tests, must all pass) +``` + +## Running Tests + +### Single Perl5 core test +```bash +cd perl5_t/t +../../jperl op/bop.t +``` + +### With environment variables (for specific tests) +```bash +# For re/pat.t and similar regex tests +JPERL_UNIMPLEMENTED=1 JPERL_OPTS=-Xss256m PERL_SKIP_BIG_MEM_TESTS=1 ./jperl perl5_t/t/re/pat.t + +# For op/sprintf2.t +JPERL_UNIMPLEMENTED=1 ./jperl perl5_t/t/op/sprintf2.t +``` + +### Test runner (parallel, with summary) +```bash +perl dev/tools/perl_test_runner.pl perl5_t/t/op +perl dev/tools/perl_test_runner.pl --jobs 8 --timeout 60 perl5_t/t +``` + +### Interpreter mode +```bash +./jperl --interpreter script.pl +./jperl --interpreter -e 'print "hello\n"' +JPERL_INTERPRETER=1 ./jperl script.pl # Global (affects require/do/eval) +``` + +## Comparing Outputs + +### PerlOnJava vs System Perl +```bash +# System Perl +perl -e 'my @a = (1,2,3); print "@a\n"' + +# PerlOnJava +./jperl -e 'my @a = (1,2,3); print "@a\n"' +``` + +### JVM backend vs Interpreter backend +```bash +./jperl -e 'code' # JVM backend +JPERL_INTERPRETER=1 ./jperl -e 'code' # Interpreter backend +``` + +### Disassembly comparison +```bash +./jperl --disassemble -e 'code' # JVM bytecode +./jperl --disassemble --interpreter -e 'code' # Interpreter bytecode +``` + +## Environment Variables + +### Compiler/Interpreter Control +| Variable | Effect | +|----------|--------| +| `JPERL_INTERPRETER=1` | Force interpreter mode globally (require/do/eval) | +| `JPERL_DISABLE_INTERPRETER_FALLBACK=1` | Disable bytecode interpreter fallback for large subs | +| `JPERL_SHOW_FALLBACK=1` | Print message when a sub falls back to interpreter | +| `JPERL_EVAL_NO_INTERPRETER=1` | Disable interpreter for `eval STRING` | +| `JPERL_OPTS="-Xss256m"` | Pass JVM options (e.g., stack size) | + +### Debugging/Tracing +| Variable | Effect | +|----------|--------| +| `JPERL_DISASSEMBLE=1` | Disassemble generated bytecode | +| `JPERL_ASM_DEBUG=1` | Print JVM bytecode when ASM frame computation crashes | +| `JPERL_EVAL_VERBOSE=1` | Verbose error reporting for eval compilation | +| `JPERL_EVAL_TRACE=1` | Trace eval STRING execution path | +| `JPERL_IO_DEBUG=1` | Trace file handle open/dup/write operations | +| `JPERL_REQUIRE_DEBUG=1` | Trace `require`/`use` module loading | + +### Perl-level +| Variable | Effect | +|----------|--------| +| `JPERL_UNIMPLEMENTED=1` | Allow unimplemented features (skip instead of die) | +| `PERL_SKIP_BIG_MEM_TESTS=1` | Skip memory-intensive tests | + +## Debugging Workflow + +### 1. Identify the regression +```bash +# Compare branch vs master +git checkout master && mvn package -q -DskipTests +./jperl -e 'failing code' + +git checkout branch && mvn package -q -DskipTests +./jperl -e 'failing code' +``` + +### 2. Bisect to find the bad commit +```bash +git log master..branch --oneline +git checkout && mvn package -q -DskipTests && ./jperl -e 'test' +``` + +### 3. Create minimal reproducer +Reduce the failing test to the smallest code that demonstrates the bug: +```bash +./jperl -e 'my $x = 58; eval q{($x) .= "z"}; print "x=$x\n"' +``` + +### 4. Compare with system Perl +```bash +perl -e 'same code' +``` + +### 5. Use disassembly to understand +```bash +./jperl --disassemble -e 'minimal code' 2>&1 | grep -i "relevant" +``` + +### 6. Add debug prints (if needed) +In Java source, add: +```java +System.err.println("DEBUG: var=" + var); +``` +Then rebuild with `mvn package -q -DskipTests`. + +### 7. Fix and verify +```bash +# After fixing +mvn package -q -DskipTests +./jperl -e 'test code' # Verify fix +mvn test # No regressions in unit tests +``` + +## Architecture: Two Backends + +``` +Source → Lexer → Parser → AST ─┬─→ JVM Compiler → JVM bytecode (default) + └─→ BytecodeCompiler → InterpretedCode → BytecodeInterpreter +``` + +Both backends share the parser (same AST) and runtime (same operators, same RuntimeScalar/Array/Hash). + +## Key Source Files + +| Area | File | Notes | +|------|------|-------| +| **Bytecode Compiler** | `backend/bytecode/BytecodeCompiler.java` | AST → interpreter bytecode | +| **Bytecode Interpreter** | `backend/bytecode/BytecodeInterpreter.java` | Main dispatch loop | +| **Assignment (interp)** | `backend/bytecode/CompileAssignment.java` | Assignment compilation | +| **Binary ops (interp)** | `backend/bytecode/CompileBinaryOperator.java` | Binary operator compilation | +| **Unary ops (interp)** | `backend/bytecode/CompileOperator.java` | Unary operator compilation | +| **Opcodes** | `backend/bytecode/Opcodes.java` | Opcode constants | +| **JVM Compiler** | `backend/jvm/EmitterMethodCreator.java` | AST → JVM bytecode | +| **JVM Subroutine** | `backend/jvm/EmitSubroutine.java` | Sub compilation (JVM) | +| **JVM Binary ops** | `backend/jvm/EmitBinaryOperator.java` | Binary ops (JVM) | +| **Compilation router** | `app/scriptengine/PerlLanguageProvider.java` | Picks backend | +| **Runtime scalar** | `runtime/runtimetypes/RuntimeScalar.java` | Scalar values | +| **Runtime array** | `runtime/runtimetypes/RuntimeArray.java` | Array values | +| **Runtime hash** | `runtime/runtimetypes/RuntimeHash.java` | Hash values | +| **Math operators** | `runtime/operators/MathOperators.java` | +, -, *, /, etc. | +| **String operators** | `runtime/operators/StringOperators.java` | ., x, etc. | +| **Bitwise operators** | `runtime/operators/BitwiseOperators.java` | &, |, ^, etc. | +| **Regex runtime** | `runtime/regex/RuntimeRegex.java` | Regex matching | +| **Regex preprocessor** | `runtime/regex/RegexPreprocessor.java` | Perl→Java regex | + +All paths relative to `src/main/java/org/perlonjava/`. + +## Common Bug Patterns + +### 1. Context not propagated correctly +**Symptom**: Operation returns wrong type (list vs scalar). +**Pattern**: Code uses `node.accept(this)` instead of `compileNode(node, -1, RuntimeContextType.SCALAR)`. +**Fix**: Use `compileNode()` helper with explicit context. + +### 2. Missing opcode implementation +**Symptom**: "Unknown opcode" or silent wrong result. +**Fix**: Add opcode to `Opcodes.java`, handler to `BytecodeInterpreter.java`, emitter to `BytecodeCompiler.java`, disassembly to `InterpretedCode.java`. + +### 3. Closure variable not accessible +**Symptom**: Variable returns undef inside eval/sub. +**Pattern**: Variable not registered in symbol table. +**Fix**: Ensure `detectClosureVariables()` registers captured variables via `addVariableWithIndex()`. + +### 4. Double compilation of RHS +**Symptom**: Side effects happen twice (e.g., `shift` removes two elements). +**Pattern**: RHS compiled once at top of function, then again in specific handler. +**Fix**: Remove redundant compilation, use `valueReg` from first compilation. + +### 5. Lvalue not preserved +**Symptom**: Assignment doesn't modify original variable. +**Pattern**: Expression returns copy instead of lvalue reference. +**Fix**: Ensure lvalue context is preserved through compilation chain. + +### 6. LIST_TO_COUNT destroys value +**Symptom**: Numeric value instead of expected string/reference. +**Pattern**: Incorrect scalar context conversion. +**Fix**: Remove spurious `LIST_TO_COUNT` or use proper scalar coercion. + +## Test File Categories + +| Directory | Tests | Notes | +|-----------|-------|-------| +| `perl5_t/t/op/` | Core operators | bop.t, sprintf.t, etc. | +| `perl5_t/t/re/` | Regex | pat.t needs special env vars | +| `perl5_t/t/io/` | I/O operations | filetest.t, etc. | +| `perl5_t/t/uni/` | Unicode | | +| `perl5_t/t/mro/` | Method resolution | | + +## Commit Message Format + +``` +Fix by + +
+ +Generated with [Devin](https://cli.devin.ai/docs) + +Co-Authored-By: Devin +``` + +## Quick Reference Commands + +```bash +# Build +mvn package -q -DskipTests + +# Test +mvn test +perl dev/tools/perl_test_runner.pl perl5_t/t/op/bop.t + +# Debug +./jperl --disassemble -e 'code' +./jperl --disassemble --interpreter -e 'code' + +# Compare +diff <(./jperl -e 'code') <(perl -e 'code') + +# Bisect +git log master..HEAD --oneline +git checkout && mvn package -q -DskipTests && ./jperl -e 'test' +```