Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
121 changes: 78 additions & 43 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,37 +50,62 @@ Total: 60 tests
Your suite is getting heavier.
```

Pyrameter is a PHPUnit extension that reports the shape of your test suite after PHPUnit runs. It classifies each executed test as `unit`, `functional`, `integration`, or `e2e`, compares the totals with your target shape, and can fail CI when the suite drifts too far.

It works from the classes and namespaces used by your test files, so you can define what counts as "heavy" in your project instead of relying on test directory names.
Pyrameter classifies executed tests as `unit`, `functional`, `integration`, or `e2e` based on the code they use, then compares the totals with your target shape.

## Quick start

Install Pyrameter as a dev dependency:
1. Install with Composer:

```bash
composer require --dev boundwize/pyrameter
```

Register the extension in `phpunit.xml`:
2. Register the extension in `phpunit.xml`:

```xml
<extensions>
<bootstrap class="Boundwize\Pyrameter\Extension"/>
</extensions>
```

Run PHPUnit:
3. Run PHPUnit as usual:

```bash
vendor/bin/phpunit
```

Without extra configuration, Pyrameter uses its default rules and target shape. If a `pyrameter.php` file exists in the current working directory, it is loaded automatically.
This uses the default rules and target shape.

## Configure

Create `pyrameter.php` when you want to tune classification rules, target percentages, or CI behavior:
### Default or empty configuration

Choose the starting point before adding rules:

| Start with | Behavior |
| --- | --- |
| `PyrameterConfig::defaults()` | Starts with built-in rules and the default target shape. The rules cover common database, cache, and filesystem usage; Symfony and CodeIgniter functional tests; and Panther and WebDriver browser tests. Classification methods add rules; `targetShape()` replaces the targets. |
| `PyrameterConfig::create()` | Starts with no rules or targets. Only rules you add can classify tests as heavier than `unit`. |

Extend the built-in configuration:

```php
return PyrameterConfig::defaults()
->usesClass(App\Search\ExternalSearch::class, TestKind::Integration);
```

Define the complete configuration yourself:

```php
return PyrameterConfig::create()
->usesClass(PDO::class, TestKind::Integration)
->targetShape(
unit: ['min' => 80],
integration: ['max' => 20],
);
```

A complete `pyrameter.php` can combine rules, targets, and CI behavior:

```php
<?php
Expand All @@ -99,42 +124,52 @@ return PyrameterConfig::defaults()
functional: ['max' => 15],
integration: ['max' => 7],
e2e: ['max' => 2],
);
)
->failOnViolation();
```

`PyrameterConfig::defaults()` includes rules for common database usage (including CodeIgniter database tests),
cache, filesystem, Symfony and CodeIgniter controller functional tests, Panther, and WebDriver usage.
CodeIgniter tests using both `ControllerTestTrait` and `DatabaseTestTrait` remain functional; database-only tests
are integration tests.
Rules can match a class or trait, a namespace prefix, or a function:

Use `PyrameterConfig::create()` instead when you want to start with no rules or targets and define everything yourself:
| Rule | Example |
| --- | --- |
| `usesClass()` | `->usesClass(PDO::class, TestKind::Integration)` |
| `usesNamespace()` | `->usesNamespace('App\Tests\Browser\\', TestKind::E2E)` |
| `usesFunction()` | `->usesFunction('file_put_contents', TestKind::Integration)` |

```php
<?php
### Rule exceptions

declare(strict_types=1);
Use `unless` to ignore a rule when the test also consumes another class or trait:

use Boundwize\Pyrameter\Config\PyrameterConfig;
use Boundwize\Pyrameter\TestKind;
```php
use App\Tests\Concerns\InteractsWithDatabase;
use App\Tests\Concerns\MakesHttpRequests;

return PyrameterConfig::create()
->usesClass(PDO::class, TestKind::Integration)
->usesNamespace('App\Controller\\', TestKind::Functional)
->usesNamespace('App\Browser\\', TestKind::E2E)
->usesFunction('file_put_contents', TestKind::Integration)
->targetShape(
unit: ['min' => 70],
functional: ['max' => 20],
integration: ['max' => 8],
e2e: ['max' => 2],
);
->usesClass(
InteractsWithDatabase::class,
TestKind::Integration,
unless: [MakesHttpRequests::class],
)
->usesClass(MakesHttpRequests::class, TestKind::Functional);
```

With `create()`, only the rules you add are used for heavier classifications; all other executed tests stay `unit`.
| Traits used by the test | Result |
| --- | --- |
| `InteractsWithDatabase` | `integration` |
| `MakesHttpRequests` | `functional` |
| Both traits | `functional` |

The optional `unless` argument is also available on `usesNamespace()` and `usesFunction()`.

The equivalent CodeIgniter exception is already included in `defaults()`:

Use `usesClass()` for a specific class, `usesNamespace()` for a namespace prefix, and `usesFunction()` for a function call that should classify a test as a heavier kind. Names can be configured with or without a leading backslash.
```php
return PyrameterConfig::defaults();
```

Tests that use only `DatabaseTestTrait` are classified as `integration`; tests that also use `ControllerTestTrait` remain `functional`.

To load a config file from another path, pass the `config` parameter to the PHPUnit extension:
Load a configuration file from another path:

```xml
<extensions>
Expand All @@ -146,28 +181,28 @@ To load a config file from another path, pass the `config` parameter to the PHPU

## Classification

Pyrameter scans the consumed classes, namespaces, and function calls in each test file:

| Usage | Kind |
| --- | --- |
| No configured heavy usage | `unit` |
| No matching heavier rule | `unit` |
| Framework test runtime | `functional` |
| Database, cache, queue, filesystem, or external boundary | `integration` |
| Browser driver usage | `e2e` |

When multiple rules match, the heaviest kind wins. Mocked heavy dependencies stay `unit` unless the test file also consumes a class or namespace you configured as heavier.

Counts follow PHPUnit's executed test count, so data-provider datasets are counted separately.
- The heaviest matching rule wins.
- Mocked dependencies do not trigger a rule by themselves.
- Data-provider datasets are counted separately.

## Targets and CI

Targets are percentages. Missing `min` means `0`; missing `max` means `100`.

By default, Pyrameter is report-only. To fail PHPUnit when the target shape is violated, enable `failOnViolation()`:

```php
return PyrameterConfig::defaults()
->targetShape(
unit: ['min' => 70],
functional: ['max' => 20],
integration: ['max' => 8],
e2e: ['max' => 2],
)
->failOnViolation();
```

Pyrameter is a pressure gauge for suite shape, not a perfect taxonomy judge. Tune the rules to match how your team defines unit, functional, integration, and e2e tests.
Targets are percentages. An omitted `min` defaults to `0`; an omitted `max` defaults to `100`. Without `failOnViolation()`, Pyrameter only reports violations.
39 changes: 27 additions & 12 deletions src/Config/PyrameterConfig.php
Original file line number Diff line number Diff line change
Expand Up @@ -111,15 +111,14 @@ public static function defaults(): self
->usesNamespace('Predis\\', TestKind::Integration)
->usesNamespace('Symfony\Bundle\FrameworkBundle\Test\\', TestKind::Functional)
->usesClass(ControllerTestTrait::class, TestKind::Functional)
->usesClass(
DatabaseTestTrait::class,
TestKind::Integration,
unless: [ControllerTestTrait::class],
)
->usesNamespace('Symfony\Component\Panther\\', TestKind::E2E)
->usesNamespace('Facebook\WebDriver\\', TestKind::E2E);

$pyrameterConfig->usageRules[] = new UsageRule(
DatabaseTestTrait::class,
TestKind::Integration,
unless: [ControllerTestTrait::class],
);

foreach (self::FILE_OPERATION_FUNCTIONS as $functionName) {
$pyrameterConfig->usesFunction($functionName, TestKind::Integration);
}
Expand All @@ -134,24 +133,40 @@ functional: ['max' => 18],

/**
* @param class-string $className
* @param list<string> $unless
*/
public function usesClass(string $className, TestKind $testKind): self
public function usesClass(string $className, TestKind $testKind, array $unless = []): self
{
$this->usageRules[] = new UsageRule(ltrim($className, '\\'), $testKind);
$this->usageRules[] = new UsageRule(ltrim($className, '\\'), $testKind, unless: $unless);

return $this;
}

public function usesNamespace(string $namespace, TestKind $testKind): self
/**
* @param list<string> $unless
*/
public function usesNamespace(string $namespace, TestKind $testKind, array $unless = []): self
{
$this->usageRules[] = new UsageRule(rtrim(ltrim($namespace, '\\'), '\\') . '\\', $testKind);
$this->usageRules[] = new UsageRule(
rtrim(ltrim($namespace, '\\'), '\\') . '\\',
$testKind,
unless: $unless,
);

return $this;
}

public function usesFunction(string $functionName, TestKind $testKind): self
/**
* @param list<string> $unless
*/
public function usesFunction(string $functionName, TestKind $testKind, array $unless = []): self
{
$this->usageRules[] = new UsageRule(ltrim($functionName, '\\'), $testKind, UsageType::Function);
$this->usageRules[] = new UsageRule(
ltrim($functionName, '\\'),
$testKind,
usageType: UsageType::Function,
unless: $unless,
);

return $this;
}
Expand Down
18 changes: 18 additions & 0 deletions tests/Config/PyrameterConfigTest.php
Original file line number Diff line number Diff line change
Expand Up @@ -95,4 +95,22 @@ public function testUsesFunctionMatchesFunctionUsageCaseInsensitively(): void

$this->assertSame(TestKind::Integration, $usageClassifier->classify(['function:\FILE_GET_CONTENTS']));
}

public function testUsesClassCanBeSuppressedByAnotherConsumedClass(): void
{
$pyrameterConfig = PyrameterConfig::create()
->usesClass(
PDO::class,
TestKind::Integration,
unless: [self::class],
)
->usesClass(self::class, TestKind::Functional);
$usageClassifier = new UsageClassifier($pyrameterConfig->usageRules());

$this->assertSame(
TestKind::Functional,
$usageClassifier->classify([PDO::class, self::class]),
);
$this->assertSame(TestKind::Integration, $usageClassifier->classify([PDO::class]));
}
}
Loading