Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 31 additions & 0 deletions .github/workflows/backend-tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
name: Backend tests

on:
pull_request:
push:
branches: [master]
workflow_dispatch:

permissions:
contents: read

jobs:
test:
runs-on: ubuntu-latest
container: maven:3.9-eclipse-temurin-21
services:
postgres:
image: postgres:14
env:
POSTGRES_DB: platform_test
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
options: >-
--health-cmd pg_isready
--health-interval 5s
--health-timeout 5s
--health-retries 10
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Unit, HTTP contract, migration, and persistence tests
run: mvn -B -ntp -Ppostgres-integration -Dit.database.url=jdbc:postgresql://postgres:5432/platform_test clean verify
14 changes: 7 additions & 7 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,9 @@ This repository contains the Spring Boot backend for the Medical Informatics Pla
- `src/main/java/hbp/mip/configurations`: security, OAuth2/JWT, persistence, OpenAPI, and redirect/filter configuration.
- `src/main/java/hbp/mip/algorithm`: algorithm metadata API/service and Exaflow specification passthrough.
- `src/main/java/hbp/mip/datamodel`: data model API/service backed by Exaflow metadata endpoints.
- `src/main/java/hbp/mip/experiment`: experiment API, service, repository, JPA entity, specifications, and DTOs.
- `src/main/java/hbp/mip/analysis`: analysis API, service, repository, JPA entity, specifications, and DTOs.
- `src/main/java/hbp/mip/user`: active user API/service, user repository, JPA entity, and DTO.
- `src/main/java/hbp/mip/folder`: experiment folder/set API, service, repository, JPA entities, and DTOs.
- `src/main/java/hbp/mip/study`: Study/Analysis Group API, service, repository, JPA entities, and DTOs.
- `src/main/java/hbp/mip/utils`: shared logging, JSON/HTTP helpers, claim validation, and exception handling.
- `src/main/resources`: local runtime config, Log4j2 config, and Flyway migrations under `db/migration`.
- `.github/workflows`: release image publishing and EBRAINS mirror automation.
Expand Down Expand Up @@ -55,7 +55,7 @@ The Maven build produces `target/platform-backend.jar`.
mvn test
```

There is currently no committed `src/test/java` tree. Treat `mvn test` as the baseline smoke check until focused tests are added.
`mvn test` runs JUnit/Mockito and MockMvc tests. Run `mvn -Ppostgres-integration verify` against a disposable PostgreSQL database for migration and persistence coverage; connection properties are documented in `docs/context/testing.md`.

## Lint / Format / Typecheck
```bash
Expand All @@ -74,7 +74,7 @@ mvn clean package
## Architecture Rules
- Put HTTP endpoints in `*API` classes under the owning feature package.
- Put business logic in `*Service` classes; controllers should delegate rather than implement workflows directly.
- Put persistence in Spring Data repositories and JPA `*DAO` entities. Current JPA packages are `hbp.mip.experiment`, `hbp.mip.user`, and `hbp.mip.folder` (list them in `PersistenceConfiguration` when adding another).
- Put persistence in Spring Data repositories and JPA `*DAO` entities. Current JPA packages are `hbp.mip.analysis`, `hbp.mip.user`, and `hbp.mip.study` (list them in `PersistenceConfiguration` when adding another).
- Put API/request/response shapes in `*DTO` records or DTO classes close to the feature package.
- Put cross-cutting helpers in `hbp.mip.utils` only when they are genuinely shared.
- Put security, persistence, OpenAPI, and web filter wiring in `hbp.mip.configurations`.
Expand Down Expand Up @@ -113,10 +113,10 @@ mvn clean package
## Security and Privacy Rules
- Never commit secrets, tokens, keys, cookies, credentials, or private environment values.
- Never print secrets in logs or final reports.
- Treat auth, authorization claims, CSRF, token exposure, migrations, data deletion, and experiment execution as human-review areas.
- Treat auth, authorization claims, CSRF, token exposure, migrations, data deletion, and analysis execution as human-review areas.
- Preserve OAuth2 login and bearer-token behavior unless the task explicitly changes auth.
- Preserve dataset and experiment access checks in `ClaimUtils` and `ExperimentService`.
- Review logs carefully when touching experiment execution, active user token handling, or external service calls.
- Preserve dataset and analysis access checks in `ClaimUtils` and `AnalysisService`.
- Review logs carefully when touching analysis execution, active user token handling, or external service calls.

## PR Expectations
Every agent change should include:
Expand Down
7 changes: 5 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ To use this image, you need a running instance of PostgreSQL and to configure th

#### AUTHENTICATION ###
* AUTHENTICATION: true for production, false for development.
* `authentication.all_analyses_allowed_claim`: defaults to the existing external authority `research_experiment_all`. Preserve that authority and existing grants.

#### DATABASE CONFIGURATION ###
* PLATFORM_DB_URL: JDBC URL to connect to the platform database, default value is "jdbc:postgresql://127.0.0.1:5432/platform".
Expand All @@ -27,8 +28,10 @@ To use this image, you need a running instance of PostgreSQL and to configure th
#### EXTERNAL SERVICES ###
* EXAFLOW_URL: URL to Exaflow server. Default is "http://localhost:5000" .

#### EXPERIMENTS ###
* EXPERIMENT_RUN_DRAFT_TTL: How long an unsaved studio run can still be saved without re-running it, as an ISO-8601 duration. Default is "PT24H".
#### ANALYSES ###
* ANALYSIS_RUN_DRAFT_TTL: How long an unsaved studio run can still be saved without re-running it, as an ISO-8601 duration. Default is "PT24H".

The backend uses Analysis, Study, and Analysis Group terminology. See the [module index](docs/context/module-index.md) for API and persistence details. Run `mvn test`, `mvn -Ppostgres-integration verify` against a disposable PostgreSQL database, and `mvn clean package`; connection settings are in [the testing guide](docs/context/testing.md).

#### KEYCLOAK ###
* KEYCLOAK_AUTH_URL: Keycloak authentication URL.
Expand Down
28 changes: 14 additions & 14 deletions docs/context/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,10 @@ This is a backend service for the Medical Informatics Platform. It is a Java 21
## High-Level Structure
The application entrypoint is `hbp.mip.MIPApplication`. APIs are grouped by feature package:

- `algorithm`: runs analyses through Exaflow (for the `experiment` package) and exposes Exaflow's inputdata, preprocessing and algorithm specifications (`/specifications/*`).
- `algorithm`: runs analyses through Exaflow (for the `analysis` package) and exposes Exaflow's inputdata, preprocessing and algorithm specifications (`/specifications/*`).
- `datamodel`: exposes data models and dataset metadata from Exaflow, filtered by user claims when authentication is enabled.
- `experiment`: creates, lists, updates, deletes, and executes experiments; persists experiment records in PostgreSQL.
- `analysis`: manages platform Analysis records and the preview/run/save workflow.
- `study`: owns Studies, Analysis Groups, and ordered Analysis memberships.
- `user`: resolves the active user from OAuth2/OIDC or JWT authentication and persists user profile/NDA state.
- `configurations`: configures security, OAuth2/OIDC, JWT authority mapping, persistence, Flyway, OpenAPI, and SPA redirects.
- `utils`: shared HTTP, JSON, logging, claim, resource, and exception utilities.
Expand All @@ -28,7 +29,7 @@ flowchart TD
- HTTP requests enter controllers named `*API`.
- Controllers create a `hbp.mip.utils.Logger` with active user and endpoint context, then delegate to services.
- Services enforce business rules, authorization checks, external calls, and persistence workflows.
- Repositories and JPA `*DAO` entities handle database access for users and experiments.
- Repositories and JPA `*DAO` entities handle database access for users and analyses.
- DTOs are returned from controllers as API response shapes.
- Exceptions are mapped by `ControllerExceptionHandler`.

Expand All @@ -47,17 +48,17 @@ Persistence is configured in `PersistenceConfiguration`.

- Data source properties come from `spring.datasource`.
- Flyway runs at startup through a `flyway` bean.
- JPA scans `hbp.mip.experiment` and `hbp.mip.user`.
- Existing migration `V1__InitialSchema.sql` creates `user` and `experiment` tables.
- `V3__ExperimentRunDrafts.sql` creates `experiment_run`, which keeps unsaved studio run results until they are saved or expire.
- Hibernate is configured with `ddl-auto: validate`, so schema changes should be made through Flyway migrations.
- JPA scans `hbp.mip.analysis`, `hbp.mip.study`, and `hbp.mip.user`.
- Shipped migrations V1–V5 create the historical schema. V6 renames records to `analysis`, `analysis_run`, `study`, `analysis_group`, and `study_member`, with complete requests in `analysis_request`.
- Flyway owns schema changes. PostgreSQL integration tests explicitly validate JPA mappings against the migrated schema.

## API Layer
The servlet context path is `/services`. Main endpoints include:

- `/algorithms`
- `/specifications/{inputdata,preprocessing,algorithms}`
- `/data-models`
- `/experiments` (`/experiments/preview` computes without storing, `/experiments/runs` computes and keeps the result, `POST /experiments` saves a kept run)
- `/analyses` (`/analyses/preview` computes without storing, `/analyses/runs` computes and keeps the result, `POST /analyses` saves a kept run)
- `/studies` (owner-scoped Study, Analysis Group, and membership operations)
- `/activeUser`

OpenAPI is configured by `OpenApiConfig` and springdoc dependencies.
Expand All @@ -72,21 +73,20 @@ OpenAPI is configured by `OpenApiConfig` and springdoc dependencies.
- Raw authority mapping for claims and Keycloak roles, intentionally without adding a `ROLE_` prefix.
- Anonymous development behavior when `authentication.enabled` is disabled.

Authorization-sensitive logic appears in `ClaimUtils`, `ExperimentService`, `DataModelService`, and active user handling.
Authorization-sensitive logic appears in `ClaimUtils`, `AnalysisService`, `DataModelService`, and active user handling.

## Background Work and External Services
## Execution and External Services
- `SpecificationsService` fetches inputdata, preprocessing, and algorithm specifications from Exaflow per request.
- `ExperimentService` starts a background `Thread` for persisted experiment execution.
- `AnalysisService` executes preview/run synchronously through `AnalysisExecutionService`. Preview stores nothing; successful runs retain owner-only drafts. Save copies a retained result without execution.
- Data model and algorithm metadata are fetched through `HTTPUtil` using configured Exaflow URLs.

## Config and Environment Model
- All runtime config is in `src/main/resources/application.yml` with `${ENV:default}` placeholders.
- Containers pass environment variables (see `mip/deployment`); no separate config template is rendered at startup.
- Important config areas include datasource, OAuth2/Keycloak, authentication flags, Exaflow URLs, frontend base URL, `MIP_VERSION`, `EXPERIMENT_RUN_DRAFT_TTL`, and logging.
- Important config areas include datasource, OAuth2/Keycloak, authentication flags, Exaflow URLs, frontend base URL, `MIP_VERSION`, `ANALYSIS_RUN_DRAFT_TTL`, and logging.
- Do not introduce direct environment reads in business logic; route configuration through Spring properties.

## Known Gaps
- No committed `src/test/java` tree.
- No Maven wrapper.
- No Docker Compose or local database bootstrap file found.
- No dedicated Java lint/formatter/static-analysis command found.
Expand Down
2 changes: 1 addition & 1 deletion docs/context/code-review-checklist.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
- [ ] No secrets are committed.
- [ ] Sensitive values are not logged.
- [ ] Auth/authz behavior is preserved or explicitly reviewed.
- [ ] Dataset and experiment access checks are preserved.
- [ ] Dataset and analysis access checks are preserved.
- [ ] Token exposure and CSRF behavior are reviewed if touched.

## Maintainability
Expand Down
8 changes: 4 additions & 4 deletions docs/context/commands.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ target/platform-backend.jar
mvn test
```

There is no committed `src/test/java` tree. This command currently functions as the baseline Maven test phase/smoke check.
Runs JUnit/Mockito and HTTP contract tests. PostgreSQL integration tests require `mvn -Ppostgres-integration verify`; see [testing.md](testing.md) for connection properties.

## Lint
```bash
Expand Down Expand Up @@ -70,9 +70,9 @@ Flyway migrations live in:
src/main/resources/db/migration
```

The application configures Flyway startup migration through `PersistenceConfiguration`. Hibernate uses `ddl-auto: validate`, so schema changes should be made through Flyway migrations.
The application configures Flyway startup migration through `PersistenceConfiguration`. Flyway owns schema changes; PostgreSQL integration tests explicitly validate Hibernate mappings.

Unknown / TODO: verify the preferred command for applying migrations against a local PostgreSQL instance outside app startup.
The `postgres-integration` Maven profile applies migrations to isolated test schemas, tests fresh and populated upgrades, and removes those schemas afterwards.

## Docker
Build the container image:
Expand All @@ -89,4 +89,4 @@ Detected workflows:
- `.github/workflows/publish_images.yml`: on published release, builds and pushes Docker images to Docker Hub and EBRAINS Harbor.
- `.github/workflows/ebrains.yml`: mirrors `master` and tags to EBRAINS GitLab on push.

No regular pull-request test workflow was found.
`backend-tests.yml` runs the package build, unit/HTTP tests, and PostgreSQL migration/persistence tests on pull requests and master pushes.
10 changes: 5 additions & 5 deletions docs/context/conventions.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
- Repositories use the `*Repository` suffix.
- JPA entities use the `*DAO` suffix.
- API/request/response shapes use the `*DTO` suffix, often as Java records.
- Feature modules are grouped by package: `algorithm`, `datamodel`, `experiment`, and `user`.
- Feature modules are grouped by package: `algorithm`, `datamodel`, `analysis`, `study`, and `user`.

### Backend Layering
- Controllers accept HTTP input, create user/action logs, and delegate to services.
Expand All @@ -26,7 +26,7 @@
### Logging
- User-facing actions use `hbp.mip.utils.Logger`.
- Logs include username, endpoint, and message.
- Existing code logs some request and experiment details; avoid expanding sensitive logging.
- Existing code logs some request and analysis details; avoid expanding sensitive logging.

### Configuration
- Spring properties are defined in `src/main/resources/application.yml`.
Expand All @@ -42,7 +42,7 @@
### Persistence
- Flyway owns migrations.
- Hibernate validates schema with `ddl-auto: validate`.
- Current JPA repositories/entities are in `experiment` and `user`.
- Current JPA repositories/entities are in `analysis`, `study`, and `user`.

### Agent Tool and Token Budget
- Agents should use focused repository inspection commands and avoid dumping broad context.
Expand All @@ -51,8 +51,8 @@
- When large context is unavoidable, the agent should ask first, explain the expected size and reason, and summarize results instead of pasting raw output.

## Recommended Conventions to Verify
- Add a test dependency such as Spring Boot test support before creating substantive Java tests, if not already added by a future change.
- Use existing Spring Boot test support, JUnit, Mockito, and MockMvc. Use real PostgreSQL for schema and SQL-function checks.
- Prefer constructor-bound configuration properties for new configuration-heavy code, but match local `@Value` style for small scoped changes.
- Prefer managed Spring executors for new background work instead of raw `Thread`, unless intentionally preserving existing behavior.
- Preserve synchronous preview/run/save. A terminology rename must not introduce background execution.
- Prefer a Spring HTTP client abstraction for larger external integration changes; keep small fixes consistent with `HTTPUtil`.
- Add a Java formatter/static-analysis command before enforcing style mechanically.
Loading
Loading