feat: migrate from Granite 3 to Granite 4 hybrid models #357

planetf1 · 2026-01-26T12:05:48Z

Migrate from Granite 3.x to Granite 4.0 Models

Type of PR

Bug Fix
New Feature
Documentation
Other

Description

Link to Issue: Closes feat: Update tests & examples to use granite4 #344

Summary

This PR migrates Mellea from Granite 3.x to Granite 4.0 hybrid models across all backends, tests, and documentation. Note: HuggingFace tests remain on Granite 3.3 due to adapter availability constraints (see below).

Changes

Model Definitions (`mellea/backends/model_ids.py`)

Added Granite 4 hybrid model identifiers:
- IBM_GRANITE_4_HYBRID_MICRO (granite-4.0-h-micro)
- IBM_GRANITE_4_HYBRID_TINY (granite-4.0-h-tiny)
- IBM_GRANITE_4_HYBRID_SMALL (granite-4-h-small)
Restored IBM_GRANITE_4_MICRO_3B with per-backend model selection (Ollama: MICRO, Watsonx: SMALL)
Marked Granite 3 models as deprecated (kept for backward compatibility)
Added vision model: IBM_GRANITE_3_3_VISION_2B

Backend Updates

WatsonxAIBackend: Default model → IBM_GRANITE_4_HYBRID_SMALL
All other backends: Use Granite 4 hybrid models in tests

Test Updates (19 files)

✅ Migrated to Granite 4:

test/backends/test_watsonx.py
test/backends/test_ollama.py
test/backends/test_litellm_*.py (3 files)
test/backends/test_vllm*.py (2 files)
test/stdlib/components/*.py (8 files)
test/stdlib/requirements/*.py (3 files)

⚠️ Remains on Granite 3.3:

test/backends/test_huggingface.py - See "HuggingFace Test Exception" below

⚠️ Remains on Granite 3.2:

test/backends/test_vision_ollama.py - See "Vision Model Exception" below

Documentation Updates

docs/tutorial.md: Updated all examples to Granite 4
docs/alora.md: Updated training examples, added note about non-hybrid models for adapter training
docs/examples/*.py: Updated all example scripts

Test Infrastructure

Removed 48GB memory markers (Granite 4 micro models require ~16GB)
Fixed CI memory constraints by using MICRO models for Ollama tests
Restored per-backend model selection for IBM_GRANITE_4_MICRO_3B (matches upstream pattern)

HuggingFace Test Exception

HuggingFace tests remain on Granite 3.3 due to missing aLoRA adapters for Granite 4.

The HF tests require the requirement_check intrinsic adapter, which is only available for Granite 3.x models in ibm-granite/rag-intrinsics-lib. While ibm-granite/granite-lib-rag-r1.0 has Granite 4 support for RAG intrinsics (answerability, context_relevance, etc.), the core intrinsics needed for tests are not yet available.

Follow-up Issue: #359 tracks migration once Granite 4 adapters are released.

Vision Model Exception

Vision tests remain on granite3.2-vision due to Ollama compatibility issues.

The ibm/granite3.3-vision:2b model causes Ollama server crashes with segmentation fault (null pointer dereference in llama runner). Reverted to granite3.2-vision which works reliably.

Follow-up Issue: #360 documents the crash with full stack traces and debugging information.

Testing

Tests added to the respective file if code was changed
New code has 100% coverage if code as added
Ensure existing tests and github automation passes (a maintainer will kick off the github automation when the rest of the PR is populated)

Local Testing

# Fast tests (skip LLM quality checks)
uv run pytest -m "not qualitative"

# Full test suite
uv run pytest

Test Results: 204 passed, 6 skipped, 69 deselected, 1 xpassed

CI Testing

All tests pass in CI with CICD=1 (skips qualitative markers).

Related Issues

Closes feat: Update tests & examples to use granite4 #344 (Granite 4 migration)
Follow-up Migrate HuggingFace Tests to Granite 4 When Adapters Available #359 (HuggingFace Granite 4 adapter migration)
Follow-up Investigate granite-vision-3.3-2b Ollama Compatibility Issue #360 (Vision model Ollama crash investigation)

github-actions · 2026-01-26T12:06:01Z

The PR description has been updated. Please fill out the template for your PR to be reviewed.

mergify · 2026-01-26T12:06:25Z

Merge Protections

Your pull request matches the following merge protections and will not be merged until they are valid.

🟢 Enforce conventional commit

Wonderful, this rule succeeded.

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert|release)(?:\(.+\))?:

planetf1 · 2026-01-26T12:47:23Z

Issue with HuggingFace tests

I moved to using granite v4. First hybrid, then regular. However the intrinsics repo doesn't have granite4 options yet.
Checking status of intrinsics/aLora (including with hybrid - extra parameters for mamba).

planetf1 · 2026-01-26T15:57:37Z

Looking at CI failures...

- Migrate from Granite 3 to Granite 4 hybrid models - Document Granite 4 adapter limitations for HF tests - Add comprehensive HuggingFace adapter search results - Remove incorrect heavy_ram marker from test_spans.py - Revert HF tests to upstream approach (granite-3.3-8b, no env var) - Revert vision tests to granite3.2-vision due to Ollama crash - Map deprecated IBM_GRANITE_4_MICRO_3B to SMALL for watsonx support - Use MICRO model in Ollama tests to fit CI memory constraints - Restore per-backend model selection for IBM_GRANITE_4_MICRO_3B - Revert intrinsics examples to Granite 3.3 models

planetf1 · 2026-01-30T14:35:22Z

Rebased onto upstream/main and squashed to single commit (bc476b9).

Resolved conflicts - added pytest markers from upstream while preserving model selections.

Note: docs/examples/aLora/101_example.py has pre-existing bug (broken since Nov 2025 refactor, commit 1229206). Separate fix needed.

jakelorocco

Looks good! Lets make sure all the intrinsics / adapters tests still run (let me know if you need help testing those). Those tests won't run during the github cicd, so we will need to run them manually.

jakelorocco · 2026-01-27T20:53:35Z

docs/examples/intrinsics/intrinsics.py

 # backend._server_type = _ServerType.REMOTE_VLLM

-backend = LocalHFBackend(model_id="ibm-granite/granite-3.3-8b-instruct")
+backend = LocalHFBackend(model_id="ibm-granite/granite-4.0-tiny-preview")


Note to self: look at this part again.

jakelorocco · 2026-02-02T14:02:45Z

test/backends/test_openai_vllm/test_openai_vllm.py

+        model_id="ibm-granite/granite-4.0-h-tiny",
+        formatter=TemplateFormatter(model_id="ibm-granite/granite-4.0-h-tiny"),
        base_url="http://localhost:8000/v1",
        api_key="EMPTY",
    )
    backend.add_adapter(
        GraniteCommonAdapter(
            "requirement_check", base_model_name=backend.base_model_name
        )
    )


Were you able to test if this (and the other) intrinsic / adapter tests still worked? I think in this case at least, there's no requirement_check adapter trained for this model?

jakelorocco · 2026-02-02T14:04:04Z

GRANITE4_ADAPTER_STATUS.md

+# Granite 4 Adapter Availability Analysis
+
+## Summary
+**Finding**: HuggingFace tests CANNOT be migrated to Granite 4 because `requirement_check` adapter is not available for Granite 4 models.
+
+**Comprehensive Search**: Searched all 180+ ibm-granite repositories on HuggingFace. Found 22 standalone adapter repositories, ALL for Granite 3.x. Zero Granite 4 standalone adapters exist.
+
+## Adapter Repository Types
+
+### 1. Consolidated Libraries (Multi-Intrinsic)
+**ibm-granite/rag-intrinsics-lib** (Core Intrinsics):
+- `requirement_check`: ❌ Granite 3.3 only (2b, 8b)
+- `uncertainty`: ❌ Granite 3.3 only


I think this document should be removed.

planetf1 force-pushed the feat/issue-344 branch from 8c9543e to d19052b Compare January 26, 2026 12:11

planetf1 marked this pull request as ready for review January 26, 2026 15:43

planetf1 force-pushed the feat/issue-344 branch 2 times, most recently from 1e70336 to 014c09d Compare January 28, 2026 08:44

planetf1 requested review from jakelorocco and nrfulton January 29, 2026 12:43

planetf1 force-pushed the feat/issue-344 branch from 014c09d to bc476b9 Compare January 30, 2026 14:35

planetf1 mentioned this pull request Jan 30, 2026

fix: formatting model_ids for better readability #386

Merged

8 tasks

jakelorocco requested changes Feb 2, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: migrate from Granite 3 to Granite 4 hybrid models #357

feat: migrate from Granite 3 to Granite 4 hybrid models #357

planetf1 commented Jan 26, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Jan 26, 2026

Uh oh!

mergify bot commented Jan 26, 2026

Uh oh!

planetf1 commented Jan 26, 2026

Uh oh!

planetf1 commented Jan 26, 2026

Uh oh!

planetf1 commented Jan 30, 2026

Uh oh!

jakelorocco left a comment

Uh oh!

jakelorocco Jan 27, 2026

Uh oh!

jakelorocco Feb 2, 2026

Uh oh!

jakelorocco Feb 2, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

feat: migrate from Granite 3 to Granite 4 hybrid models #357

Are you sure you want to change the base?

feat: migrate from Granite 3 to Granite 4 hybrid models #357

Conversation

planetf1 commented Jan 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Migrate from Granite 3.x to Granite 4.0 Models

Type of PR

Description

Summary

Changes

Model Definitions (mellea/backends/model_ids.py)

Backend Updates

Test Updates (19 files)

Documentation Updates

Test Infrastructure

HuggingFace Test Exception

Vision Model Exception

Testing

Local Testing

CI Testing

Related Issues

Uh oh!

github-actions bot commented Jan 26, 2026

Uh oh!

mergify bot commented Jan 26, 2026

Merge Protections

🟢 Enforce conventional commit

Uh oh!

planetf1 commented Jan 26, 2026

Uh oh!

planetf1 commented Jan 26, 2026

Uh oh!

planetf1 commented Jan 30, 2026

Uh oh!

jakelorocco left a comment

Choose a reason for hiding this comment

Uh oh!

jakelorocco Jan 27, 2026

Choose a reason for hiding this comment

Uh oh!

jakelorocco Feb 2, 2026

Choose a reason for hiding this comment

Uh oh!

jakelorocco Feb 2, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

planetf1 commented Jan 26, 2026 •

edited

Loading

Model Definitions (`mellea/backends/model_ids.py`)