OpenMed can export token-classification checkpoints for Android ONNX Runtime
Mobile with the android ONNX profile:
python -m pip install --upgrade "openmed[onnx,hf]".venv/bin/python -m openmed.onnx.convert \
--model dslim/bert-base-NER \
--output dist/example-android-onnx \
--profile androidThe profile writes fp32/fp16 ONNX graphs, an optional dynamic INT8 graph, and, when ONNX Runtime's ORT conversion tooling is installed, an ORT mobile artifact plus the minimal-build operator configuration:
dist/example-android-onnx/
model.onnx
model_fp16.onnx
model_int8.onnx
model.ort
model.required_operators_and_types.config
config.json
id2label.json
openmed-onnx.json
Python CPU:
from openmed import OnnxModel
model = OnnxModel.from_pretrained(
"OpenMed/OpenMed-PII-ClinicalE5-Small-33M-v1-onnx-android"
)
entities = model("Patient Alice Nguyen was seen in cardiology.")WebGPU or WebAssembly:
npm install openmed @huggingface/transformersimport { loadOnnxModel } from "openmed";
const model = await loadOnnxModel(
"OpenMed/OpenMed-PII-ClinicalE5-Small-33M-v1-onnx-android",
);
const entities = await model("Patient Alice Nguyen was seen in cardiology.");OpenMedKit for Android is published from immutable OpenMed release tags.
Add the scoped JitPack repository in settings.gradle.kts:
dependencyResolutionManagement {
repositories {
google()
mavenCentral()
maven {
url = uri("https://jitpack.io")
content { includeGroup("com.github.maziyarpanahi") }
}
}
}Add the OpenMed v3.2.0 release to the app module:
dependencies {
implementation("com.github.maziyarpanahi:openmed:v3.2.0")
}Then load the repository after it is stored in an app-local model directory:
val model = OpenMedKit.fromDirectory(modelDirectory)
val entities = model.analyzeText("Patient Alice Nguyen was seen in cardiology.")JitPack consumers do not need GitHub credentials. Each runtime reads the repository's tokenizer and label assets. Python and Web download once and reuse their local cache; Android performs inference from the app-local directory and makes no network request during inference.
For the full OpenMed PyTorch token-classification inventory, use the resumable
batch runner. It reads models.jsonl, selects source PyTorch rows by default,
skips MLX/CoreML/ONNX-derived rows, and writes compact status evidence to
dist/onnx-android/status.jsonl. The runner disables Hugging Face Xet transfers
by default for rollout reliability; set HF_HUB_DISABLE_XET=0 to opt back in:
.venv/bin/python scripts/onnx/batch_android_convert_publish.py --dry-runTo convert and upload privately, set a write token in HF_WRITE_TOKEN and run:
HF_WRITE_TOKEN=... \
.venv/bin/python scripts/onnx/batch_android_convert_publish.py \
--publish-to-hub \
--publish-manifest dist/onnx-android/private-models.jsonlPublished repositories are created private by the batch runner. There is no
public-upload flag in this path. Keep release promotion and updates to the
public models.jsonl as a separate audited step after Android parity,
quantization, and catalog checks pass.
GLiNER checkpoints are intentionally excluded from this batch. Their zero-shot span contract requires prompt labels, word masks, candidate-span indices, and span-logit decoding; it is not compatible with OpenMedKit's standard Android token-classification session. GLiNER remains available through the Python and MLX runtimes until a dedicated Android session is implemented and certified.
Privacy-filter mixture-of-experts checkpoints are also excluded. Their current PyTorch forward pass uses data-dependent expert routing that cannot be safely captured by the legacy ONNX tracer: the exported graph hard-codes routing split sizes and fails parity execution on different synthetic inputs. These models remain available through their supported Python and MLX runtimes.
model.onnx is the fp32 graph. model_fp16.onnx stores fp16 weights while
keeping public input and output tensor types stable for Android callers.
model_int8.onnx is produced with ONNX Runtime dynamic QInt8 quantization over
the MatMul/Gemm weights used by transformer token classifiers. This is the ARM
CPU starting point for Android Runtime Mobile because it needs no calibration
corpus and keeps the exported graph callable through the same inputs and
logits output.
model.ort is the ORT mobile-format model generated from model.onnx.
model.required_operators_and_types.config is the required-operators/types
file to pass into an ONNX Runtime Android minimal build.
The graph family and operator list in openmed-onnx.json also feed
Android QNN and NNAPI Acceleration. The
Android runtime combines that manifest contract with device-observed provider
coverage so unsupported subgraphs stay on CPU.
If the optional ONNX Runtime conversion tooling is not available, export still
finishes with the .onnx artifacts and logs a dependency-only skip reason. The
message does not include source model text or sample input content.
The Android profile validates the exported graph with onnx.checker and then
checks the token-classification contract used by OpenMedKit:
- inputs:
input_ids,attention_mask, and optionaltoken_type_ids - input dtype:
int64 - output:
logits - output dtype:
float32 - input axes: dynamic
batchand dynamicsequence - output axes: dynamic
batch, dynamicsequence, and staticlabels
The profile fixes the ONNX opset at 18. A fixed opset is required because the
follow-up ONNX Runtime Mobile .ort conversion and minimal-build operator
configuration need a predictable graph contract. The dynamic batch and
sequence axes are retained so one exported artifact can serve variable
request batches and note lengths on device.
The ORT mobile step uses ONNX Runtime's Python conversion tool with the fixed
optimization style, Android ARM target platform, and type reduction enabled.
The resulting model.required_operators_and_types.config can be passed to the
ONNX Runtime build as the include-ops config, for example:
./build.sh \
--android \
--minimal_build \
--include_ops_by_config dist/example-android-onnx/model.required_operators_and_types.configBuilding the actual Android AAR is a separate Android-module task; this export only writes the model and operator/type configuration consumed by that build.
Published Android rows in models.jsonl can describe their deployment contract
with the canonical formats list and four optional top-level fields:
{
"formats": ["onnx-android", "onnx-int8", "ort-android"],
"nnapi_compatible": true,
"min_sdk": 27,
"execution_providers": ["NNAPI", "XNNPACK"],
"tokenizer_assets": ["tokenizer.json", "tokenizer_config.json"]
}| Field | Meaning |
|---|---|
formats |
Android artifacts present in the model repository. Use onnx-android for the fp32/fp16 graphs, onnx-int8 for the dynamic INT8 graph, and ort-android for the ORT Mobile model. |
nnapi_compatible |
Whether the published graph is compatible with the Android NNAPI execution provider. |
min_sdk |
Lowest supported Android SDK/API level as a positive integer. |
execution_providers |
Ordered runtime provider hints, such as NNAPI and XNNPACK. Availability still depends on the target device and runtime build. |
tokenizer_assets |
Repository-relative tokenizer files bundled with the artifacts. |
The four Android metadata fields are optional. Existing manifest rows validate unchanged when they are absent. When at least one is present, the generated model card includes an Android metadata section and the autonomous-clinical- decision disclaimer.
openmed-onnx.json records the Android artifacts with the format string
onnx-android for fp32/fp16 and onnx-int8 for the dynamic INT8 graph. When
ORT conversion succeeds, the manifest also records an ort-android artifact.
Each artifact entry includes profile metadata with the opset, tensor contract,
execution-provider hints (NNAPI, XNNPACK), graph operators, and any
operators that are outside OpenMed's Android mobile safe set. The INT8 artifact
metadata records the dynamic quantization recipe and target model_int8.onnx
path. The ORT artifact metadata records the model.ort path, the
required-operators/types config path, the fixed optimization style, Android ARM
target platform, type-reduction status, opset, and graph operators.
Unsupported operators are reported as warnings and recorded in artifact
metadata. The converter does not delete, rewrite, or silently drop graph nodes;
the warnings are used by the .ort and Android runtime tasks to decide whether
a model needs a fallback or exporter adjustment.
When an eval suite is available, pass it to the Android export:
.venv/bin/python -m openmed.onnx.convert \
--model dslim/bert-base-NER \
--output dist/example-android-onnx \
--profile android \
--eval-suite openmed/eval/golden/fixtures/checksum_ids.jsonThe converter writes recall_delta.json next to the ONNX artifacts. The report
compares the Hugging Face full-precision parent with model_int8.onnx on the
same fixtures and records:
format:onnx-int8quantization: dynamic QInt8 metadata for the ARM CPU artifactfp_parent_per_label_recallcandidate_per_label_recallper_label, with fp32 recall, INT8 recall, recall delta, and aggregate gold span/character countsquant_recall_delta, the single max recall-loss figure consumed by G4limit, currently0.005for INT8certified, true only when the measured delta is below the INT8 limit
The report is aggregate-only. It does not serialize fixture text, record IDs, span text, offsets, gold spans, or predicted spans. Use synthetic or approved eval fixtures and keep any DUA-governed corpus outside committed artifacts.
The artifact config.json and openmed-onnx.json mirror the certification
fields under quant_recall_delta, certified, recall_delta_path, and
quantization. An over-budget INT8 export still writes model_int8.onnx and
the report, but records certified: false so release gates can block the
format.