Skip to content

feat(android): add on-device RAG and guard pipeline - #40

Open
Si1as-code wants to merge 1 commit into
OpenBMB:mainfrom
Si1as-code:codex/android-rag-guard-v4-2
Open

feat(android): add on-device RAG and guard pipeline#40
Si1as-code wants to merge 1 commit into
OpenBMB:mainfrom
Si1as-code:codex/android-rag-guard-v4-2

Conversation

@Si1as-code

Copy link
Copy Markdown

Summary

This contribution evolves MiniCPM-V-demo-Android into an offline, office-oriented multimodal assistant with persistent conversations, editable message history, document ingestion, encrypted local knowledge bases, hybrid retrieval, HNSW indexing, and a local RAG answer guard.

The branch is rebased as one commit on the current upstream main (bbd042e) to preserve upstream changes and simplify review.

Main changes

  • Improved Android chat UX: permanent status bar, camera entry, cancellable image preprocessing, original-image viewer, keyboard viewport anchoring, editable/deletable messages, rollback, and multi-conversation persistence.
  • Added local document knowledge bases with user-defined names, import progress stages, retry/failure reasons, document and knowledge-base deletion, and per-conversation RAG selection.
  • Added encrypted local storage using Room/SQLCipher, Android Keystore-backed keys, AES-GCM file storage, WorkManager ingestion, and recovery-safe publication.
  • Added TXT/Markdown/HTML/DOCX/PDF parsing, OCR fallback, chunking, multilingual E5 embeddings, BM25, reciprocal-rank fusion, and a JNI HNSW vector backend.
  • Added checkpoint-aware generation so retrieved evidence remains transient until a response is accepted.
  • Added local visual-context, privacy/content-safety, Answerability (3-class), and Groundedness (4-class) policy layers.
  • Added bilingual training/evaluation tooling, dataset provenance records, regression suites, Android/JVM/instrumented tests, and a persistent Graphify code graph.

Recorded validation

The complete formal-version branch records:

  • Android JVM tests: 308/308 passed.
  • Python RAG Guard tests: 143 passed plus 9 parameterized subtests.
  • Device validation on vivo V2359A, including signing-aware reinstall persistence.
  • HNSW 20k-vector recall and latency benchmarks.
  • On-device RAG Guard latency and semantic integration tests.

These are recorded results from the formal branch. The detailed commands, artifacts, limitations, and code/test traceability are documented in the formal optimization report.

Model artifact note

The Android integration uses model.int8.onnx (118,171,779 bytes, SHA-256 d674ef4ef4fb2b4dce37d43c46eeb4b0e8038eb66da7cde1b568ca78dc45e1c2). GitHub does not permit contributors to upload new Git LFS objects to a public fork because LFS storage is owned by the upstream repository, so this PR intentionally omits that binary object.

The verified object remains available in the complete public formal-version repository. The contribution branch documents how to place it under models/rag-guard-v4-2-e5/ or supply it through RAG_GUARD_ARTIFACT_DIR. If the maintainers accept the integration, the object can be copied into upstream-owned LFS or distributed using the maintainers' preferred release channel.

Review notes and limitations

  • This is intentionally Android-only; iOS and HarmonyOS demos are unchanged.
  • The v4.2 frozen test split was not read or evaluated during export. Calibration observations are recorded without presenting them as a release gate.
  • The change is large because it contains the complete end-to-end feature, test, training, and audit trail. I am happy to split it into smaller PRs if the maintainers prefer staged review.
  • No private training corpus or credentials are included.

Reference implementation

Integrate persistent conversations, document ingestion, encrypted local storage, hybrid retrieval, HNSW indexing, checkpoint-aware generation, content and visual guards, Android UI improvements, training/evaluation tooling, tests, and the formal optimization report. The verified INT8 model object remains in the complete public release repository because GitHub public forks cannot accept contributor-owned LFS objects.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant