Skip to content

About

Adaptive extractive text compression for AI agents. Shorten academic documents without paraphrasing, with traceable edits, semantic review and optional lightweight OCR.

Topics

Resources

Stars

10 stars

Watchers

0 watching

Forks

Latest commit

 

History

7 Commits

Folders and files

Repository files navigation

Doc Compression — adaptive extractive text compression

Doc Compression — adaptive extractive text compression

Shorten documents by removing redundant text while keeping retained passages in the author's words and order. An agent skill for papers, chapters, reports and documentation, with source-traceable selections and a separate semantic review.

English · Español · Discovery notes

v4.0.0: adaptive compression with traceable edits, separate semantic review and optional lightweight OCR. See the changelog for breaking changes from v3.

Install

Requires Node.js/npm to run the Skills CLI, an agent that loads SKILL.md, and Python 3.10 or newer for the core scripts, which use only the Python standard library. Extraction, OCR and PDF rendering have separate tool requirements; OCR is optional.

# Inspect available skills without installing
npx skills add fedeclavero/doc-compression-skill --list

# Install the named skill; follow the CLI's agent/scope options
npx skills add fedeclavero/doc-compression-skill --skill doc-compression

To use this local checkout, run from the repository root:

npx skills add . --skill doc-compression

The remote listing was verified on 2026-10-01. See the official CLI source and installation options. Installation does not bundle an AI model or every document converter.

Try it

Attach a document or give your agent a local file path, then ask:

Use doc-compression to shorten chapter.md by removing repetition. Keep definitions, quotations and qualifications. Review each identified fragment and report the actual reduction; do not force a percentage.

Compress report.pdf extractively. Try native text extraction first; use optional OCR only where needed. Deliver Markdown and a PDF if a renderer is available, and identify extraction uncertainties.

Comprimí este capítulo por eliminación, sin reescribir. Conservá citas, datos y matices; revisá el significado y explicá qué se pudo reducir.

How v4 works

  1. Extract and inspect. Read native text first. Check completeness and reading order before compressing; use lightweight, adaptive OCR when necessary and available.
  2. Identify fragments. Divide the source into traceable fragments so selections can be tied to their original location.
  3. Choose deletions adaptively. Review redundancy within each fragment and its surrounding argument. Dense passages may remain unchanged. There is no forced default reduction ratio.
  4. Check provenance and structure. Scripts validate source selections and supported structural constraints, with strict preservation of quotations, tables, code and every numeric occurrence. Counts describe the result rather than set a deletion quota.
  5. Review meaning. The agent checks negation, qualifications, dependencies, evidence and continuity against the source. The review covers individual blocks and comprehension, and is bound by hashes to the artifacts being reviewed. Mechanical validation alone cannot certify that meaning survived.
  6. Deliver and report. Provide the compressed document, actual counts, validation findings and any unresolved extraction or semantic concerns.

A requested length is a preference to assess against the content. If reaching it would require losing necessary meaning, explain the constraint rather than silently cutting further.

A small example

Illustrative text, not a benchmark:

Source

The sample contains 24 observations. The analysis is preliminary. The analysis is preliminary. The result may reflect sampling error; it does not establish causation.

Compressed

The sample contains 24 observations. The analysis is preliminary. The result may reflect sampling error; it does not establish causation.

The duplicated nonnumeric sentence is removed; the sole numeric occurrence is retained. The number, uncertainty and negation remain. An ordered selection that deleted “not” could still change the conclusion: this is why provenance checks and semantic review are separate.

Formats and requirements

Material Handling
Markdown / plain text Work directly with the text and its structure.
PDF with usable text Native extraction first; inspect page coverage and reading order.
Scanned or mixed PDF Optional Tesseract OCR for pages that need it, subject to installed tools and language data. OCR errors require review.
DOCX, EPUB, HTML, PPTX and other formats Use additional installed reader skills/capabilities; these readers are not bundled.
Output Markdown is the fallback. The PDF renderer supports a subset of document features; each generated PDF requires visual verification. Other formats depend on installed capabilities.

This reduces document wording, not PDF file size. It does not promise a fixed reduction, a paraphrased abstract, automatic semantic equivalence or universal OCR accuracy. Quotations, definitions, data, headings and other protected material need both structural checks and contextual review.

Core CLI and quick checks

From the repository root:

python3 doc-compression/scripts/compress.py --help
python3 -m unittest discover -s tests -v

The core workflow uses prepare, apply, review-template and finalize. Run a subcommand with --help for its arguments. The agent selects source fragments and completes the block-level and comprehension review; the scripts construct and validate the artifacts. Hash binding ties the review to the reviewed content, not to an automatic guarantee of meaning.

Inspect the skill

Compatibility depends on the agent's ability to follow the workflow, read files and execute the required tools. A successful CLI listing is not an end-to-end compatibility test.

Verified example

The academic fixture was reduced from 585 to 531 words (9.23%), without a forced quota. PDF and traceable artifacts · Validation scope and limits. This is one reviewed example, not a universal compression benchmark.

MIT license · Changelog

About

Adaptive extractive text compression for AI agents. Shorten academic documents without paraphrasing, with traceable edits, semantic review and optional lightweight OCR.

Topics

Resources

Stars

10 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages