DocCompare

Document similarity built on histogram projections, so hidden characters cannot move the score.

---

🗺️ Map

• 📖 Motivation
• 📦 Installation
• 🚀 Usage
• 🧪 Validation
• 🤝 Contributing
• 📝 License

📖 Motivation

$$\color{#00BFFF}Purpose$$

There is a constant rise in the amount of data being copied or plagiarized because of the abundance of content freely available across the internet. Institutions run every submission through similarity software, and those tools work well, which is exactly why people started looking for ways around them.

One of those ways is character injection. A character is added at the end of every word, its font size is minimized and its colour is changed to white. The reader sees nothing. The extractor sees all of it. Because similarity checks based on k-grams hash the characters they are given, every injected letter shifts the grams around it, the hashes stop matching, and a document that was copied word for word comes back with a good enough score to make it look original.

This repository is the reference implementation of the method we published for fixing this. Instead of trusting the text inside the file, the page is rendered, binarised, and segmented with multiple histogram projections. A character painted in the background colour deposits no ink, so it never appears in the projection at all. There is nothing to filter out and nothing to guess at.

Namburu, A.; Surendran, A.; Balaji, S.V.; Mohan, S.; Iwendi, C. DocCompare: An Approach to Prevent the Problem of Character Injection in Document Similarity Algorithm. Mathematics 2022, 10(22), 4256. doi.org/10.3390/math10224256

$$\color{#00BFFF}Features \space \color{#56565E}Included$$

• Histogram projection segmentation — HPH[i] for lines, VPH[j] for words, straight from the paper.
• Exact injection detection: every character's glyph box is tested against the binarised page, so anything invisible is found regardless of alphabet or language.
• Winnowing fingerprints with a deterministic FNV-1a hash, so results reproduce across runs.
• Jaccard, Dice and Cosine coefficients computed over the fingerprint sets, as published.
• Live progress streamed over NDJSON.
• Downloadable PDF report with both score sets, the injection findings and the k-gram hashes.

Process flow for both features — the similarity check and the character injection test.

$$\color{#00BFFF}Quick \space \color{#56565E}Start$$

If you want the fastest path to a working local setup, clone the repo, install dependencies with uv, and run the guided setup script:

git clone https://github.com/SVijayB/DocCompare
cd DocCompare
uv sync
uv run python setup.py

The setup script checks your Python version, installs the project, builds the paper's test fixtures, and then verifies the pipeline end to end by reproducing the published result.

📦 Installation

$$\color{#00BFFF}Clone \space \color{#56565E}Repository$$

git clone https://github.com/SVijayB/DocCompare
cd DocCompare

$$\color{#00BFFF}Install \space \color{#56565E}Prerequisites$$

This project uses uv for fast, reliable Python package management. Install it with:

pip install uv

You will also need Node.js 20 or newer if you want to run the web interface. s

$$\color{#00BFFF}Setup \space \color{#56565E}Environment$$

Create a virtual environment and install all requirements:

uv sync

Then install the frontend dependencies:

cd frontend && npm install && cd ..

If the API does not run on its default address, point the frontend at it by creating frontend/.env.local:

REACT_APP_API_URL=http://localhost:8000

Once your environment is configured, run the setup script to verify everything works:

uv run python setup.py

🚀 Usage

$$\color{#00BFFF}Launch \space \color{#56565E}Application$$

Start the API:

uv run uvicorn api:app --reload

Then, in a second terminal, start the web interface:

cd frontend && npm start

The API serves on http://localhost:8000 with interactive docs at /docs, and the web interface runs on http://localhost:3000.

There is also an interactive terminal menu if you would rather not use the browser:

uv run python cli.py

$$\color{#00BFFF}Project \space \color{#56565E}Demo$$

Hosted frontend with sample runs

🤝 Contributing

$$\color{#00BFFF}How \space \color{#56565E}to \space Contribute$$

To contribute to DocCompare, fork the repository, create a new branch and send us a pull request. Make sure you read CONTRIBUTING.md before sending us Pull requests.

Thanks for contributing to Open-source! ❤️

📝 License

This project is licensed under the MIT License. Read the LICENSE file for details.

██╗    ██╗███████╗    ██╗      ██████╗ ██╗   ██╗███████╗      ██╗██████╗                 
██║    ██║██╔════╝    ██║     ██╔═══██╗██║   ██║██╔════╝     ██╔╝╚════██╗                
██║ █╗ ██║█████╗      ██║     ██║   ██║██║   ██║█████╗      ██╔╝  █████╔╝                
██║███╗██║██╔══╝      ██║     ██║   ██║╚██╗ ██╔╝██╔══╝      ╚██╗  ╚═══██╗                
╚███╔███╔╝███████╗    ███████╗╚██████╔╝ ╚████╔╝ ███████╗     ╚██╗██████╔╝                
 ╚══╝╚══╝ ╚══════╝    ╚══════╝ ╚═════╝   ╚═══╝  ╚══════╝      ╚═╝╚═════╝                 
                                                                                         
 ██████╗ ██████╗ ███████╗███╗   ██╗    ███████╗ ██████╗ ██╗   ██╗██████╗  ██████╗███████╗
██╔═══██╗██╔══██╗██╔════╝████╗  ██║    ██╔════╝██╔═══██╗██║   ██║██╔══██╗██╔════╝██╔════╝
██║   ██║██████╔╝█████╗  ██╔██╗ ██║    ███████╗██║   ██║██║   ██║██████╔╝██║     █████╗  
██║   ██║██╔═══╝ ██╔══╝  ██║╚██╗██║    ╚════██║██║   ██║██║   ██║██╔══██╗██║     ██╔══╝  
╚██████╔╝██║     ███████╗██║ ╚████║    ███████║╚██████╔╝╚██████╔╝██║  ██║╚██████╗███████╗
 ╚═════╝ ╚═╝     ╚══════╝╚═╝  ╚═══╝    ╚══════╝ ╚═════╝  ╚═════╝ ╚═╝  ╚═╝ ╚═════╝╚══════╝