DocGen CLI

A simple CLI tool that converts any documentation page into a clean, structured README file using AI.

What It Does

Takes a documentation URL and generates a complete README with:

• Introduction and key features
• Installation steps
• Code examples
• Supports Arabic and English output

Features

• Smart Content Processing: Fetches, cleans, and structures any documentation page
• Intelligent Chunking: Automatically splits long content while preserving headings and context
• Dynamic Filenames: Extracts meaningful filenames from URLs (e.g., machine-learning.md)
• Bilingual Support: Generate documentation in English or Arabic
• Auto-Merge: Seamlessly combines multiple chunks into one coherent document
• Metadata Injection: Adds source URL, language, and timestamp to every output
• Error Handling: Validates URLs and handles errors gracefully
• Progress Tracking: Shows real-time progress for each processing step

How It Works

URL → Fetch Content → Clean → Split into Chunks → AI Processing → Generate README

1. You provide a documentation URL
2. Tool fetches and cleans the content
3. Splits content into chunks (if too long)
4. Sends to AI (Groq) to generate README
5. Saves output to output/ folder

Requirements

• Python 3.8+
• Groq API key

Setup

1. Clone the repository

git clone https://github.com/RayidAlshammari/docgen-cli.git
cd docgen-cli

2. Install uv (if you don't have it)

curl -LsSf https://astral.sh/uv/install.sh | sh

3. Create .env file

cp .env.example .env

4. Add your API key to .env

GROQ_API_KEY=your_api_key_here

Usage

Run with uv:

uv run docgen/cli.py

Follow the prompts:

1. Enter documentation URL (e.g., https://example.com/docs)
2. Choose language: 1 for English, 2 for Arabic
3. Wait for processing
4. Find your README in output/ folder

Example

$ uv run docgen/cli.py

Welcome to DocGen CLI!
Enter the documentation URL: https://www.geeksforgeeks.org/machine-learning/
Select explanation language:
1️⃣ English
2️⃣ Arabic
Enter 1 or 2: 2
Processing chunk 1/1...
✅ README has been saved in output/ directory!

Output: output/machine-learning.md

Project Structure

docgen-cli/
├── docgen/
│   ├── cli.py          # Main entry point
│   ├── fetcher.py      # Fetches URL content
│   ├── cleaner.py      # Cleans HTML content
│   ├── chunker.py      # Splits long content
│   ├── llm.py          # AI integration
│   ├── prompt.py       # Prompt templates
│   ├── renderer.py     # Saves output
│   └── validator.py    # Input validation
├── prompts/
│   ├── en.txt          # English prompt
│   └── ar.txt          # Arabic prompt
├── output/             # Generated READMEs
├── .env.example        # API key template
└── README.md

Notes

• Free Groq API has rate limits
• Large documentation may take a few minutes
• Output is saved with timestamp and language info

License

MIT