Refactor/nitrodigrst improvements

Projects/Nitrodigest/Docs/Getting Started/Quickstart.md

@@ -15,6 +15,16 @@ nitrodigest example.txt
 
 This command will use the default settings to summarize `example.txt`. The tool will connect to the local Ollama model, generate a summary, and save the result.
 
+### Alternative: Process Current Directory
+
+You can also run NitroDigest without any arguments to process all supported files in your current directory:
+
+```bash
+nitrodigest
+```
+
+This will automatically find and summarize all text files (`.txt`, `.md`, `.html`, `.json`, `.csv`, `.log`, etc.) in the current working directory. NitroDigest processes multiple files in parallel (up to 4 simultaneously by default) and shows a progress bar during processing.
+
 ## 3. Observe the output
 
 You should see console messages indicating the file is being processed and where the summary is saved. For example:
@@ -39,6 +49,16 @@ When the process is done, you can simply see the summary from file. Example:
 cat summary.md
 ```
 
+## 4. Including Original Text (Optional)
+
+If you want to include the original text alongside the summary, use the `--include-original` flag:
+
+```bash
+nitrodigest example.txt --include-original > summary-with-original.md
+```
+
+This will append the original text to the output after the summary, which can be useful for reference or comparison purposes.
+
 Read next:
 
 - [Summarizing All Files in a Directory](Summarizing%20All%20Files%20in%20a%20Directory.md)

Projects/Nitrodigest/Docs/Guides/Summarizing All Files in a Directory.md

@@ -15,8 +15,19 @@ This command will:
 
 1. Scan the directory and all subdirectories
 2. Find all supported text files
-3. Process each file individually using your default model
-4. Output each summary to the terminal in sequence
+3. Process multiple files in parallel using your default model (up to 4 files simultaneously by default)
+4. Display a progress bar during processing
+5. Output all summaries at the end
+
+### Process Current Directory
+
+You can also run NitroDigest without any arguments to automatically process all supported files in your current working directory:
+
+```bash
+nitrodigest
+```
+
+This is particularly useful when you're already in the directory you want to process and don't want to specify the path explicitly.
 
 ## Supported File Types
 
@@ -56,19 +67,22 @@ Will process `meeting-notes.txt`, `project-report.md`, `data-analysis.csv`, `web
 
 ### Terminal Output (Default)
 
-By default, all summaries are displayed in your terminal one after another:
+By default, all summaries are displayed in your terminal after processing completes:
 
 ```bash
 nitrodigest documents/
 ```
 
-You'll see processing messages and formatted summaries for each file:
+You'll see a progress bar during processing, followed by all summaries:
 
 ```bash
 Processing directory: documents/
-Processing file: documents/meeting-notes.txt
-Generating summary for meeting-notes.txt...
-2025-05-26 07:55:42,615 - cli.summarizer.base.OllamaSummarizer - INFO - Sending request to Ollama API using model mistral
+Found 5 files to process with 4 workers
+
+Processing:  100%|████████████████████| 5/5 [00:12<00:00, 2.45s/file] ✓ more-notes.txt
+
+Processing complete: 5 successful, 0 failed
+
 ---
 date: '2025-05-16 07:50:22'
 id: documents/meeting-notes.txt
@@ -81,10 +95,15 @@ tokens: 189
 
 <summary of meeting-notes.txt>
 
-Processing file: documents/project-report.md
-Generating summary for project-report.md...
+================================================================================
+
+---
+date: '2025-05-16 08:15:10'
+id: documents/project-report.md
 ...
-Directory processing complete: 4 of 4 files processed successfully
+---
+
+<summary of project-report.md>
 ```
 
 ### Save All Summaries to One File
@@ -115,6 +134,13 @@ For directories with just a few files, processing is straightforward and fast:
 nitrodigest my_notes/
 ```
 
+Or if you're already in the directory:
+
+```bash
+cd my_notes/
+nitrodigest
+```
+
 ## Directory Processing Behavior
 
 ### Recursive Processing
@@ -132,12 +158,33 @@ project/
 
 All three files (`overview.md`, `specifications.txt`, and `notes.txt`) will be processed.
 
-### File Ordering
+### Parallel Processing
+
+NitroDigest processes multiple files simultaneously to improve performance. By default, it uses 4 parallel workers, meaning up to 4 files can be processed at the same time.
 
-Files are processed in the order they're discovered by the file system, which typically means:
+#### Adjusting Parallel Workers
 
-- Files in the main directory first
-- Then files in subdirectories
+You can control the number of parallel workers based on your system resources and needs:
+
+```bash
+# Use 8 workers for faster processing (good for powerful systems)
+nitrodigest documents/ --max-workers 8
+
+# Use 2 workers for slower systems or to reduce resource usage
+nitrodigest documents/ --max-workers 2
+
+# Use 1 worker for sequential processing
+nitrodigest documents/ --max-workers 1
+```
+
+**When to adjust workers:**
+- **Increase workers (6-8):** If you have a powerful system and want maximum speed
+- **Decrease workers (1-2):** If you have limited RAM, CPU, or want to reduce system load
+- **Keep default (4):** For most use cases, this provides a good balance
+
+### File Ordering
+
+Files are processed in parallel, so they may complete in a different order than discovered. However, all files in the directory and subdirectories will be processed.
 
 ## Practical Use Cases
 
@@ -175,6 +222,20 @@ nitrodigest meeting_notes_march/ > march_meetings_summary.md
 
 ## Tips and Best Practices
 
+### Performance Optimization
+
+For best performance when processing large directories:
+
+```bash
+# Use more workers on powerful systems
+nitrodigest large_directory/ --max-workers 8 > summaries.md
+
+# Monitor your system resources (CPU, RAM) and adjust workers accordingly
+# If Ollama is running on the same machine, consider your model's resource needs
+```
+
+**Pro tip:** The optimal number of workers depends on your Ollama setup. If Ollama is using significant resources, fewer workers may actually be faster.
+
 ### Organize Your Input
 
 Structure your directories logically before processing:
@@ -236,6 +297,18 @@ If your directory contains specialized content, use a custom prompt:
 nitrodigest technical_docs/ --prompt "Summarize this technical document focusing on implementation details and requirements" > tech_summaries.md
 ```
 
+### Combining Parallel Processing with Other Options
+
+You can combine `--max-workers` with other options for optimized processing:
+
+```bash
+# Fast processing with custom model and 8 workers
+nitrodigest documents/ --model llama3 --max-workers 8 > summaries.md
+
+# Slower but thorough processing with 2 workers and custom prompt
+nitrodigest research/ --max-workers 2 --prompt-file research_prompt.txt > research_summaries.md
+```
+
 ## Next Steps
 
 - **[Custom Prompts](./Overriding%20Prompt%20Templates.md):** Explore Overriding Prompt Templates for specialized content

Projects/Nitrodigest/Docs/Guides/Understanding the Output Format.md

@@ -183,6 +183,61 @@ Create a summary table for this document:
 | Timeline | Important dates or deadlines |
 ```
 
+### Including Original Text
+
+By default, NitroDigest only outputs the summary. You can include the original text alongside the summary using the `--include-original` flag:
+
+```bash
+# Include original text with summary
+nitrodigest document.txt --include-original
+```
+
+**Text Format Output with Original:**
+
+```yaml
+---
+title: document-name.txt
+source: file:///absolute/path/to/document-name.txt
+date: '2025-05-16 07:50:22'
+id: document-name.txt
+summary_date: '2025-05-26 07:55:46'
+model: mistral
+tokens: 189
+---
+
+# Summary
+
+1. Key summary points here...
+2. Additional summary content...
+
+# Tags
+
+1. tag1
+2. tag2
+
+---
+
+## Original Text
+
+[The complete original text content would appear here]
+```
+
+**JSON Format with Original:**
+
+```json
+{
+  "summary": ["Summary content here..."],
+  "tags": ["tag1", "tag2"],
+  "metadata": {
+    "title": "document.txt",
+    "source": "file:///path/to/document.txt",
+    "date": "2025-05-16 07:50:22",
+    "id": "document.txt"
+  },
+  "original_text": "The complete original text content would appear here"
+}
+```
+
 ### JSON Structured Output
 
 You can use the `--format` flag to change output format to JSON:

Projects/Nitrodigest/Docs/Guides/Using a Custom Configuration.md

@@ -95,6 +95,22 @@ Change output format to `json`:
 nitrodigest  document.txt --format json
 ```
 
+### Include Original Text
+
+**Default:** `False`
+
+Include the original text alongside the summary in the output:
+
+```bash
+# Include original text in summary output
+nitrodigest document.txt --include-original
+
+# Exclude original text (default behavior)
+nitrodigest document.txt
+```
+
+When the `--include-original` flag is present, the original text will be appended after the summary in text format, or included as an `original_text` field in JSON format.
+
 ## Setting Up Default Configurations
 
 ### Environment Variables

Projects/Nitrodigest/Docs/NitroDigest – Documentation.md

@@ -10,7 +10,7 @@ permalink: projects/nitrodigest/docs
 - **Local AI Summarization:** Uses Ollama to run LLMs on your machine, preserving privacy and working offline.
 - **Multiple Input Formats:** Supports plain text, Markdown, HTML, CSV, JSON, and other text-based files.
 - **Multiple Output Formats: By default NitroDigest returns Text, but for advanced processing it can return JSON.
-- **Batch Processing:** Summarize a single file or all files in a directory in one command.
+- **Parallel Batch Processing:** Summarize a single file or process multiple files in a directory simultaneously with configurable parallel workers for faster processing.
 - **Configurable Prompts:** Uses prompt templates that you can customize to change the style or content of summaries.
 - **Extensible:** Easily switch to different models (e.g., use a larger or domain-specific Ollama model) and adjust token budgets or segmentation for large inputs.
 
@@ -32,12 +32,7 @@ Note: this documentation is under development just like the tool. Not all sectio
 - **Use Cases**
     - [Summarizing Email Newsletters](Summarizing%20Email%20Newsletters.md)
     - [Summarizing Web Pages](Summarizing%20Web%20Pages.md)
-    - [Summarizing Slack Messages](Summarizing%20Slack%20Messages.md)
     - [Summarizing GitHub Pull Requests](Summarizing%20GitHub%20Pull%20Requests.md)
-- **Customization**
-    - [Custom Prompt Templates](Custom%20Prompt%20Templates.md)
-    - [Switching Models (Ollama Integration)](Switching%20Models%20(Ollama%20Integration).md)
-    - [Adjusting Token Budgets & Segmentation](Adjusting%20Token%20Budgets%20&%20Segmentation.md)
 - **Contributing**
 	- [Getting started](Getting%20started.md)
 	- [Ollama setup](Ollama%20setup.md)

Projects/Nitrodigest/README.md

@@ -34,6 +34,12 @@ nitrodigest <file or directory you want to summarize> > <destination where to wa
 
 #### Examples
 
+**Process current directory** (summarize all files in the current working directory):
+
+```bash
+nitrodigest > summary.md
+```
+
 Summarize one file and save it to summary.md:
 
 ```bash
@@ -63,7 +69,9 @@ Available arguments:
 - `--prompt`: Direct prompt content (overrides prompt-file)
 - `--model`: Model that will be used for summarization (default: mistral)
 - `--ollama_api_url`: URL of Ollama API (default: <http://localhost:11434>)
-- `--format`: Output format. Can be `text` or `json` (default: text)"
+- `--format`: Output format. Can be `text` or `json` (default: text)
+- `--include-original`: Include original text in the summary output (default: False)
+- `--max-workers`: Maximum number of parallel workers for directory processing (default: 4)
 
 ### Custom Prompt Configuration
 

Projects/Nitrodigest/setup.cfg

@@ -1,6 +1,6 @@
 [metadata]
 name = nitrodigest-cli
-version = 0.2.0
+version = 0.3.0
 author = Marcin Kwiatkowski
 author_email = marcin@frodigo.com
 description = The privacy‑first, local‑LLM text‑summariser for developers.
@@ -20,6 +20,7 @@ install_requires =
     pyyaml>=6.0
     nltk>=3.9.1
     emoji>=2.14.1
+    tqdm>=4.67.1
 
 [options.packages.find]
 where = src

Projects/Nitrodigest/src/cli/__init__.py

@@ -1,6 +1,6 @@
 """nitrodigest CLI package"""
 
-__version__ = "0.2.0"
+__version__ = "0.3.0"
 
 from .main import main
 from .config import Config