fix: use cast_column for faster string conversion of label columns (Fixes #145)

[rtmalikian]

Fixes #145

Problem

The AutoProcessor in cnlp_processors.py converts classification label columns from integers to strings using a slow pattern:

self.dataset[split] = self.dataset[split].add_column(task_str, [str(x) for x in self.dataset[split][task]])
self.dataset[split] = self.dataset[split].remove_columns(task)
self.dataset[split] = self.dataset[split].rename_column(task_str, task)

This creates a new column with a Python list comprehension (str(x) for every element), removes the old column, and renames the new one. For large datasets, this is extremely slow because it iterates through every element in Python.

Solution

Replace the slow pattern with datasets.cast_column(task, datasets.Value("string")), which operates at the Apache Arrow level:

self.dataset[split] = self.dataset[split].cast_column(task, datasets.Value("string"))

This is dramatically faster because:

1. No intermediate column creation
2. No column removal or renaming
3. Arrow's native type casting is orders of magnitude faster than Python list comprehensions

Verification

# All existing tests pass
python -m pytest test/data/test_cnlp_dataset.py -v
# 3 passed in 5.07s

# Manual verification with AutoProcessor
python3 -c "
from cnlpt.legacy.cnlp_processors import AutoProcessor
# ... creates processor with classification tasks, labels are correctly converted to strings
# SUCCESS: AutoProcessor works with cast_column fix!
"

---

About the Author: Raphael Malikian — Clinical AI Solutions Architect. I specialise in building and fixing AI/ML systems for healthcare, including vector databases, RAG pipelines, and clinical NLP. If you need help with your project or think I can add value to your organisation, feel free to reach out — I'd love to connect.

📧 rtmalikian@gmail.com
🔗 GitHub: https://github.com/rtmalikian
🔗 LinkedIn: http://www.linkedin.com/in/raphael-t-malikian-mbbs-bsc-hons-71075436a

---

Disclosure: This code was developed with assistance from mimo-2.5-pro (Xiaomi) via Hermes Agent (Nous Research). All changes were reviewed, tested against the actual codebase, and verified for correctness.