Skip to content

Add Wiktionary th-pron transliterator to pythainlp.transliterate#1437

Merged
wannaphong merged 45 commits into
devfrom
copilot/add-module-th-pron
Jun 24, 2026
Merged

Add Wiktionary th-pron transliterator to pythainlp.transliterate#1437
wannaphong merged 45 commits into
devfrom
copilot/add-module-th-pron

Conversation

@wannaphong

@wannaphong wannaphong commented Jun 24, 2026

Copy link
Copy Markdown
Member

What do these changes do

Adds a new wiktionary transliteration module (ported from Wiktionary Module:th-pron) into pythainlp.transliterate, with outputs for paiboon, royin, and ipa. It also exposes a small public API and adds core tests for expected behavior.

  • New transliteration module

    • Added pythainlp/transliterate/wiktionary.py
    • Implements:
      • transliterate_wiktionary(text, mode="ipa")
      • get_word_dict(word) (returns word, paiboon, royin, ipa)
      • transliterate(...) alias for compatibility
  • Package-level API exposure

    • Updated pythainlp/transliterate/__init__.py to export:
      • transliterate_wiktionary
      • get_word_dict
  • Core test coverage

    • Added tests/core/test_transliterate_wiktionary.py
    • Registered it in tests/core/__init__.py
from pythainlp.transliterate import get_word_dict, transliterate_wiktionary

transliterate_wiktionary("แมว", mode="royin")  # "maeo"
get_word_dict("ปฺระ-ชา")
# {"word": "ปฺระ-ชา", "paiboon": "...", "royin": "...", "ipa": "..."}

What was wrong

pythainlp.transliterate did not provide the Wiktionary th-pron logic requested by the issue, so there was no built-in way to generate these three th-pron-style outputs from one implementation.

How this fixes it

Ports the th-pron transliteration logic into a dedicated module, wires it into package exports, and adds focused core tests to lock expected outputs and mode behavior (including unknown-mode passthrough and Thai digit handling).

Your checklist for this pull request

  • Passed code styles and structures
  • Passed code linting checks and unit test

dependabot Bot and others added 30 commits April 30, 2026 14:55
Bumps [dieghernan/cff-validator](https://github.com/dieghernan/cff-validator) from 4 to 5.
- [Release notes](https://github.com/dieghernan/cff-validator/releases)
- [Changelog](https://github.com/dieghernan/cff-validator/blob/main/CHANGELOG.md)
- [Commits](dieghernan/cff-validator@114aae5...54c8e73)

---
updated-dependencies:
- dependency-name: dieghernan/cff-validator
  dependency-version: '5'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [DavidAnson/markdownlint-cli2-action](https://github.com/davidanson/markdownlint-cli2-action) from 23.1.0 to 23.2.0.
- [Release notes](https://github.com/davidanson/markdownlint-cli2-action/releases)
- [Commits](DavidAnson/markdownlint-cli2-action@v23.1.0...v23.2.0)

---
updated-dependencies:
- dependency-name: DavidAnson/markdownlint-cli2-action
  dependency-version: 23.2.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 4 to 4.35.4.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](github/codeql-action@v4...v4.35.4)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-version: 4.35.4
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
…idAnson/markdownlint-cli2-action-23.2.0

Bump DavidAnson/markdownlint-cli2-action from 23.1.0 to 23.2.0
Bumps [actions/stale](https://github.com/actions/stale) from 10 to 10.2.0.
- [Release notes](https://github.com/actions/stale/releases)
- [Changelog](https://github.com/actions/stale/blob/main/CHANGELOG.md)
- [Commits](actions/stale@v10...v10.2.0)

---
updated-dependencies:
- dependency-name: actions/stale
  dependency-version: 10.2.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
…hub/codeql-action-4.35.4

Bump github/codeql-action from 4 to 4.35.4
…ions/stale-10.2.0

Bump actions/stale from 10 to 10.2.0
…ghernan/cff-validator-5

Bump dieghernan/cff-validator from 4 to 5
Bumps [actions/stale](https://github.com/actions/stale) from 10.2.0 to 10.3.0.
- [Release notes](https://github.com/actions/stale/releases)
- [Changelog](https://github.com/actions/stale/blob/main/CHANGELOG.md)
- [Commits](actions/stale@v10.2.0...v10.3.0)

---
updated-dependencies:
- dependency-name: actions/stale
  dependency-version: 10.3.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 4.35.4 to 4.35.5.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](github/codeql-action@v4.35.4...v4.35.5)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-version: 4.35.5
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
…ions/stale-10.3.0

Bump actions/stale from 10.2.0 to 10.3.0
…hub/codeql-action-4.35.5

Bump github/codeql-action from 4.35.4 to 4.35.5
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 4.35.5 to 4.36.0.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](github/codeql-action@v4.35.5...v4.36.0)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-version: 4.36.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [peaceiris/actions-gh-pages](https://github.com/peaceiris/actions-gh-pages) from 4 to 4.1.0.
- [Release notes](https://github.com/peaceiris/actions-gh-pages/releases)
- [Changelog](https://github.com/peaceiris/actions-gh-pages/blob/main/CHANGELOG.md)
- [Commits](peaceiris/actions-gh-pages@v4...v4.1.0)

---
updated-dependencies:
- dependency-name: peaceiris/actions-gh-pages
  dependency-version: 4.1.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [actions/checkout](https://github.com/actions/checkout) from 6 to 6.0.2.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](actions/checkout@v6...v6.0.2)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: 6.0.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
…ions/checkout-6.0.2

Bump actions/checkout from 6 to 6.0.2
…ceiris/actions-gh-pages-4.1.0

Bump peaceiris/actions-gh-pages from 4 to 4.1.0
…hub/codeql-action-4.36.0

Bump github/codeql-action from 4.35.5 to 4.36.0
- Add bandit>=1.9.4 to the dev dependency group in pyproject.toml
- Add [tool.bandit] config section with skips for false positives and
  accepted risks (B105, B107, B110, B202, B310, B311, B614, B615)
- Add .github/workflows/bandit.yml workflow triggered on push/PR to
  both dev and main branches
- Add # nosec B324 to MD5 hash line in corpus/core.py (already had
  # noqa: S324 for ruff; this suppression is for bandit)

All issues accounted for: bandit now runs cleanly with 0 findings.
Add revision parameter to all user-supplied model loaders so that
callers can pin to a specific commit hash for reproducible and secure
downloads. Use # nosec B615 for module-level singletons with
hardcoded well-known model names where parameterization is impractical.
Remove B615 from the global bandit skips list.

Files with new revision parameter:
- pythainlp/corpus/core.py (get_hf_hub)
- pythainlp/generate/wangchanglm.py (WangChanGLM.load_model)
- pythainlp/lm/qwen3.py (Qwen3.load_model)
- pythainlp/parse/transformers_ud.py (Parse.__init__)
- pythainlp/parse/ud_goeswith.py (Parse.__init__)
- pythainlp/phayathaibert/core.py (PartOfSpeechTagger, NamedEntityTagger)
- pythainlp/summarize/mt5.py (mT5Summarizer.__init__)
- pythainlp/tag/pos_tag.py (pos_tag_transformers)
- pythainlp/translate/small100.py (Small100Translator.__init__)
- pythainlp/translate/th_fr.py (ThFrTranslator.__init__)
- pythainlp/translate/zh_th.py (ThZhTranslator, ZhThTranslator)
- pythainlp/wangchanberta/core.py (NamedEntityRecognition.__init__)

Files with # nosec B615 (hardcoded model names):
- pythainlp/augment/lm/phayathaibert.py
- pythainlp/augment/lm/wangchanberta.py
- pythainlp/phayathaibert/core.py (module-level singleton)
- pythainlp/spell/wanchanberta_thai_grammarly.py
- pythainlp/wangchanberta/core.py (hardcoded revision="main")

bandit now reports 0 B615 findings (11 lines suppressed via # nosec).
The revision parameter belongs on cached_file() (to fetch the right
version from HuggingFace Hub), not on AutoConfig.from_pretrained()
which receives the resulting local path. Add # nosec B615 there since
the revision is already handled upstream by cached_file().
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 4.36.0 to 4.36.2.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](github/codeql-action@v4.36.0...v4.36.2)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-version: 4.36.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [actions/checkout](https://github.com/actions/checkout) from 6.0.2 to 6.0.3.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](actions/checkout@v6.0.2...v6.0.3)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: 6.0.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
…ions/checkout-6.0.3

Bump actions/checkout from 6.0.2 to 6.0.3
…hub/codeql-action-4.36.2

Bump github/codeql-action from 4.36.0 to 4.36.2
@wannaphong wannaphong added this to the 6.0 milestone Jun 24, 2026
@wannaphong wannaphong linked an issue Jun 24, 2026 that may be closed by this pull request
@coveralls

coveralls commented Jun 24, 2026

Copy link
Copy Markdown

Coverage Status

coverage: 66.979% (+0.2%) from 66.741% — copilot/add-module-th-pron into dev

@wannaphong
wannaphong merged commit dbc5e3a into dev Jun 24, 2026
23 of 26 checks passed
@sonarqubecloud

Copy link
Copy Markdown

@wannaphong
wannaphong deleted the copilot/add-module-th-pron branch June 24, 2026 10:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add Module:th-pron from wiktionary

4 participants