It is possible to ask Suachi to return only a subset of fields in the WordInfo.
To do that, you can use the fields parameter of the :py:meth:`sudachipy.Dictionary.tokenizer()` method.
The parameter accepts a set of strings, each one representing a field to be returned.
By default, all fields are returned.
Allowed values:
surface: in-dictionary surface word form.- :py:meth:`sudachipy.Morpheme.surface()` method returns the slice of the input text and is not affected by that flag.
posorpos_id: part-of-speech tag.normalized_formdictionary_formreading_formword_structuresynonym_group_idsuser_datasplit_asplit_b
Note
If you want only tokenization (e.g. use only :py:meth:`sudachipy.Morpheme.surface()`, passing empty set is allowed.
You need to load splits if you want to use :py:meth:`sudachipy.Morpheme.split` method. If performing the tokenization with non-default mode, the required splits will be loaded automatically and can be omitted.:
dic.tokenizer(SplitMode.B, fields=set()) # implicitly becomes fields={'split_b'}
To access user-defined dictionary data when field subsetting is enabled, load
user_data explicitly.:
tokenizer = dic.tokenizer(fields={"surface", "user_data"})
morpheme = tokenizer.tokenize("すだち")[0]
assert morpheme.user_data() == "徳島県産"
If fields is specified without user_data, :py:meth:`sudachipy.Morpheme.user_data()`
follows the same partial-loading rule as other fields and may return an incorrect result.
Warning
Using a field not included in the passed subset will produce an incorrect result without any warning. Use tests to ensure that the needed fields are loaded when using this parameter.