Fix encoded_missing_value being ignored and all-missing datetime columns crashing#76
Open
devYRPauli wants to merge 2 commits into
Open
Conversation
devYRPauli
requested review from
abhidas,
erzel,
rajatsen91,
siriuz42,
tamannarayan and
weihaokong
as code owners
July 20, 2026 15:11
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two independent edge case fixes in
tabfm/src/classifier_and_regressor.py, one per commit.1.
CategoricalOrdinalEncoderignoresencoded_missing_valueencoded_missing_valueis accepted in__init__and documented as "Encoded value used for NaN / missing values", buttransform()never reads it. Missing values and unseen categories are both filled withunknown_value, so the parameter has no effect.The fix records the original NaN positions before mapping and fills those with
encoded_missing_value, leavingunknown_valuefor values genuinely absent from the category map. Both parameters default to-1, so behaviour is unchanged unless a caller sets them differently.2.
DatetimeTransformercrashes on an all-missing datetime columnfit()storesseries.mean()as the fill value. For a column with no valid timestamps that mean isNaT, so the laterfillnais a no-op and the subsequent.astype(np.int64)on the extracted parts raises:The fix falls back to the epoch when the computed mean is
NaT. Columns with at least one valid timestamp are unaffected.Tests
Added
CategoricalOrdinalEncoderTestandDatetimeTransformerTesttoclassifier_and_regressor_test.py. Both were confirmed to fail against the unpatched source (-1.0vs-99for the first,IntCastingNaNErrorfor the second) and pass with the change.Full existing suite after the change: 48 tests run, 20 passed, 28 skipped because JAX is not installed in my environment. Those 28 remain unverified locally.
These changes do not touch any line modified by my open PRs #59 or #69.