Skip to content

Fix dtype of empty feature shuffle patterns#73

Open
Echo-Nie wants to merge 1 commit into
google-research:mainfrom
Echo-Nie:main
Open

Fix dtype of empty feature shuffle patterns#73
Echo-Nie wants to merge 1 commit into
google-research:mainfrom
Echo-Nie:main

Conversation

@Echo-Nie

Copy link
Copy Markdown

Problem

Thx for open-sourcing TabFM — really appreciate the work!

While running TabFM on the KC1 dataset with an n-shot retrieved context, predict_proba crashes with:

IndexError: arrays used as indices must be of integer (or boolean) type

The relevant traceback is:

File "tabfm/src/classifier_and_regressor.py", line 1630, in _transform_features
    shuffled_cols = X_variant_instance[:, shuffle_pattern]
IndexError: arrays used as indices must be of integer (or boolean) type

The shuffle patterns are converted with:

return [np.array(p) for p in shuffle_patterns]

When p is empty, NumPy infers float64. The resulting array is later used for column indexing, which requires an integer or boolean dtype.

Fix

Convert shuffle patterns using NumPy's native indexing dtype:

- return [np.array(p) for p in shuffle_patterns]
+ return [np.asarray(p, dtype=np.intp) for p in shuffle_patterns]

This preserves the values and ordering of non-empty patterns while making empty patterns valid NumPy index arrays.

Reproduce

import numpy as np

shuffle_pattern = np.array([])
X = np.empty((3, 0))

print(shuffle_pattern.dtype)  # float64
X[:, shuffle_pattern]         # IndexError before this change

After this change:

shuffle_pattern = np.asarray([], dtype=np.intp)
X = np.empty((3, 0))

result = X[:, shuffle_pattern]
print(result.shape)  # (3, 0)

Test

Ran the existing classifier and regressor tests:

pytest -q tabfm/src/classifier_and_regressor_test.py

18 passed, 28 skipped in 3.06s

@google-cla

google-cla Bot commented Jul 20, 2026

Copy link
Copy Markdown

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@Echo-Nie

Copy link
Copy Markdown
Author

The PR is ready to be reviewed, thx!
@weihaokong

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant