[github-actions]

✨ FAQ NEW

Course: machine-learning-zoomcamp
Section: module-3-homework (This is clearly about Module 3’s classification homework (mutual information + categorical features), so it belongs in the Module 3 homework section.)
Related Issue: #468

Question

Homework 3 (2026) Q3: Should I use mutual_info_classif or mutual_info_score for categorical MI? Why do my results change with random_state?

Decision Rationale

The proposed question asks which mutual-information function to use for HW3 Q3 and explains why results differ; none of the existing FAQs cover the mutual_info_score vs mutual_info_classif choice or the specific tie/seed behavior.

Placement Details

• Section ID: module-3-homework
• Sort Order: End of section
• Filename Slug: hw3-q3-mutual-info-score-vs-classif

---

🤖 Generated by FAQ Bot

Closes #468

[saeidsoheily]

For Homework 3 (2026) Q3, use sklearn.metrics.mutual_info_score, matching the Module 3 lesson.
It computes mutual information directly between two discrete variables, so for categorical features it is deterministic:

from sklearn.metrics import mutual_info_score
mi = {c: round(mutual_info_score(y_train, df_train[c]), 2) for c in categorical}

mutual_info_classif is a different estimator intended for feature selection. Depending on how the input is represented, it may use an approximate nearest-neighbor method, which can make results depend on random_state.

Two common causes of unexpected results are:

1. Label-encoded categorical values treated as continuous. If categories are converted to integers and passed in a dense matrix, sklearn may interpret them as numerical continuous features. Then the estimated MI can vary with the random seed.
2. Computing MI after one-hot encoding. With get_dummies() or DictVectorizer, each category becomes a separate binary feature. Then you are measuring MI for individual dummy columns rather than for the original categorical variable requested in Q3.

So for this homework, compute MI on the original categorical columns, using only the training set:

from sklearn.metrics import mutual_info_score

for col in categorical_features:
    score = mutual_info_score(df_train[col], y_train)
    print(f"{col}: {score:.4f}")

[alexeygrigorev]

Thanks! Merged into main with edits in e2dfd0b: removed the leftover sentence about a solution repo link.