From cd4862a0a01410f87ee54c0cbbacf6fe95e1e20d Mon Sep 17 00:00:00 2001 From: FAQ Bot Date: Mon, 19 Oct 2026 20:39:39 +0000 Subject: [PATCH] NEW: Why should we use the same KFold splits when comparing different hyperpa --- ...07_kfold-same-splits-hyperparameter-comparison.md | 12 ++++++++++++ 1 file changed, 12 insertions(+) create mode 100644 _questions/machine-learning-zoomcamp/module-4/031_8e58068607_kfold-same-splits-hyperparameter-comparison.md diff --git a/_questions/machine-learning-zoomcamp/module-4/031_8e58068607_kfold-same-splits-hyperparameter-comparison.md b/_questions/machine-learning-zoomcamp/module-4/031_8e58068607_kfold-same-splits-hyperparameter-comparison.md new file mode 100644 index 00000000..0cf06c50 --- /dev/null +++ b/_questions/machine-learning-zoomcamp/module-4/031_8e58068607_kfold-same-splits-hyperparameter-comparison.md @@ -0,0 +1,12 @@ +--- +id: 8e58068607 +question: Why should we use the same KFold splits when comparing different hyperparameter + values (e.g., different Logistic Regression `C` values)? +sort_order: 31 +--- + +When comparing different hyperparameter values (like Logistic Regression’s `C`), you want the score differences to reflect the hyperparameter effect—not randomness from changing train/validation splits. + +If you use the same `KFold` configuration (for example, `KFold(n_splits=5, shuffle=True, random_state=1)`), then each hyperparameter value is evaluated on the same fold assignments. That makes the comparison fairer because every value of `C` sees the same subsets as validation across the cross-validation runs. + +If you instead generate different random splits for each hyperparameter value, the mean AUC (or accuracy) can shift partly due to the particular split rather than due to the hyperparameter itself. Reusing the same folds also improves reproducibility, since rerunning the experiment gives the same fold partitions. \ No newline at end of file