Error Corrections and Suggestions for Improvement (6 items)

[iembry]

Hi Russ, I want to say thank you for the continuous improvement on the NumericEnsembles and related packages. I have some error corrections & suggestions for improvement.

1. In the NumericEnsembles Introduction vignette, you have the following code examples:

Example 1

> library(NumericEnsembles)
> Numeric(data = MASS::Boston,
        colnum = 14,
        numresamples = 2,
        remove_VIF_above = 5.00,
        remove_ensemble_correlations_greater_than = 100,
        scale_all_predictors_in_data = "N",
        data_reduction_method = 0,
        ensemble_reduction_method = 0,
        how_to_handle_strings = 0,
        predict_on_new_data = "N",
        stratified_random_sampling = "N",
        save_all_trained_models = "N",
        set_seed = "Y",
        save_all_plots = "N",
        use_parallel = "Y",
        train_amount = 0.60,
        test_amount = 0.20,
        validation_amount = 0.20)

which results in this error:

# Error in Numeric(data = MASS::Boston, colnum = 14, numresamples = 2, remove_VIF_above = 5,  :
#  unused argument (stratified_random_sampling = "N")

Example 2

> library(NumericEnsembles)
> Numeric(data = ISLR::Carseats,
+         colnum = 1,
+         numresamples = 2,
+         remove_VIF_above = 5.00,
+         remove_ensemble_correlations_greater_than = 1.00,
+         scale_all_predictors_in_data = "N",
+         data_reduction_method = 0,
+         ensemble_reduction_method = 0,
+         how_to_handle_strings = 2,
+         predict_on_new_data = "N",
+         save_all_trained_models = "N",
+         set_seed = "Y",
+         save_all_plots = "N",
+         use_parallel = "Y",
+         train_amount = 0.60,
+         test_amount = 0.20,
+         validation_amount = 0.20)
Which integer would you like to use for the seed? 12345

which results in this error:

Error in Numeric(data = ISLR::Carseats, colnum = 1, numresamples = 2,  :
  argument "stratified_random_column" is missing, with no default

2. When using the stratified_random_column parameter = 0 in the Numeric function with xgboost version 1.7.5.1 with the following code:

> library(NumericEnsembles)

> Numeric(data = MASS::Boston, colnum = 14, numresamples = 2, remove_VIF_above = 5.00, remove_data_correlations_greater_than = 0.99, remove_ensemble_correlations_greater_than = 100, scale_all_predictors_in_data = "N", data_reduction_method = 0, ensemble_reduction_method = 0, how_to_handle_strings = 0, predict_on_new_data = "N", set_seed = "N", save_all_trained_models = "N", save_all_plots = "N", use_parallel = "Y", stratified_random_column = 0, train_amount = 0.60, test_amount = 0.20, validation_amount = 0.20)

Resampling number 1 of 2,

Working on Bagging
Working on BayesGLM
Working on BayesRNN
Number of parameters (weights and biases) to estimate: 30
Nguyen-Widrow method
Scaling factor= 0.7015619
gamma= 29.2179   alpha= 4.7623   beta= 19818.09
Working on Cubist
Working on Earth
Working on Elastic
Working on Generalized Additive Models with Smoothing Splines
Working on Gradient Boosted
Using 100 trees...

Using 100 trees...

Using 100 trees...

Using 100 trees...

Using 100 trees...

Using 100 trees...

Working on Lasso
Working on Linear
Working on Neuralnet
# weights:  13
initial  value 579835.644333
iter  10 value 309875.230199
final  value 7381.250996
converged
Working on Partial Least Squares
Working on Principal Components Regression
Working on Ridge
Working on RPart
Working on Support Vector Machines
Working on Trees
Working on XGBoost

I get the following error:

Error: 'xgb.params' is not an exported object from 'namespace:xgboost'

3. You include R packages in a Depends statement rather than an Imports statement in the DESCRIPTION file. I offer the following articles with more information on the difference betwween the 2:

https://stackoverflow.com/questions/8637993/better-explanation-of-when-to-use-imports-depends
r - Better explanation of when to use Imports/Depends - Stack Overflow

https://r-pkgs.org/dependencies-in-practice.html
"R Packages" (2e): 11 Dependencies: In Practice
By Hadley Wickham and Jennifer Bryan

4. Instead of solely predicting on "new data", will you also add in the ability to forecast as well?

Your ForecastingEnsembles package does not work on daily data & I have had success with the training and validation of daily data (the time element is numeric) using the NumericEnsembles; however, it does not do any forecasting.

5. Instead of using message(), you may want to consider using either cat() or print(), see below:

> tempdir1 <- tempdir()
> tempdir1
[1] "/tmp/RtmpJtdnMX"

> message("The trained models are temporariliy saved in this directory: tempdir1. This directory is automatically deleted at the end of the R session.\n          You may save the trained models before you end this session if you chose to do so.")
The trained models are temporariliy saved in this directory: tempdir1. This directory is automatically deleted at the end of the R session.
          You may save the trained models before you end this session if you chose to do so.
          
> cat("Current tempdir(): ", tempdir(), "\n")
Current tempdir():  /tmp/RtmpJtdnMX

> print(tempdir())
[1] "/tmp/RtmpJtdnMX"

6. In order to run the Numeric function within a Markdown document with set_seed & predict_on_new_data, is it possible to create a Numeric function that allows the direct setting of an integer value for set_seed in the function call and also the direct setting of the data URL for predict_on_new_data too, i.e. rather than using the readline function call?

Thank you.

Irucka

[InfiniteCuriosity]

Thank you very much for the comments - they are the kind of comments are very helpful!

1. In the NumericEnsembles Introduction vignette, you have the following code examples:

Thank you for catching this. I changed the name of the stratified random column in the function, and forgot to go back and update it in the vignette. This is fixed, the vignette runs without any errors using the current version of NumericEnsembles (1.0.0)

2. Error in Numeric(data = ISLR::Carseats, colnum = 1, numresamples = 2, :
   argument "stratified_random_column" is missing, with no default:

Same issue, which is fixed now. Runs without any errors using the current version of NumericEnsembles (1.0.0)

2a Error: 'xgb.params' is not an exported object from 'namespace:xgboost'

There was an update to XGBoost at nearly the exact same time I did an update to NumericEnsembles. The good news is NumericEnsembles has been updated to reflect the updates in XGBoost, and everything runs without any errors (including XGBoost!) in the current version of NumericEnsembles (1.0.0)

3. You include R packages in a Depends statement rather than an Imports statement in the DESCRIPTION file. I offer the following articles with more information on the difference betwween the 2:

Thank you! My initial understanding was that the package needed to use Depends, since it uses every package. I will check the articles and improve NumericEnsembles.

4. Instead of solely predicting on "new data", will you also add in the ability to forecast as well?

I will be happy to work on this! At this time the ForecastingEnsembles package is mainly designed to work best with data from the United States Department of Labor. As a result, it is currently less mature than NumericEnsembles, ClassificationEnsembles and LogisticEnsembles, and ForecastingEnsembles is my next project to update. ForecastingEnsembles still has work to be done, but the other three are virtually complete now.

Your point about ForecastingEnsembles not working well with daily data is correct (of course), is because it was not designed (yet) to work with that frequency of data. Everything in ForecastingEnsembles is based on the great work of Rob Hyndman, and this is part of what he writes:

Daily and sub-daily (such as hourly) data are challenging for a different reason — they often involve multiple seasonal patterns, and so we need to use a method that handles such complex seasonality.
Source: https://otexts.com/fpp3/weekly.html

My honest goal is to get ForecastingEnsembles (or any of my other packages) to predict accurately on daily and sub-daily data (since I can see a HUGE need to do this!), and still have all the power of a great forecast (seasonality, trends, cycles, and much more, that are very helpful in forecasting).

One other possibility I'm considering (in light of the great question!) is to offer a light version of forecasting in NumericEnsembles, and the full power of all the forecasting tools in ForecastingEnsembles, but that remains to be seen.

5. Instead of using message(), you may want to consider using either cat() or print(), see below:

My recollection is that my very first submittal to CRAN used print, and CRAN asked me to change to message:

(this is from CRAN to me) "Instead of print() rather use message()/warning() or if(verbose)cat(..) (or maybe stop()) if you really have to write text to the console. (except for print, summary, interactive functions)"

More info: message

6 In order to run the Numeric function within a Markdown document with set_seed & predict_on_new_data, is it possible to create a Numeric function that allows the direct setting of an integer value for set_seed in the function call and also the direct setting of the data URL for predict_on_new_data too, i.e. rather than using the readline function call?

I will experiment with setting up NumericEnsembles to be used on Markdown documents and share updates. While the NumericEnsembles package is complete, I want to extend the functionality so it can be used in a wide range of situations, such as a Markdown or Quarto document, and a few excellent situations I'm working on. Preliminary tests are very good for these new situations, and I will post updates.

I want to say your comments are excellent, and greatly appreciated. Please continue to make other comments as you wish, this is how the field gets better. Thank You!!

Russ Conte
Forest Park, Illinois

[iembry]

Hi Russ, I haven't had a chance to read over all of you responses yet; however, I did re-run the examples in the vignette and you can view the PDF document to see the full R code with the error messages:

NumericEnsembles-1.0.0-Examples.pdf

Also, I have another suggestion for you:

Instead of "data can be a CSV file or within an R package, such as MASS::Boston"; I suggest something like

data can be a CSV file or an R object, such as a data.frame/data.table/tibble, list, or matrix

Thank you.

Irucka

[InfiniteCuriosity]

Hi Irucka,

My apologies for the delay, I was informed of a change in one of the packages that NumericEnsembles depends upon that will break NumericEnsembles, so I had to get that fixed (and I'm uploading the new version today (Thursday February 26, 2026)) so the problem will never impact any users of NumericEnsembles.

Thank you very much for the detailed examples, I wish more people provided detailed examples!

Good news - the error from the vignette is from a prior version of NumericEnsembles that was fixed and posted to CRAN a few days ago. I just downloaded NumericEnsembles 1.0.0 from CRAN, and the code in the vignette ran perfectly (so did lots of other code, too!)

I'm uploading a new version to CRAN today, and the vignette (and everything else) runs great!

On the second suggestion:

NumericEnsembles does not run if the data is a tibble, list, or matrix.

A tibble returns this error (basically the XGBoost models do not run, and it crashes everything)

Error in .internal.setinfo.xgb.DMatrix(object, name, info) : 
  [08:44:47] xgboost_R.cc:167: Array or matrix has unsupported type.
Stack trace:
  [bt] (0) 1   xgboost.so                          0x0000000113231e3c dmlc::LogMessageFatal::~LogMessageFatal() + 124
  [bt] (1) 2   xgboost.so                          0x000000011321f028 (anonymous namespace)::MakeArrayInterfaceFromRVector(SEXPREC*) + 232
  [bt] (2) 3   xgboost.so                          0x000000011321ee14 XGDMatrixSetInfo_R + 84
  [bt] (3) 4   libR.dylib                          0x00000001067539f4 R_doDotCal

A matrix returns this error, which crashes everything:

Error in UseMethod("relocate") : 
  no applicable method for 'relocate' applied to an object of class "c('matrix', 'array', 'double', 'numeric')"

A list returns this error, which crashes everything:

Error in `colnames<-`(`*tmp*`, value = `*vtmp*`) : 
  attempt to set 'colnames' on an object with less than two dimensions

NumericEnsembles is optimized to work with data frames, so my recommendation is to convert a tibble, list or matrix into a data frame first, then run it in NumericEnsembles, though I am very open to any other recommendations!

Thank you very much, please keep the comments and recommendations coming, that is how open source gets better!

Best wishes,

Russ Conte

[iembry]

Hi Russ, thank you for your response & for updating your packages.

For data, you can use the following language, if you wish:

data can be a CSV file or an R object, such as a data.frame/data.table

Regarding the newest version and a new error message, please see the following:

NumericEnsembles-1.0.3-Examples.pdf

Irucka

[InfiniteCuriosity]

Hi Irucka,

THANK YOU (again!) for your help and pointing out this error! I was able to reproduce the error, so I was able to fix it. My apologies this was missed in the testing. I just submitted a corrected version to CRAN (version 1.0.5) that fixes the issue with set_seed. I tested version 1.0.5 a few times, and NumericEnsembles consistently passed, and ran without any errors.

Not sure how long before version 1.0.5 will be posted to CRAN, but it should not be too long.

Thank you again very much for your help, it is truly making this a much better solution!!

Russ Conte