append_stats() adds or recalculates statistics on
existing ALE objects, using saved bootstrap data and a
complete outcome reference when needed; append_p() adds or
replaces p-values on ALE objects or ModelBoot
objects with retained bootstrap data, using an explicitly supplied
ALEpDist distribution and refreshing ALER bands (#53).summary() methods have been implemented for
ALE and ModelBoot objects. They print a
summary of ALE statistics to the console. When there are no ALE
statistics available, they print a message saying so.show_2d_bin_sizes logical argument
ALEPlots(), the 2D ALE plots display of hollow bin-size
squares and a % data legend is now optional. (#44)aled_fun argument of the ALE()
and ALEpDist() constructors, we may optionally choose to
calculate ALE deviation based on standard deviations or variances
instead of the default mean absolute deviation.min_col_widths argument of
the plot() method for ALE.consolid_cats argument of the plot()
method for ALE). The top consolid_cats - 1
values in ALE strength are shown and all other values are consolidated
into an “other” category."levels" of the
fct_order argument to the ALE() constructor,
unordered factors are now sorted in the order of the factor levels
(characters columns are sorted in alphabetical order of their unique
values). An alternative "y_col" sorts based on the
increasing mean values of the predictions of the outcome variable for
each factor level. The "ksd" option allows compatibility
with the original ALEPlot reference implementation.max_num_bins argument to the ALE()
constructor now accepts a special list format that allows the
specification of per-column maximum ALE bin sizes for numeric input
columns, with dimension-specific defaults for columns not named. For
details, see documentation for ALE().ale.parallel. For example, you can set 4 CPU cores with
options(ale.parallel = 4).{ranger} package is automatically recognized so
that y_col and pred_fun don’t need to be
specified. In the future, a very few other very popular packages and
frameworks will also be automatically recognized, but there will be no
attempt to cover most packages: pred_fun remains the
general way to customize predict functions for packages that don’t work
out of the box.ALE(), ALEpDist(), and
ModelBoot() run locally when parallel = 0 or a
batch has at most one job, avoiding repeated future preparation inside
model and random-variable iterations (#noissue). They thus execute
faster than before.We have dealt with innumerable bugs during our development journey but, fortunately, very few publicly signalled bugs. Only fixes for publicly reported bugs are indicated here.
ALE() now warns early about requested predictors with
only one observed value and returns zero main and interaction effects
for them, including bootstrap iterations. (#43)nnet::nnet(), are now accepted and named
after the outcome column. (#41)get() for ALE and ModelBoot
objects now returns named statistics in the requested order within each
term. (#40)get() now raises a clear error when confidence regions
are requested without p-values. summary() for
ALE and ModelBoot objects silently omits
confidence-region sections when p-values are absent. (#38)customize() calls with zoom_x or
zoom_y now preserve the existing limits of the other axis.
(#37)ALE() now checks the outcome column and prediction type
when accepting an ALEpDist distribution, in addition to the
model identity. ALEpDist retains the original prediction
settings, including for surrogates. (#36)parallelly::availableCores() directly in
preparation for its re-export being removed from {future}.
The {future} and {parallelly} packages are now
suggested dependencies because both are required dependencies of the
imported {furrr} package (#18).ALE(), ALEpDist(), and
ModelBoot() use reproducible iteration streams with
matching R random inputs in serial and parallel execution, and restore
the caller’s RNG kinds and full seed state. The new stream policy
changes historical random results (#noissue).
ModelBoot() completes the full-data calculation
before bootstrapping and passes its canonical ALE bins explicitly to
every iteration, giving serial and parallel bootstrap calculations the
same grids (#noissue).
ALE(), ModelBoot(),
ALEPlots(), their get() methods, and
customize() expose complete defaults for fixed string
choices and match them exactly with rlang::arg_match().
Nullable and mixed-type arguments retain their existing validation and
forms, and coordinated selector behaviour is preserved.
x_medoids() also matches its choices exactly; abbreviated
sort_by values such as “sil” must now be written in full as
“silhouette” (#noissue).
max_num_bins defaults to
list(d1 = 1000L, d2 = 10L) and supports independent
per-column except_d1 and except_d2 limits.
Explicit lists need defaults or exceptions only for the numeric
variables requested in each dimension. Use list() for
non-numeric effects; NULL, scalar inputs, and the former
default and except elements are
rejected.
The require_same_p argument of ALE()
has been renamed to require_p_model_match without a
compatibility alias. Setting it to FALSE disables all
p-value compatibility checks.
max_num_bins now creates n+1 bins
instead of exactly the specified n bins. The bottom bin is
reserved for the minimum value in the data. This is a reversion to the
original behaviour of the reference {ALEPlot} package.
Parallelization has been disabled by default
(parallel = 0) (#16).
The pred_fun argument in ALE() and
other constructors now defaults to NULL. However, the
functionality is unchanged: the default still creates the same generic
custom prediction function as before.
customize() function for ALEPlots: “Customizing
ALE plots”.vignette('ale-intro') have been removed from the CRAN
package and are now available only on the development website at https://tripartio.github.io/ale/articles/.We now use OpenAI Codex to help development and maintenance. Codex changes are not specifically highlighted in this change log, but numerous PRs are heavily assisted by Codex.
retrieve_rds() is a documentation utility function to
either retrieve a serialized RDS file from a URL or else run a code
block if the URL retrieval fails. It is used in vignettes, examples and
other documentation to create or retrieve large objects that are too
slow to be created on the spot.ALEPlots by appending ggplot
layers with the customize() function.invert_probs() inverts probabilities
(subtracts from 1) for ALE and ALEpDist
objects.We have deeply rethought the vision of this package and have completely rewritten the entire package to support existing, new, and future planned functionality. The changes are so radical that there is no continuity with the previous version 0.3.1. Thus, we’ve skipped a version number and now are at version 0.5.0.
Honestly, we can’t keep track of all the changes; experienced users are advised to rerun the vignettes to get up to speed with the new version. We apologize for the discontinuity but we trust that the latest version is easier to use and much more functional. What follows is a list of some of the most notable changes we’ve kept track of.
ale package objects has been completely rewritten. The
latest objects are not compatible with earlier versions. However, the
new structure supports the roadmap of future functionality, so we hope
that there will be minimal changes in the future that interrupt backward
compatibility.{S7} classes to represent
different kinds of ale package objects:
ALE: the core ale package object that
holds ALE data for a model (replaces the former ale() and
ale_ixn() functions).ModelBoot: results of full-model bootstrapping
(replaces the former model_bootstrap() function).ALEPlots: store ALE plots generated from either
ALE or ModelBoot with convenient
print() and plot() methods.ALEpDist: p-value distribution information (replaces
the former create_p_dist() function).{ALEPlot} package code and so now claim full authorship of
the code. One of the most significant implications of this is that we
have decided to change the package license from the GPL 2 to MIT, which
permits maximum dissemination of our algorithms.ale_ixn() has been eliminated and now both 1D and 2D
ALE are calculated with the ALE() constructor.ALE object constructor no longer produces plots
directly. ALE plots are now created as ALEPlots objects
using the newly added plot() methods that create all
possible plots from the ALE data from ALE or
ModelBoot objects. Thus, serializing ALE
objects now avoids the previous problems of environment bloat of the
included ggplot objects.rug_sample_size argument of the
ALE constructor to sample_size. Now it
reflects the size of data that should be sampled in the
ale object, which can be used not only for rug plots but
for other purposes.x_cols argument in ALE() now supports
a complex syntax for specifying which specific columns for 1D ALE or
pairs of columns for 2D interactions are desired. It also supports
specification using standard R formula syntax.get() methods now provide convenient access to
ALE, ModelBoot, and ALEPlots
objects.plot() methods, eliminated the
compact_plots to ale().print() and plot() methods have been added
to the ale_plots object.print() method has been added to the ALE
object.model_bootstrap() has added various model performance
measures that are validated using bootstrap validation with the .632
correction.p_funs has been completely changed; it
has now been converted to an object named ale_p and the
functions are separated from the object as internal functions. The
function create_p_funs() has been renamed
create_p_dist().ALEpDist() now produces three types of p-values:
“exact” (very slow) with at least 1000 random iterations on the original
model; “approx” for 100 to 999 iterations on the original model; and
“surrogate” for much faster but less reliable p-values based on a
surrogate linear model. See ALEpDist() for details.We have dealt with innumerable bugs during our development journey but, fortunately, very few publicly signalled bugs. Only fixes for publicly reported bugs are indicated here.
One of the most fundamental changes is not directly visible but
affects how some ALE values are calculated. In certain very specific
cases, the ALE values are now slightly different from those of the
reference {ALEPlot} package. These are only for
non-numerical variables for some prediction types other than predictions
scaled on the response variable. (E.g., a binary or categorical variable
for a logarithmic prediction not scaled to the same scale as the
response variable.) We made this change for two reasons:
{ALEPlot} implementation. These cases are not covered at
all in the base ALE scientific article and they are poorly documented in
the {ALEPlot} code. We cannot help users to interpret
results that we do not understand ourselves.{ALEPlot}
reference implementation is not scalable: custom code must be written
for each type and each degree of interaction.Other than for these edge cases, our implementation continues to give
identical results to the reference {ALEPlot} package.
Other notable changes that might not be readily visible to users:
{staccuracy}.{rlang} and
{cli} packages. Reduced the imported functions to a
minimum.{cli}.{assertthat} with custom validation functions
that adapt some {assertthat} code.helper.R test files so that some testing objects
are available to the loaded package.{future} parallelization code to restore
original values on exit.ale_p objects.{ggplot2}
3.5.{covr}.The most significant updates are the addition of p-values for the ALE statistics, the launching of a pkgdown website which will henceforth host the development version of the package, and parallelization of core functions with a resulting performance boost.
One of the key goals for the ale package is that it
would be truly model-agnostic: it should support any R object that can
be considered a model, where a model is defined as an object that makes
a prediction for each input row of data that it is provided. Towards
this goal, we had to adjust the custom predict function to make it more
flexible for various kinds of model objects. We are happy that our
changes now enable support for tidymodels objects and
various survival models (but for now, only those that return
single-vector predictions). So, in addition to taking required
object and newdata arguments, the custom
predict function pred_fun in the ale()
function now also requires an argument for type to specify
the prediction type, whether it is used or not. This change
breaks previous code that used custom predict functions, but it allows
ale to analyze many new model types than before. Code that
did not require custom predict functions should not be affected by this
change. See the updated documentation of the ale() function
for details.
Another change that breaks former code is that the arguments for
model_bootstrap() have been modified. Instead of a
cumbersome model_call_string,
model_bootstrap() now uses the
{insight} package to automatically detect many R models and
directly manipulate the model object as needed. So, the second
argument is now the model object. However, for non-standard
models that {insight} cannot automatically parse, a
modified model_call_string is still available to assure
model-agnostic functionality. Although this change breaks former code
that ran model_bootstrap(), we believe that the new
function interface is much more user-friendly.
A slight change that might break some existing code is that the
conf_regions output associated with ALE statistics has been
restructured. The new structure provides more useful information. See
help(ale) for details.
pkgdown website located
at https://tripartio.github.io/ale/. This is where
the most recent development features will be documented.create_p_funs() function for details and an
example.vignette('ale-statistics') for
details. The vignette has been expanded with more details on how to
properly interpret normalized ALE statistics.vignette('ale-statistics') for details.{furrr} library. In our tests, practically, we typically
found speed-ups of n – 2 where n is the number
of physical cores (machine learning is generally unable to use logical
cores). For example, a computer with 4 physical cores should see at
least ×2 speed-up and a computer with 6 physical cores should see at
least ×4 speed-up. However, parallelization is tricky with our
model-agnostic design. When users work with models that follow standard
R conventions, the ale package should be able to
automatically configure the system for parallelization. But for some
non-standard models users may have to explicitly list the model’s
packages in the new model_packages argument so that each
parallel thread can find all necessary functions. This is only a concern
if you get weird errors. See help(ale) for details.ale() function. See
help(ale) for details.median_band_pct argument to ale() now
takes a vector of two numbers, one for the inner band and one for the
outer.{gridExtra} with {patchwork} for
examples and vignettes for printing plots.ale() function documentation from
ale-package documentation.alt tags to describe plots for accessibility.{insight} package to automatically detect
y_col and model call objects when possible; this increases the range of
automatic model detection of the ale package in
general.{progressr} package for
progress bars. With the cli progression handler, this
enables accurate estimated times of arrival (ETA) for long procedures,
even with parallel computing. A message is displayed once per session
informing users of how to customize their progress bars. For details,
see help(ale), particularly the documentation on progress
bars and the silent argument.{ggplot2} from a dependency to an import. So, it
is no longer automatically loaded with the package.var_summary()
function. In particular, encodes whether the user is using p-values
(ALER band) or not (median band).validation.R file.compact_plots to plotting functions
to strip plot environments to reduce the size of returned objects. See
help(ale) for details.package_scope environment.ale_ixn()).ale_ixn()).ale() does not yet support multi-output model
prediction types (e.g., multi-class classification and multi-time
survival probabilities).This version introduces various ALE-based statistics that let ALE be used for statistical inference, not just interpretable machine learning. A dedicated vignette introduces this functionality (see “ALE-based statistics for statistical inference and effect sizes” from the vignettes link on the main CRAN page at https://CRAN.R-project.org/package=ale). We introduce these statistics in detail in a working paper: Okoli, Chitu. 2023. “Statistical Inference Using Machine Learning and Classical Techniques Based on Accumulated Local Effects (ALE).” arXiv. https://doi.org/10.48550/arXiv.2310.09877. Please note that they might be further refined after peer review.
ale() and model_bootstrap() now output these
statistics. (ale_ixn() will come later.)ale package with
the reference {ALEPlot} package:
“Comparison between {ALEPlot} and ale
packages” (available from the vignettes link on the main CRAN page at https://CRAN.R-project.org/package=ale).var_cars is a modified version of mtcars that features
many different types of variables.census is a polished version of the adult income
dataset used for a vignette in the {ALEPlot} package.silent = TRUE to
ale(), ale_ixn(), or
model_bootstrap().seed argument to ale(),
ale_ixn(), or model_bootstrap().By far the most extensive changes have been to assure the accuracy and stability of the package from a software engineering perspective. Even though these are not visible to users, they make the package more robust with hopefully fewer bugs. Indeed, the extensive data validation may help users debug their own errors.
{assertthat} package; if
not, the function fails quickly with an appropriate error message.{testthat} package is now used for testing
the outputs of each user-facing function. This should help the code base
to be more robust going forward with future developments.{ALEPlot} package. These
tests should ensure that any future code that breaks the accuracy of ALE
calculations will be caught quickly.ale_ixn()).ale_ixn()).This is the first CRAN release of the ale package. Here
is its official description with the initial release:
Accumulated Local Effects (ALE) were initially developed as a model-agnostic approach for global explanations of the results of black-box machine learning algorithms. (Apley, Daniel W., and Jingyu Zhu. “Visualizing the effects of predictor variables in black box supervised learning models.” Journal of the Royal Statistical Society Series B: Statistical Methodology 82.4 (2020): 1059-1086 doi:10.1111/rssb.12377.) ALE has two primary advantages over other approaches like partial dependency plots (PDP) and SHapley Additive exPlanations (SHAP): its values are not affected by the presence of interactions among variables in a model and its computation is relatively rapid. This package rewrites the original code from the ‘ALEPlot’ package for calculating ALE data and it completely reimplements the plotting of ALE values.
(This package uses the same GPL-2 license as the
{ALEPlot} package.)
This initial release replicates the full functionality of the
{ALEPlot} package and a lot more. It currently presents
three functions:
ale(): create data for and plot one-way ALE (single
variables). ALE values may be bootstrapped.ale_ixn(): create data for and plot two-way ALE
interactions. Bootstrapping of the interaction ALE values has not yet
been implemented.model_bootstrap(): bootstrap an entire model, not just
the ALE values. This function returns the bootstrapped model statistics
and coefficients as well as the bootstrapped ALE values. This is the
appropriate approach for small samples.This release provides more details in the following vignettes (they are all available from the vignettes link on the main CRAN page at https://CRAN.R-project.org/package=ale):
ale packageale() function handling of various datatypes for xThe version numbering system for the {ale} package is an adaptation of the standard three-number system. So, in a version number of M.m.b (e.g., 0.5.3), the numbers mean:
That said, major version 0 has special meaning: it indicates that the package is in beta, that is, still in experimental status, meaning that a stable interface has not yet been settled. New features might be written that break the functionality of previous 0.x versions. That said, every minor version is tested with {revdepcheck}. A notification or pull request will be sent to the maintainers of any packages whose functionality might be broken due to a change in {ale} functionality.
Normally, if new functionality is added along with bug fixes, then the bug fixes will be released with the next minor version. However, with the major version 0, small changes in functionality will sometimes be released in a bug-fix version.
The numbering system described above is used for packages released to CRAN. For development versions in between CRAN releases, the {ale} package adds a fourth component D, that is, an M.m.b.D format. The D is in YYYYMMDD date format, indicating the release date of the latest development version after the latest CRAN release. So, version 0.5.3.20260217 is a development version released on February 17, 2026, after the 0.5.3 CRAN version. However, as the specific date keeps on changing, the latest development version is simply indicated in the change log below as “ale (development version)”.