---
title: "Kolmogorov-Arnold networks with interchangeable edge bases"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Kolmogorov-Arnold networks with interchangeable edge bases}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r, include = FALSE}
knitr::opts_chunk$set(collapse = TRUE, comment = "#>")
```

```{r}
library(thangldKANr)
```

## One engine, many edge types

A Kolmogorov-Arnold network computes, in each layer,
$h^{(l+1)}_j = \sum_i \phi_{j,i}(h^{(l)}_i)$. The variants published since
Liu et al. (2024) differ only in how an edge function $\phi_{j,i}$ is
written: a Chebyshev, Legendre or Jacobi expansion of $\tanh x$, Gaussian
radial basis functions (FastKAN), Mexican-hat wavelets (Wav-KAN), safe Pade
rational functions, a hybrid SiLU + Chebyshev + RBF sum, or squared ReLU
bumps (ReLU-KAN). thangldKANr keeps one engine (training loop with both
backends, preprocessing, early stopping, prediction, edge plots,
persistence, validation) and treats the edge type as an argument.

```{r}
kan_bases()[, c("name", "label")]
```

Layers are stacked; interactions between inputs arise through composition,
so a one-layer network is an additive model and should be reported as such.

## Fitting and comparing bases

Inputs and response are standardized with constants estimated on the
training rows; predictions are returned on the original scale. The base R
backend (`backend = "r"`) is used here so the vignette builds without
libtorch; `backend = "torch"` (the default) uses R 'torch' autograd and
follows the same trajectory in double precision.

```{r}
set.seed(1)
n <- 400
x <- matrix(runif(2 * n, -2, 2), n, 2, dimnames = list(NULL, c("x1", "x2")))
f <- function(x) x[, "x1"] * x[, "x2"] + sin(x[, "x1"])
y <- f(x) + rnorm(n, sd = 0.05)
train <- 1:240
valid <- 241:300
test <- 301:400
fit_one <- function(basis, ...) {
  thangld_kan_fit(thangld_kan(c(2, 4, 1), basis = basis, ..., seed = 1),
                x[train, ], y[train],
                validation_data = list(x = x[valid, ], y = y[valid]),
                epochs = 150, batch_size = 32, lr = 0.02, patience = 30,
                seed = 1, backend = "r")
}
fits <- list(chebyshev = fit_one("chebyshev", degree = 4),
             rbf = fit_one("rbf"),
             relu = fit_one("relu"))
t(sapply(fits, function(m) thangld_kan_metrics(y[test], predict(m, x[test, ]))))
```

The epoch with the lowest validation loss is restored; the test rows were
not used for any choice. This small example is not a benchmark: the
ranking of bases depends on the data, the width, the basis size and the
training budget.

```{r, fig.width = 6, fig.height = 4}
plot(fits$chebyshev, type = "loss")
plot(fits$rbf, layer = 1, input = 1, output = 1, scale = "original")
```

An edge function is one component of the fitted network. In a multi-layer
network it is not the partial effect of a predictor, and it is never a
causal effect.

## A different basis in each layer

`basis` may have one entry per layer. Basis-specific arguments go to the
layers whose basis uses them; `options` sets values for one basis only.

```{r}
m <- thangld_kan(c(2, 4, 3, 1), basis = c("rbf", "chebyshev", "jacobi"),
               degree = 3, options = list(jacobi = list(alpha = 0.5)),
               seed = 2)
m
kan_layer_bases(m)
```

## Adding a basis

Other packages add their edge type with `register_kan_basis()`; every
function above then works with it, including `thangld_kan_validate()`. The
interface is described in `?register_kan_basis`. A toy linear edge:

```{r}
lin <- list(
  label = "linear (toy)",
  layer_specs = function(widths) rep(list(list()), length(widths) - 1L),
  describe = function(spec) "linear",
  param_dims = function(p, q, spec) list(w = c(p, q)),
  init = function(p, q, spec) list(w = matrix(rnorm(p * q, sd = 0.3), p, q)),
  check_params = function(par, spec) invisible(TRUE),
  forward_r = function(h, par, spec) list(out = h %*% par$w, cache = h),
  backward_r = function(cache, g, par, spec)
    list(grads = list(w = crossprod(cache, g)), dh = g %*% t(par$w)),
  forward_torch = function(h, par, spec) torch::torch_matmul(h, par$w),
  edge_r = function(x, par, spec, i, j) data.frame(phi = par$w[i, j] * x))
register_kan_basis("toy_linear", lin)
res <- thangld_kan_validate("r", basis = "toy_linear", mixed = FALSE)
res[, c("group", "check", "pass")]
unregister_kan_basis("toy_linear")
```

The missing basis-specific checks of the toy basis are reported as skipped
(`NA`), not as passed.

## Persistence and numerical checks

```{r}
f <- tempfile(fileext = ".rds")
thangld_kan_save(fits$relu, f)
all.equal(predict(fits$relu, x[test, ]), predict(thangld_kan_load(f), x[test, ]))
res <- thangld_kan_validate("r")
aggregate(pass ~ basis, res, function(p) sprintf("%d/%d", sum(p), length(p)))
```

`thangld_kan_validate("both")` with the torch runtime installed adds the
torch forward, autograd and training checks; without the runtime they are
reported as skipped, never as passed.
