toolchain_check() diagnostic function reports the
status of the Python (via reticulate), Julia, and Rust toolchains:
interpreter/compiler discovery in a fresh worker process,
environment-manager presence (uv, Poetry, or conda for Python; juliaup
and Julia’s Pkg project mechanism for Julia; rustup, cargo,
and, on Windows, the GNU toolchain and Rtools for Rust), and
multi-version discovery that lists every installed version per language
and marks the one that would be used by default. Use
toolchains = c("py", "jl", "rs") to check a subset, and
deep = FALSE to skip the slower compile-reachability check
for Rust.toolchain_check() (all languages, or Python/Julia/Rust
individually) from the Addins menu.polyglot_log(), passed to
polyglot_controller(log = ...), turns on per-step
stdout/stderr log files (<step name>.out /
<step name>.err) for Python, Julia and C++
steps. Since crew launches worker processes before
it knows which target they will run, the configuration is stashed as
environment variables that every worker inherits, and
run_py_step() / run_jl_step() /
run_cpp_step() do the actual redirection once they know the
step name and resolved toolchain; append controls whether a
re-run truncates or accumulates (with a two-blank-line separator) the
log, and header prepends the step name,
date(), interpreter version/path, and whether an explicit
environment was used. On C++ the capture reaches
Rcpp::Rcout and Rcpp::Rcerr, which route
through R’s own output connection system and so are captured the same
way cat() output is, confirmed empirically including from a
fresh crew worker process; raw std::cout /
std::cerr / printf() write straight to the OS
file descriptor and are not captured. Deliberately does not
cover tar_target_rs(), for two independent
reasons: rextendr-compiled code writes straight to the OS
file descriptor too, bypassing the redirection reticulate/JuliaCall
provide for their embedded interpreters; and Rust steps don’t pay a
per-branch interpreter start-up cost the way Python/Julia do in the
first place, since a Rust library under pattern is already
compiled once and reused across every branch
(tarpolyglot_map() and friends,
compile_rs_lib() / run_rs_step_prebuilt()), so
there is correspondingly less need for a per-step Rust log. Use
crew’s own options_local(log_directory = ...)
for Rust step output instead (it also covers Python/Julia when
log is left unset, since tarpolyglot never spawns a
subprocess for them).tar_target_cpp() and
tar_target_cpp_raw() run C++ as targets steps
via Rcpp, mirroring
tar_target_rs() closely: no live interpreter,
// [[Rcpp::export]] functions in script are
compiled with Rcpp::sourceCpp() and exposed to an R
post-script, and run_cpp_step() puts R’s own
bin directory, and on Windows Rtools, on PATH
for the build. There is no separate toolchain or ABI to select the way
Rust needs on Windows, sourceCpp() always compiles with
whatever compiler R itself already uses, so
tar_target_cpp() has no toolchain argument. A
new depends argument
(e.g. depends = "RcppArmadillo") is a convenience
alternative to a // [[Rcpp::depends(pkgname)]] source
attribute for header only extension packages such as RcppArmadillo and
RcppEigen, which otherwise need no tarpolyglot specific handling. A
compiled function can also declare an Rcpp::Function
parameter to call back into an ordinary R function, including one the
caller supplies, with no extra plumbing needed. See
vignette("cpp").tarpolyglot_*() pattern helpers
(tarpolyglot_map(), tarpolyglot_cross(),
tarpolyglot_slice(), tarpolyglot_head(),
tarpolyglot_tail(), tarpolyglot_sample()) now
also compile once on tar_target_cpp(), the same idea as for
Rust but adapted to how Rcpp::sourceCpp() binds functions:
since its generated wrappers capture a raw compiled routine pointer at
bind time rather than resolving by name at call time the way rextendr’s
do, the new compile_cpp_lib() embeds the compiled library’s
bytes together with its generated R wrapper source, and
run_cpp_step_prebuilt() re-evaluates that source in
whichever process reloads the library, re-binding valid closures without
recompiling. As with Rust, sourceCpp() names every freshly
compiled library sourceCpp_<N> from a per-session
counter, so run_cpp_step_prebuilt() tracks the loaded
library by content and hot-swaps when a different one needs the same
module name, the same fix run_rs_step_prebuilt() already
has.toolchain_check() gained a "cpp" toolchain
(now the default toolchains is
c("py", "jl", "rs", "cpp")): a presence check for Rcpp
itself only, not the extension packages (RcppArmadillo, RcppEigen,
RcppParallel are the caller’s own choice via depends, not
something tarpolyglot depends on); an OS appropriate compiler presence
check (Rtools on Windows; on macOS and Linux, whichever compiler
R CMD config CXX reports, queried rather than guessed so it
is correct whether R was configured to use gcc or clang); and, when
deep = TRUE (the default), compiling a trivial
// [[Rcpp::export]] function in a fresh worker. The RStudio
addin gained a matching C++ only entry.files argument has been removed from
tar_target_rs(), tar_target_cpp(), their
_raw() forms, and the matching workers
(run_rs_step(), run_cpp_step(),
run_rs_step_prebuilt(),
run_cpp_step_prebuilt()). It could never do anything there:
a Rust or C++ script only compiles its #[extendr] /
// [[Rcpp::export]] functions, so with files
and no post_script nothing ever calls them and the declared
file is never written, leaving targets to fail the target
with Error resolving output location: missing files. A
post-script that writes the file but does not return its path did not
fall back to files either, and when both were supplied the
post-script silently won. In file mode these constructors now require a
post_script that returns the path(s), and say so:
output = "file" needs a post_script returning paths. Only
the Rust half is a break against a released version, since
tar_target_cpp() is new in this release.
tar_target_py() and tar_target_jl() keep
files unchanged: their scripts run and write on their own,
so declaring a fixed path up front and omitting the post-script
genuinely works.tar_target_jl() and tar_target_jl_raw() no
longer bake a machine-specific Julia path into the target they build.
Their julia_home argument defaulted to
getOption("tarpolyglot.julia_home"), which was evaluated
when the target was constructed and spliced into the generated
run_jl_step() call, so a project that set that option (the
natural place being .Rprofile) put this computer’s Julia
directory into the command that targets hashes. Two
machines with different Julia installations, or one with the option
unset, then disagreed about every Julia step’s command hash, and a store
copied between them reported all of those steps as outdated even though
nothing had changed. julia_home now defaults to
NULL in both constructors and in
run_jl_step(), and the option is read once at run time
inside the resolver, where its value cannot reach the command. Behaviour
is otherwise unchanged: an explicit julia_home, then the
option, then julia_version, then JuliaCall’s own discovery
on PATH. A new
tests/testthat/test-command-portability.R pins this down,
asserting that a step’s command hash survives both relocating the
project directory and changing tarpolyglot.julia_home, that
an existing relative env is not absolutised into a Python
step’s command, and that the resolver still honours the option at run
time.rextendr<N> from a per-process counter, so two
libraries built in separate crew workers can both be
rextendr1; a worker that later reused both would have
resolved calls to whichever library loaded first.
run_rs_step_prebuilt() now tracks the loaded library by
content and hot-swaps when a different one needs the same module name.
As a result, pipelines with multiple tar_target_rs() steps
(branching or not), reusing a compiled library across steps, and using
more than one Rust library in a single step all behave correctly. See
vignette("rust").vignette("scripts") covers choosing between the
three script forms and what each one means for change detection, mixing
forms in a single call, the common mistake of giving
tar_target_path() a file path instead of a target name, and
why code assembled with glue() can use R objects that exist
when _targets.R is sourced but cannot see the values of
upstream targets.vignette("scripts") also documents tracking helper
files that a script imports, by pointing inputs at a
format = "file" target, with matching examples on the
Python and Julia constructors. Left untracked, editing a helper leaves
the step up to date and returns a stale value. The pattern works for
Python and Julia, where the helper is loaded at run time from the path
bound in the step environment; Julia additionally needs that path made
absolute, since include() resolves a relative path against
the directory of the file doing the including rather than the working
directory. It does not work for C++ and Rust, where
run_cpp_step() and run_rs_step() hand the
script’s text to Rcpp::sourceCpp(code = ...) and
rextendr::rust_source(code = ...): the compiler then runs
in a temporary build directory, so a neighbouring helper is not on its
search path and a relative #include "helper.h" or
include!("helper.rs") does not resolve. Listing the helper
in inputs still registers it as a dependency there, so
editing it re-runs and recompiles the step; only the loading half is
unavailable.vignette("rust") now documents compiling a Rust library
once and reusing it across unrelated steps
(compile_rs_lib() / run_rs_step_prebuilt()),
defining several #[extendr] functions in one script, using
more than one library in a step, and the accompanying limitations.vignette("rust") and the tarpolyglot_*()
pattern helpers (tarpolyglot_map(),
tarpolyglot_cross(), tarpolyglot_slice(),
tarpolyglot_head(), tarpolyglot_tail(),
tarpolyglot_sample()) were modified to reflect the
previously described limitations of these helpers on
targets::tar_target() and
targets::tar_target_raw().vignette("python"), vignette("julia"), and
vignette("rust") were modified to improve their
examples.vignette("cpp") covers the new C++ (Rcpp) workflow
of tarpolyglot: when to use it over a plain tar_target()
calling Rcpp::sourceCpp() directly, extension packages,
bringing an R function into C++ with Rcpp::Function,
compile once branching, reusing a compiled library across steps, and
polyglot_log() support.crew workers documented for
the first time, in vignette("cpp"),
vignette("python"), vignette("julia"), and a
new item in vignette("get_started")’s “Limitations to know
first”: a crew worker has access to every core on the
machine, not a restricted slice of it, so the real risk with a step that
does internal parallel work (RcppParallel, Python
multiprocessing, Julia Distributed) is several
workers oversubscribing the machine by each spawning a full width thread
or process pool at once, not lack of access to cores.
deployment = "main" on the parallel-heavy step is the
documented fix, since targets only ever builds one
main-deployed target at a time. Demonstrated with a working
deployment = "main" target in each of
examples/tarpolyglot_cpp,
examples/tarpolyglot_py, and
examples/tarpolyglot_jl.tar_target_py(),
tar_target_jl(), tar_target_rs(),
tar_target_cpp(), and their _raw() forms) now
document the three ways to supply script,
pre_script, and post_script, in a shared
“Script options” section with identical wording everywhere: a literal
path (untracked, so editing the file does not re-run the step), a
tar_target_path() reference (tracked, so editing it does),
or inline code from tar_code() (hashed as part of the
target’s command, so editing it does). Each of the eight gained
\dontrun{} examples showing the three forms side by
side.run_py_step(),
run_jl_step(), run_rs_step(),
run_cpp_step(), and the compile_*_lib() /
run_*_step_prebuilt() helpers) gained a deliberately
separate “Script arguments” section rather than repeating the
constructors’ three forms, because a worker never sees three. By the
time one runs, the constructor has already rewritten any
tar_target_path() reference into the upstream target’s own
file path, so a worker receives either a path on disk or an inline
tar_code() carrier; handing the result of
tar_target_path() straight to a worker therefore does not
resolve to a file.man/figures/logo.svg, man/figures/logo.png,
and every file in pkgdown/favicon/ were regenerated to use
this new logo. It was redraw to use original type-set marks (no
third-party logo artwork).vignette("python") and
vignette("julia") now documents both ways a step can report
files written on disk using the ‘files’ or ‘output’ arguments of the
constructors.0.2.0 version, the tarpolyglot_*() pattern
helpers (tarpolyglot_map(),
tarpolyglot_cross(), tarpolyglot_slice(),
tarpolyglot_head(), tarpolyglot_tail(),
tarpolyglot_sample()) are recognised only inside the
tarpolyglot constructors (tar_target_rs(),
tar_target_py(), tar_target_jl(),
tar_target_cpp(), and their _raw() forms).
Used directly in a plain targets::tar_target() or
targets::tar_target_raw() they raise an
invalid dynamic branching pattern ... Illegal symbols found
error. This cannot be corrected from tarpolyglot: targets
validates a pattern against a fixed set of pattern functions held in a
locked internal environment, so recognising a new helper would require
modifying the targets package itself. In a plain
targets target (which has no foreign code to compile) use
the native map() / cross() /
slice() / head() / tail() /
sample() instead.Threads.@threads multi-threading does
not work through JuliaCall: Threads.nthreads() stays at
1 regardless of JULIA_NUM_THREADS, a
limitation of how JuliaCall embeds Julia. Use Distributed
(addprocs() + pmap()) instead, which works
correctly; see vignette("julia"), “Parallel computing
(Distributed, not Threads)”.pattern shares a single
polyglot_log() file, for any language: the log file name is
fixed to the target’s own name before branching happens, and since
append = FALSE truncates on every branch’s run, only the
last branch to run leaves output in the file. Use
append = TRUE to keep all branches’ output instead, or
crew’s own options_local(log_directory = ...)
for a genuinely one file per worker process log.run_*_step() function
introduce modification in the command hash prompting targets to rebuild
the steps. # tarpolyglot 0.2.1polyglot_controller() now fails gracefully when
crew cannot build a controller on the host system,
reporting what is wrong and what to do about it instead of passing the
underlying error through. crew routes every controller
through nanonext’s TLS layer: crew::crew_tls() validates by
default, and that validation calls nanonext::tls_config()
as a self-test whatever the TLS mode, so it runs even though this
controller requests no TLS. Where nanonext links a system NNG built
without TLS support, rather than compiling its bundled copy, the
self-test reports "Not supported" and no controller can be
constructed, which is a property of the host installation rather than of
the arguments given.crew tests skip on such a platform instead of
failing, so R CMD check no longer errors there (seen on
CRAN’s r-devel-linux-x86_64-fedora-clang). Two tests were
added covering the new behaviour: that an unusable TLS layer produces
the explanatory error, and that any other crew error is
still passed through unchanged.targets patterns (map(), cross(),
slice(), …). Used unquoted in pattern on
tar_target_rs() / tar_target_rs_raw(), they
compile the Rust crate a single time in a companion
<step name>_rust_lib target and reuse that compiled
library across every branch (each branch reloads it in milliseconds),
instead of recompiling the crate in every branch as the plain
targets patterns do. On the other constructors
(tar_target_py(), tar_target_jl(), or
tar_target()) each helper falls back to its plain
targets equivalent, so the same pattern code branches every
language.
tarpolyglot_map(): equivalent to
map()tarpolyglot_cross(): equivalent to
cross()tarpolyglot_slice(): equivalent to
slice()tarpolyglot_head(): equivalent to
head()tarpolyglot_tail(): equivalent to
tail()tarpolyglot_sample(): equivalent to
sample()tar_code() now runs
correctly when it spans multiple top-level statements (for example a
function definition followed by a call). It is evaluated as
a Julia script rather than as a single expression, so it no longer
raises ParseError("extra token after end of expression").
Inline Python and Rust were unaffected.First release. tarpolyglot adds targets
constructors that run Python, Julia, and Rust as pipeline steps.
tar_target_py(), tar_target_jl(), and
tar_target_rs() (with matching _raw()
variants) mirror targets::tar_target() /
targets::tar_target_raw(). Python and Julia steps run a
script through a live interpreter (reticulate / JuliaCall) with optional
R pre- and post-scripts; Rust steps compile #[extendr]
functions with rextendr and call them from an R post-script.output = "file").tar_target_path() tracks a script file as a real
targets dependency, so a step re-runs when its script
changes.polyglot_controller() provides a crew
controller preconfigured for per-step interpreter isolation
(tasks_max = 1).env /
env_manager (system, virtualenv, venv, uv, poetry, conda),
python_version, or an explicit python path;
Julia selection via julia_version / julia_home
/ julia_project / julia_packages.targets::tar_target_raw() argument pass-through,
including dynamic branching (pattern).tar_code() supplies inline code for the
script, pre_script, and
post_script arguments of the constructors, as an
alternative to a file path or a tar_target_path()
reference. An R {...} block is captured as inline R and is
valid in the pre_script / post_script slots; a
character string is inline source for the foreign script
(Python, Julia, or Rust) or for an R pre/post-script. Multi-line strings
are dedented, so code indented to line up with _targets.R
still starts flush-left and Python’s own block indentation stays valid.
Inline code is embedded in the target’s command, so targets
hashes it and re-runs the step when it changes.