Benchmark a list of quoted expressions. Each expression will always run at least twice, once to measure the memory allocation and store results and one or more times to measure timing.
mark(
...,
min_time = 0.5,
iterations = NULL,
min_iterations = 1,
max_iterations = 10000,
check = TRUE,
memory = capabilities("profmem"),
filter_gc = TRUE,
relative = FALSE,
time_unit = NULL,
exprs = NULL,
env = parent.frame()
)Expressions to benchmark, if named the expression column will
be the name, otherwise it will be the deparsed expression.
The minimum number of seconds to run each expression, set to
Inf to always run max_iterations times instead.
If not NULL, the default, run each expression for
exactly this number of iterations. This overrides both min_iterations
and max_iterations.
Each expression will be evaluated a minimum of min_iterations times.
Each expression will be evaluated a maximum of max_iterations times.
Check if results are consistent. If TRUE, checking is done
with all.equal(), if FALSE checking is disabled and results are not
stored. If check is a function that function will be called with each
pair of results to determine consistency.
If TRUE (the default when R is compiled with memory
profiling), track memory allocations using utils::Rprofmem(). If FALSE
disable memory tracking.
If TRUE remove iterations that contained at least one
garbage collection before summarizing. If TRUE but an expression had
a garbage collection in every iteration, filtering is disabled, with a warning.
If TRUE all summaries are computed relative to the minimum
execution time rather than absolute time.
If NULL the times are reported in a human readable
fashion depending on each value. If one of 'ns', 'us', 'ms', 's', 'm', 'h',
'd', 'w' the time units are instead expressed as nanoseconds, microseconds,
milliseconds, seconds, hours, minutes, days or weeks respectively.
A list of quoted expressions. If supplied overrides expressions
defined in ....
The environment which to evaluate the expressions
A tibble with the additional summary columns. The following summary columns are computed
expression - bench_expr The deparsed expression that was evaluated
(or its name if one was provided).
min - bench_time The minimum execution time.
median - bench_time The sample median of execution time.
itr/sec - double The estimated number of executions performed per
second.
mem_alloc - bench_bytes Total amount of memory allocated by R while
running the expression. Memory allocated outside the R heap, e.g. by
malloc() or new directly is not tracked, take care to avoid
misinterpreting the results if running code that may do this.
gc/sec - double The number of garbage collections per second.
n_itr - integer Total number of iterations after filtering
garbage collections (if filter_gc == TRUE).
n_gc - double Total number of garbage collections performed over all
iterations. This is a psudo-measure of the pressure on the garbage collector, if
it varies greatly between to alternatives generally the one with fewer
collections will cause fewer allocation in real usage.
total_time - bench_time The total time to perform the benchmarks.
result - list A list column of the object(s) returned by the
evaluated expression(s).
memory - list A list column with results from Rprofmem().
time - list A list column of bench_time vectors for each evaluated
expression.
gc - list A list column with tibbles containing the level of
garbage collection (0-2, columns) for each iteration (rows).
press() to run benchmarks across a grid of parameters.
dat <- data.frame(x = runif(100, 1, 1000), y=runif(10, 1, 1000))
mark(
min_time = .1,
dat[dat$x > 500, ],
dat[which(dat$x > 500), ],
subset(dat, x > 500))
#> # A tibble: 3 × 13
#> expression min median `itr/sec` mem_alloc `gc/sec` n_itr n_gc total_time
#> <bch:expr> <bch:> <bch:> <dbl> <bch:byt> <dbl> <int> <dbl> <bch:tm>
#> 1 dat[dat$x >… 30.9µs 32.7µs 30240. 3.98KB 10.9 2783 1 92ms
#> 2 dat[which(d… 32.4µs 34.1µs 28950. 2.65KB 0 2887 0 99.7ms
#> 3 subset(dat,… 53.3µs 55.9µs 17546. 5.29KB 12.1 1451 1 82.7ms
#> # ℹ 4 more variables: result <list>, memory <list>, time <list>, gc <list>