Skip to content

fix: keep one-time setup costs out of the first measured benchmark #123

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
not-matthias wants to merge 2 commits into
base: main
Choose a base branch
from
Draft

fix: keep one-time setup costs out of the first measured benchmark #123

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
e962378
fix(tinybench): warm up the root frame before measuring
not-matthias Nov 6, 2026
329b067
fix(core): bind the native addon's symbols at load
not-matthias Nov 6, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions packages/core/binding.gyp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,12 @@
"-Wno-unused-but-set-variable",
"-Wno-type-limits"
],
# Resolve symbols at load: lazy binding would resolve
# instrument_hooks_stop_benchmark on its first call, inside the
# first measured benchmark.
"ldflags": [
"-Wl,-z,now"
],
"sources": [
"src/native_core/linux_perf/linux_perf.cc",
"src/native_core/linux_perf/linux_perf_listener.cc",
Expand Down
10 changes: 6 additions & 4 deletions packages/tinybench-plugin/src/analysis.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,16 +32,18 @@ class AnalysisBenchRunner extends BaseBenchRunner {
const { fnOpts, fn } = this.getTaskData(task);

await fnOpts?.beforeAll?.call(task, "run");
// Warm up through the root frame: V8 compiles it on its first call, which
// must not land in the measurement.
const rootFrame = wrapWithRootFrame(fn);
await optimizeFunction(async () => {
await fnOpts?.beforeEach?.call(task, "run");
await fn();
await rootFrame();
await fnOpts?.afterEach?.call(task, "run");
});
await fnOpts?.beforeEach?.call(task, "run");
await mongoMeasurement.start(uri);

global.gc?.();
const rootFrame = wrapWithRootFrame(fn);
InstrumentHooks.startBenchmark();
try {
await rootFrame();
Expand All @@ -62,16 +64,16 @@ class AnalysisBenchRunner extends BaseBenchRunner {

fnOpts?.beforeAll?.call(task, "run");

const rootFrame = wrapWithRootFrameSync(fn);
optimizeFunctionSync(async () => {
fnOpts?.beforeEach?.call(task, "run");
await fn();
await rootFrame();
fnOpts?.afterEach?.call(task, "run");
});

fnOpts?.beforeEach?.call(task, "run");

global.gc?.();
const rootFrame = wrapWithRootFrameSync(fn);
InstrumentHooks.startBenchmark();
try {
rootFrame();
Expand Down
Loading