Skip to content

Allow builds to skip source generation - #65004

Open
Jake Bailey (jakebailey) wants to merge 3 commits into
microsoft:mainfrom
jakebailey:no-gen-build
Open

Jake Bailey (jakebailey) wants to merge 3 commits into
microsoft:mainfrom
jakebailey:no-gen-build

Conversation

@jakebailey

@jakebailey Jake Bailey (jakebailey) commented Oct 28, 2026 •

Copy link
Copy Markdown
Member

The benchmarker is now constantly regenerating when building, so now the prep phase takes a half hour. Add a flag so it can skip.

Defaults to !isCI, so it's on locally but off in CI where we already do "code was generated" checks.

Avoid unnecessary generator work when building with existing generated
sources.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues block approval of the opt-in --skipGenerate change.

No open issues 🎉

What changed in this PR

Adds an opt-in --skipGenerate flag to reduce build preparation time for benchmarks by reusing existing generated sources. Source generation remains enabled by default.

Changes:

  • Makes generation dependencies conditional for compiler, API, package, and release builds.
  • Skips runGenerate() during watch rebuilds when the flag is set.
  • Documents the flag in the compiler build task description.
File Description
Herebyfile.mjs Adds --skipGenerate and applies it to build dependencies and watch-mode generation.

Review effort: Balanced

@jakebailey Jake Bailey (jakebailey) added the Domain: Performance Reports of unusually slow behavior label Oct 29, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at 9cd7adb. Source generation remains enabled by default; the opt-in consistently removes generation dependencies and skips watch regeneration without bypassing the actual build. Syntax checking and assertions against all five real task dependency lists passed for default and --skipGenerate; the compiler task also passed the negative-flag check. No blocking issue found.

@github-project-automation github-project-automation Bot moved this from Not started to Needs merge in PR Backlog Oct 30, 2026
@jakebailey

Copy link
Copy Markdown
Member Author

I'm realizing that this also has to apply to run dt, the top tests, error deltas, in addition to the perf run. So maybe just gating this on CI is the correct move.

CI builds can use the generated sources already checked in without
running generators again. Keep an explicit override for workflows that
need generation and local builds that want to skip it.
@jakebailey

Copy link
Copy Markdown
Member Author

TypeScript Bot (@typescript-bot) perf test this faster

just to double check

@typescript-automation

typescript-automation Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
perf test this faster ✅ Started 👀 Results

@typescript-automation

Copy link
Copy Markdown
Contributor

Jake Bailey (@jakebailey)
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - native
Errors 41 41 ~ ~ ~ p=1.000 n=20
Symbols 88,466 88,466 ~ ~ ~ p=1.000 n=20
Types 79,607 79,607 ~ ~ ~ p=1.000 n=20
Memory Used 106,425k (± 0.03%) 106,396k (± 0.03%) ~ 106,238k 106,508k p=0.256 n=20
Memory Allocs 1,261,838 (± 0.01%) 1,261,814 (± 0.01%) ~ 1,261,497 1,262,237 p=0.640 n=20
Config Time 0.000s 0.000s ~ ~ ~ p=1.000 n=20
Parse Time 0.023s (± 2.87%) 0.023s (± 2.47%) ~ 0.021s 0.025s p=0.590 n=20
Bind Time 0.009s (± 5.15%) 0.008s (± 4.59%) 🟩-0.001s (- 8.77%) 0.006s 0.009s p=0.011 n=20
Check Time 0.203s (± 0.81%) 0.203s (± 0.77%) ~ 0.197s 0.210s p=0.752 n=20
Emit Time 0.130s (± 0.81%) 0.130s (± 1.16%) ~ 0.127s 0.140s p=0.730 n=20
Total Time 0.367s (± 0.55%) 0.367s (± 0.65%) ~ 0.359s 0.378s p=0.732 n=20
angular-1 - native
Errors 63 63 ~ ~ ~ p=1.000 n=20
Symbols 756,637 756,637 ~ ~ ~ p=1.000 n=20
Types 549,451 549,451 ~ ~ ~ p=1.000 n=20
Memory Used 643,923k (± 0.02%) 643,861k (± 0.02%) ~ 643,420k 644,291k p=0.588 n=20
Memory Allocs 9,494,966 (± 0.05%) 9,494,909 (± 0.04%) ~ 9,487,570 9,517,670 p=0.534 n=20
Config Time 0.007s (± 2.08%) 0.007s (± 1.50%) ~ 0.006s 0.007s p=1.000 n=20
Parse Time 0.121s (± 2.58%) 0.120s (± 3.42%) ~ 0.107s 0.146s p=0.350 n=20
Bind Time 0.041s (± 7.92%) 0.040s (± 7.33%) ~ 0.035s 0.057s p=0.332 n=20
Check Time 0.642s (± 1.47%) 0.637s (± 1.13%) ~ 0.605s 0.670s p=0.482 n=20
Emit Time 0.399s (± 1.13%) 0.399s (± 0.73%) ~ 0.388s 0.410s p=0.456 n=20
Total Time 1.224s (± 0.96%) 1.218s (± 0.64%) ~ 1.191s 1.258s p=0.578 n=20
arktype - native
Errors 0 0 ~ ~ ~ p=1.000 n=20
Symbols 686,116 686,116 ~ ~ ~ p=1.000 n=20
Types 939,797 939,797 ~ ~ ~ p=1.000 n=20
Memory Used 475,952k (± 0.01%) 475,978k (± 0.01%) ~ 475,710k 476,322k p=0.763 n=20
Memory Allocs 12,052,216 (± 0.03%) 12,048,825 (± 0.01%) ~ 12,045,637 12,053,770 p=0.165 n=20
Config Time 0.001s 0.001s ~ ~ ~ p=1.000 n=20
Parse Time 0.042s (± 2.65%) 0.040s (± 2.42%) 🟩-0.002s (- 5.58%) 0.036s 0.044s p=0.002 n=20
Bind Time 0.012s (± 7.70%) 0.013s (± 9.56%) ~ 0.010s 0.019s p=0.584 n=20
Check Time 0.827s (± 0.55%) 0.827s (± 0.68%) ~ 0.791s 0.847s p=0.733 n=20
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=20
Total Time 0.909s (± 0.52%) 0.907s (± 0.54%) ~ 0.882s 0.925s p=0.878 n=20
class-transformer - native
Errors 0 0 ~ ~ ~ p=1.000 n=20
Symbols 45,376 45,376 ~ ~ ~ p=1.000 n=20
Types 7,573 7,573 ~ ~ ~ p=1.000 n=20
Memory Used 27,739k (± 0.03%) 27,744k (± 0.02%) ~ 27,718k 27,772k p=0.417 n=20
Memory Allocs 247,684 (± 0.05%) 247,728 (± 0.07%) ~ 247,116 248,742 p=0.910 n=20
Config Time 0.001s (±15.95%) 0.001s (±15.95%) ~ 0.000s 0.001s p=1.000 n=20
Parse Time 0.011s (± 3.06%) 0.012s (± 3.07%) ~ 0.009s 0.012s p=0.309 n=20
Bind Time 0.003s (± 7.81%) 0.003s (± 7.90%) ~ 0.003s 0.005s p=0.386 n=20
Check Time 0.008s (± 6.64%) 0.008s (± 7.63%) ~ 0.006s 0.011s p=0.262 n=20
Emit Time 0.007s (± 6.71%) 0.007s (± 6.95%) ~ 0.006s 0.009s p=0.886 n=20
Total Time 0.031s (± 1.73%) 0.031s (± 1.72%) ~ 0.028s 0.033s p=0.877 n=20
effect - native
Errors 12 12 ~ ~ ~ p=1.000 n=20
Symbols 1,235,134 1,235,134 ~ ~ ~ p=1.000 n=20
Types 893,436 893,436 ~ ~ ~ p=1.000 n=20
Memory Used 941,198k (± 0.02%) 941,533k (± 0.02%) +335k (+ 0.04%) 940,608k 942,297k p=0.012 n=20
Memory Allocs 12,589,215 (± 0.04%) 12,596,127 (± 0.05%) ~ 12,581,718 12,643,980 p=0.068 n=20
Config Time 0.002s 0.002s ~ ~ ~ p=1.000 n=20
Parse Time 0.143s (± 2.21%) 0.144s (± 1.51%) ~ 0.136s 0.153s p=0.995 n=20
Bind Time 0.000s 0.000s ~ ~ ~ p=1.000 n=20
Check Time 0.898s (± 1.30%) 0.907s (± 0.99%) ~ 0.863s 0.935s p=0.337 n=20
Emit Time 0.483s (± 1.44%) 0.485s (± 1.74%) ~ 0.455s 0.523s p=0.952 n=20
Total Time 1.622s (± 0.82%) 1.628s (± 0.90%) ~ 1.567s 1.685s p=0.664 n=20
mui-docs - native
Errors 11,405 11,405 ~ ~ ~ p=1.000 n=20
Symbols 3,782,394 3,782,394 ~ ~ ~ p=1.000 n=20
Types 1,355,778 1,355,778 ~ ~ ~ p=1.000 n=20
Memory Used 1,678,275k (± 0.08%) 1,678,390k (± 0.09%) ~ 1,670,808k 1,683,078k p=0.718 n=20
Memory Allocs 25,523,259 (± 0.08%) 25,519,344 (± 0.06%) ~ 25,439,072 25,580,653 p=0.758 n=20
Config Time 0.007s (± 2.49%) 0.007s (± 3.43%) ~ 0.006s 0.007s p=0.273 n=20
Parse Time 0.281s (± 2.33%) 0.278s (± 2.52%) ~ 0.244s 0.304s p=0.490 n=20
Bind Time 0.001s 0.001s ~ ~ ~ p=1.000 n=20
Check Time 3.019s (± 0.56%) 3.021s (± 0.71%) ~ 2.948s 3.098s p=0.743 n=20
Emit Time 0.257s (± 1.27%) 0.259s (± 1.14%) ~ 0.249s 0.271s p=0.506 n=20
Total Time 4.068s (± 0.52%) 4.066s (± 0.73%) ~ 3.946s 4.175s p=0.941 n=20
sentry - native
Errors 2 2 ~ ~ ~ p=1.000 n=20
Symbols 4,793,266 4,793,266 ~ ~ ~ p=1.000 n=20
Types 2,607,066 2,607,066 ~ ~ ~ p=1.000 n=20
Memory Used 2,798,346k (± 0.04%) 2,797,539k (± 0.04%) ~ 2,793,944k 2,802,554k p=0.314 n=20
Memory Allocs 74,613,966 (± 0.06%) 74,612,393 (± 0.12%) ~ 74,325,697 75,135,416 p=0.968 n=20
Config Time 0.019s 0.019s (± 0.76%) ~ 0.018s 0.019s p=0.487 n=20
Parse Time 0.468s (± 2.57%) 0.511s (± 6.10%) 🔻+0.043s (+ 9.08%) 0.450s 0.661s p=0.005 n=20
Bind Time 0.153s (± 9.70%) 0.138s (± 3.35%) ~ 0.129s 0.168s p=0.073 n=20
Check Time 6.811s (± 0.46%) 6.836s (± 0.44%) ~ 6.723s 6.913s p=0.199 n=20
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=20
Total Time 7.500s (± 0.35%) 7.555s (± 0.52%) +0.055s (+ 0.74%) 7.401s 7.739s p=0.022 n=20
social-app - native
Errors 0 0 ~ ~ ~ p=1.000 n=20
Symbols 1,883,060 1,883,060 ~ ~ ~ p=1.000 n=20
Types 1,158,093 1,158,093 ~ ~ ~ p=1.000 n=20
Memory Used 952,220k (± 0.02%) 952,278k (± 0.01%) ~ 951,910k 952,933k p=0.862 n=20
Memory Allocs 13,408,368 (± 0.12%) 13,409,295 (± 0.11%) ~ 13,356,133 13,463,731 p=0.799 n=20
Config Time 0.006s (± 5.64%) 0.006s (± 8.62%) ~ 0.005s 0.010s p=0.927 n=20
Parse Time 0.170s (± 5.45%) 0.168s (± 5.09%) ~ 0.148s 0.223s p=0.743 n=20
Bind Time 0.041s (± 7.38%) 0.042s (± 9.77%) ~ 0.034s 0.073s p=0.920 n=20
Check Time 1.467s (± 1.47%) 1.458s (± 1.21%) ~ 1.405s 1.534s p=0.372 n=20
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=20
Total Time 1.704s (± 1.62%) 1.693s (± 1.20%) ~ 1.623s 1.794s p=0.490 n=20
strada-build-src - native
Errors 0 0 ~ ~ ~ p=1.000 n=20
Symbols 1,184,972 1,184,972 ~ ~ ~ p=1.000 n=20
Types 438,823 438,823 ~ ~ ~ p=1.000 n=20
Memory Used 1,158,427k (± 0.30%) 1,157,629k (± 0.19%) ~ 1,147,488k 1,164,430k p=0.583 n=20
Memory Allocs 47,437,586 (± 0.10%) 47,423,967 (± 0.09%) ~ 47,239,546 47,540,577 p=0.602 n=20
Config Time 0.002s (±10.63%) 0.002s (±15.95%) ~ 0.001s 0.003s p=0.853 n=20
Parse Time 0.139s (± 1.29%) 0.138s (± 1.50%) ~ 0.131s 0.146s p=0.888 n=20
Bind Time 0.000s 0.000s (±208.60%) ~ 0.000s 0.002s p=1.000 n=20
Check Time 0.799s (± 0.82%) 0.795s (± 0.67%) ~ 0.781s 0.829s p=0.261 n=20
Emit Time 0.141s (± 1.92%) 0.141s (± 1.53%) ~ 0.133s 0.151s p=0.814 n=20
Total Time 9.332s (± 0.83%) 9.274s (± 0.51%) ~ 9.113s 9.547s p=0.369 n=20
strada-compiler - native
Errors 0 0 ~ ~ ~ p=1.000 n=20
Symbols 263,481 263,481 ~ ~ ~ p=1.000 n=20
Types 196,103 196,103 ~ ~ ~ p=1.000 n=20
Memory Used 221,503k (± 0.03%) 221,497k (± 0.02%) ~ 221,292k 221,735k p=0.825 n=20
Memory Allocs 2,387,252 (± 0.02%) 2,387,375 (± 0.01%) ~ 2,386,150 2,388,418 p=0.583 n=20
Config Time 0.000s 0.000s (±208.60%) ~ 0.000s 0.001s p=1.000 n=20
Parse Time 0.049s (± 2.95%) 0.050s (± 3.71%) ~ 0.045s 0.060s p=0.701 n=20
Bind Time 0.000s 0.000s ~ ~ ~ p=1.000 n=20
Check Time 0.362s (± 0.54%) 0.365s (± 0.82%) ~ 0.355s 0.380s p=0.223 n=20
Emit Time 0.046s (± 5.94%) 0.047s (± 5.76%) ~ 0.042s 0.062s p=0.220 n=20
Total Time 0.485s (± 0.60%) 0.489s (± 0.80%) ~ 0.477s 0.510s p=0.147 n=20
t3code-server - native
Errors 0 0 ~ ~ ~ p=1.000 n=20
Symbols 4,349,937 4,349,937 ~ ~ ~ p=1.000 n=20
Types 3,042,465 3,042,465 ~ ~ ~ p=1.000 n=20
Memory Used 2,395,135k (± 0.03%) 2,394,610k (± 0.03%) ~ 2,392,725k 2,398,461k p=0.265 n=20
Memory Allocs 37,008,821 (± 0.04%) 37,015,069 (± 0.06%) ~ 36,966,381 37,152,449 p=0.779 n=20
Config Time 0.002s (± 8.70%) 0.002s ~ ~ ~ p=0.106 n=20
Parse Time 0.242s (± 1.69%) 0.238s (± 2.56%) ~ 0.219s 0.259s p=0.250 n=20
Bind Time 0.079s (± 7.07%) 0.086s (±10.57%) ~ 0.071s 0.130s p=0.222 n=20
Check Time 5.646s (± 0.60%) 5.670s (± 0.49%) ~ 5.593s 5.802s p=0.098 n=20
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=20
Total Time 5.982s (± 0.60%) 6.009s (± 0.48%) ~ 5.923s 6.146s p=0.107 n=20
tanstack-router - native
Errors 1 1 ~ ~ ~ p=1.000 n=20
Symbols 570,149 570,149 ~ ~ ~ p=1.000 n=20
Types 396,551 396,551 ~ ~ ~ p=1.000 n=20
Memory Used 315,246k (± 0.02%) 315,242k (± 0.02%) ~ 314,970k 315,466k p=0.815 n=20
Memory Allocs 4,322,401 (± 0.28%) 4,324,670 (± 0.28%) ~ 4,310,601 4,426,245 p=0.183 n=20
Config Time 0.001s 0.001s ~ ~ ~ p=1.000 n=20
Parse Time 0.045s (± 5.51%) 0.044s (± 4.33%) ~ 0.038s 0.051s p=0.742 n=20
Bind Time 0.015s (±15.07%) 0.014s (±10.38%) ~ 0.011s 0.020s p=0.973 n=20
Check Time 0.525s (± 2.27%) 0.510s (± 1.52%) -0.015s (- 2.86%) 0.490s 0.568s p=0.023 n=20
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=20
Total Time 0.604s (± 2.33%) 0.587s (± 1.46%) -0.017s (- 2.80%) 0.565s 0.648s p=0.038 n=20
ts-pre-modules - native
Errors 87 87 ~ ~ ~ p=1.000 n=20
Symbols 231,607 231,607 ~ ~ ~ p=1.000 n=20
Types 181,163 181,163 ~ ~ ~ p=1.000 n=20
Memory Used 190,833k (± 0.02%) 190,848k (± 0.01%) ~ 190,747k 190,938k p=0.441 n=20
Memory Allocs 1,310,128 (± 0.02%) 1,310,005 (± 0.02%) ~ 1,309,149 1,311,959 p=0.311 n=20
Config Time 0.000s 0.000s ~ ~ ~ p=1.000 n=20
Parse Time 0.043s (± 2.70%) 0.042s (± 2.40%) ~ 0.038s 0.046s p=0.360 n=20
Bind Time 0.020s (± 5.95%) 0.019s (± 5.26%) ~ 0.016s 0.024s p=0.887 n=20
Check Time 0.287s (± 1.01%) 0.288s (± 0.85%) ~ 0.278s 0.298s p=0.343 n=20
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=20
Total Time 0.356s (± 0.60%) 0.356s (± 0.61%) ~ 0.346s 0.365s p=0.825 n=20
vscode - native
Errors 11 11 ~ ~ ~ p=1.000 n=20
Symbols 7,905,839 7,905,839 ~ ~ ~ p=1.000 n=20
Types 4,635,312 4,635,312 ~ ~ ~ p=1.000 n=20
Memory Used 5,302,329k (± 0.01%) 5,302,367k (± 0.02%) ~ 5,299,716k 5,305,623k p=0.947 n=20
Memory Allocs 58,449,938 (± 0.02%) 58,449,413 (± 0.02%) ~ 58,431,686 58,505,970 p=0.799 n=20
Config Time 0.023s (± 0.63%) 0.023s (± 1.17%) ~ 0.022s 0.024s p=0.127 n=20
Parse Time 0.875s (± 1.23%) 0.866s (± 1.61%) ~ 0.802s 0.920s p=0.273 n=20
Bind Time 0.305s (± 5.04%) 0.305s (± 6.53%) +0.000s (+ 0.03%) 0.284s 0.446s p=0.020 n=20
Check Time 7.033s (± 0.40%) 7.006s (± 0.74%) ~ 6.851s 7.356s p=0.071 n=20
Emit Time 2.166s (± 6.69%) 2.293s (± 6.56%) ~ 1.971s 2.887s p=0.180 n=20
Total Time 10.461s (± 1.28%) 10.553s (± 1.21%) ~ 10.255s 11.134s p=0.159 n=20
webpack - native
Errors 615 615 ~ ~ ~ p=1.000 n=20
Symbols 883,525 883,525 ~ ~ ~ p=1.000 n=20
Types 621,348 621,348 ~ ~ ~ p=1.000 n=20
Memory Used 647,579k (± 0.02%) 647,482k (± 0.02%) ~ 647,065k 647,996k p=0.192 n=20
Memory Allocs 6,028,196 (± 0.12%) 6,032,070 (± 0.21%) ~ 6,010,305 6,126,386 p=1.000 n=20
Config Time 0.003s 0.003s ~ ~ ~ p=1.000 n=20
Parse Time 0.121s (± 1.96%) 0.120s (± 3.30%) ~ 0.107s 0.134s p=0.804 n=20
Bind Time 0.039s (±10.22%) 0.043s (± 9.55%) 🔻+0.004s (+ 9.92%) 0.033s 0.058s p=0.048 n=20
Check Time 0.924s (± 1.65%) 0.917s (± 0.81%) ~ 0.893s 0.943s p=0.763 n=20
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=20
Total Time 1.110s (± 1.28%) 1.106s (± 0.56%) ~ 1.082s 1.131s p=0.743 n=20
xstate-main - native
Errors 0 0 ~ ~ ~ p=1.000 n=20
Symbols 644,330 644,330 ~ ~ ~ p=1.000 n=20
Types 392,175 392,175 ~ ~ ~ p=1.000 n=20
Memory Used 346,915k (± 0.02%) 346,991k (± 0.03%) ~ 346,662k 347,294k p=0.433 n=20
Memory Allocs 4,003,682 (± 0.20%) 4,001,171 (± 0.24%) ~ 3,973,240 4,036,056 p=0.640 n=20
Config Time 0.001s 0.001s ~ ~ ~ p=1.000 n=20
Parse Time 0.061s (± 2.41%) 0.059s (± 2.26%) ~ 0.054s 0.064s p=0.053 n=20
Bind Time 0.020s (±11.28%) 0.020s (±10.22%) ~ 0.016s 0.030s p=0.656 n=20
Check Time 0.414s (± 0.69%) 0.418s (± 0.67%) ~ 0.407s 0.431s p=0.056 n=20
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=20
Total Time 0.503s (± 0.79%) 0.503s (± 0.74%) ~ 0.489s 0.519s p=0.733 n=20
System info unknown
Hosts
  • native
Scenarios
  • Compiler-Unions - native
  • angular-1 - native
  • arktype - native
  • class-transformer - native
  • effect - native
  • mui-docs - native
  • sentry - native
  • social-app - native
  • strada-build-src - native
  • strada-compiler - native
  • t3code-server - native
  • tanstack-router - native
  • ts-pre-modules - native
  • vscode - native
  • webpack - native
  • xstate-main - native
Benchmark Name Iterations
Current pr 20
Baseline baseline 20

Developer Information:

Download Benchmarks

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Author: Team Domain: Performance Reports of unusually slow behavior For Uncommitted Bug PR for untriaged, rejected, closed or missing bug

Projects

Status: Needs merge

Development

Successfully merging this pull request may close these issues.

3 participants