Skip to content

SONARJAVA-7339 Make the Java frontend switchable between ECJ and SemSitter - #6547

Draft
rombirli wants to merge 3 commits into
rombirli/semsitter-client-stubfrom
rombirli/semsitter-frontend-switch
Draft

rombirli wants to merge 3 commits into
rombirli/semsitter-client-stubfrom
rombirli/semsitter-frontend-switch

Conversation

@rombirli

Copy link
Copy Markdown
Contributor

Part of the SemSitter frontend stack: #6541, #6542, this PR.

What

Makes the Java frontend switchable between ECJ and SemSitter without touching any caller.

  • JParserProvider (SPI, found with ServiceLoader) and JParserProviders: JParser.Mode.create now asks the enabled provider and falls back to JDTParser. JavaSensor configures the providers once per analysis (not in SonarLint).
  • SemSitterParser implements the unchanged JParser contract (parse(Iterable<InputFile>, BooleanSupplier, AnalysisProgress, BiConsumer<InputFile, JParseResult>)). One server per call, files parsed in the server scan order, cancellation and syntax errors reported like ECJ, fallback to ECJ if the server cannot start.
  • SemSitterTranslator, JSema and the symbol/type model translate the typed Document to the same trees and semantic model.
  • SemSitterParserProvider enables the frontend when an executable is known (sonar.java.semsitter.executable or SEMSITTER_ANALYZER) and either sonar.java.semsitter.enabled is set, or the LaunchDarkly flag sonar-java-semsitter-frontend is on for the project (context kind project, key sonar.projectKey, client-side ID from sonar.java.semsitter.launchDarkly.clientSideId or SONAR_JAVA_LAUNCHDARKLY_CLIENT_ID). Default is off, any failure keeps ECJ.
  • The flag is read through the LaunchDarkly client-side evaluation endpoint with the JDK HTTP client. The server SDK added about 5 MB to the plugin.
  • The plugin size ceiling goes from 21.5 MB to 24 MB (protobuf runtime and the translator).

Tests

  • JParserProvidersTest: provider selection, fallback, failing providers.
  • SemSitterParserProviderTest, LaunchDarklyFeatureFlagsTest (local HTTP server), SemSitterSettingsTest, ProjectRootTest, DocumentFactsTest.
  • SemSitterParserTest: parser contract against a scripted in-memory server (ordering, settings, cancellation, fallback, syntax errors).
  • SemSitterParityTest: parses the same sources with ECJ and a real SemSitter server and compares every node kind, position, identifier type and symbol. It only runs with -Dsemsitter.executable=<udg-java-server>.

Known gaps

Parity is not complete on the check samples without a classpath. Known differences: array designators written after the name (int f()[], String c[]), the type name of anonymous and local classes, and the naming of unresolved types. They are tracked for follow-up and do not affect the default (ECJ) path.

Comment on lines +180 to +194
throw new IOException("SemSitter server: " + header.get("error").getAsString());
}
return new Answer(header, payload);
} catch (IOException e) {
String message = e.getMessage().startsWith("SemSitter") ? e.getMessage() : ("SemSitter server: " + e.getMessage());
throw new IOException(message, e);
}
}

/**
* Asks the server to quit and waits for it, forcing it to stop when it does not answer.
*/
@Override
public void close() {
try {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Edge Case: close() can hang forever on a server that stops answering

The close() javadoc says it "forc[es] it to stop when it does not answer". In fact close() first calls send(command("quit")), which writes the request and then blocks in readLine(fromServer). That read has no timeout. If the server process is still alive but stuck (deadlock, endless loop), close() never reaches waitFor(10, SECONDS) or destroyForcibly(), so the analysis hangs inside SemSitterParser.scan's try-with-resources. Every other exchange (getFile, getTypes) also blocks with no deadline. Fix: send quit without waiting for the answer, or read the answer on a separate thread with a timeout, so the destroyForcibly fallback always runs.

Was this helpful? React with 👍 / 👎

Comment on lines +109 to +123
boolean completed = false;
try (session) {
for (InputFile file : files) {
if (isCanceled.getAsBoolean()) {
if (mode == Mode.BATCH) {
throw new CancellationException("Analysis cancelled");
}
break;
}
executionTimeReport.start(file);
action.accept(file, session.parse(file));
executionTimeReport.end();
progressReport.nextFile();
}
completed = true;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Quality: Cancelled file-by-file parse is reported as a completed run

When isCanceled turns true in FILE_BY_FILE mode, SemSitterParser.scan breaks out of the loop and then sets completed = true. The finally block therefore calls progressReport.stop(), which reports a finished run, instead of progressReport.cancel(). ECJ's JDTParser.FileByFile tracks a separate cancelled flag and calls cancel() in this case. The PR says cancellation is "reported like ECJ", but here a cancelled analysis is logged as complete. Fix: set completed only when the loop was not broken by cancellation.

Was this helpful? React with 👍 / 👎

@rombirli
rombirli added this pull request to stack #6543 November 20, 2026 10:19
@rombirli
rombirli force-pushed the rombirli/semsitter-frontend-switch branch from f908cdc to 0efff46 Compare November 20, 2026 10:29
@datadog-sonarsource

This comment has been minimized.

@rombirli
rombirli force-pushed the rombirli/semsitter-frontend-switch branch from 0efff46 to f9667fa Compare November 20, 2026 10:58
private volatile Path executable;

public SemSitterParserProvider() {
this(LaunchDarklyFeatureFlags::connect, System::getenv, Optional.empty());

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Quality: Provider claims SemSitter is used but always builds the ECJ parser

After this commit the public no-arg constructor passes Optional.empty() as the translator (SemSitterParserProvider.java:59), so create() always goes through orElseGet(defaultParser) and returns JDTParser (lines 109-111). configure() still sets enabled = true and logs "The Java analyzer will use the SemSitter frontend." (lines 77-80). With sonar.java.semsitter.enabled=true or the LaunchDarkly flag on, that log is false: ECJ parses every file, and the analysis still calls the flag service for nothing. The PR description still says SemSitterTranslator/JSema translate documents, but this commit deletes them. Suggested fix: when no translator is present, keep enabled false (and skip the flag lookup), or change the log so it doesn't claim SemSitter is in use.

Only enable (and only query the flag) when a translator is available:

executable = Path.of(path.get());
if (translator.isEmpty()) {
  return;
}
enabled = configuration.getBoolean(ENABLED_PROPERTY).orElseGet(() -> isFlagOn(configuration));
  • Apply fix

Check the box to apply the fix or reply for a change | Was this helpful? React with 👍 / 👎

@rombirli
rombirli force-pushed the rombirli/semsitter-frontend-switch branch from f9667fa to 4c45d27 Compare November 20, 2026 11:25
…osen per analysis

Callers keep using JParser.Mode.create. The provider is found with the ServiceLoader and
falls back to the ECJ parser when none is enabled, so nothing changes until a frontend
module opts in.
@rombirli
rombirli force-pushed the rombirli/semsitter-frontend-switch branch from 4c45d27 to 840aeb1 Compare November 20, 2026 15:48
@gitar-bot

gitar-bot Bot commented Nov 20, 2026

Copy link
Copy Markdown
CI failed: The ruling-update job failed because the upstream ruling QA result was cancelled, so no artifacts were downloaded and the sync step errored. The analyzer attributes this to a cascading infrastructure issue, not the PR; the cause of the cancellation itself is not established by the logs.

Overview

The ruling-update job failed because the upstream ruling QA result was cancelled, so no artifacts were downloaded and the sync step errored. The analyzer attributes this to a cascading infrastructure issue, not the PR; the cause of the cancellation itself is not established by the logs.

Failures

Ruling-update sync failed: no artifacts after cancelled ruling QA (infrastructure, unrelated to PR)

  • Type: infrastructure
  • Affected jobs: 136394001663
  • Related to change: no
  • Confidence: high
  • Root cause: The workflow treats the cancelled ruling-qa result as RULING_FAILED=true. The artifact download then found 0 artifacts, and the sync step failed with No artifacts were downloaded (directory does not exist). This is a cascading failure; the log does not establish why ruling QA was cancelled. The PR changes the Java frontend, and the supplied evidence does not connect those changes to this workflow cancellation or missing artifacts.
  • Suggested fix: Investigate why the ruling-qa jobs were cancelled and rerun them so they can upload the expected actual_* artifacts. If cancellation is expected, update the ruling-update workflow to skip artifact syncing when no artifacts were produced.

Summary

  • Change-related failures: 0
  • Infrastructure/flaky failures: 1
  • Recommended action: Investigate why the ruling-qa jobs were cancelled and rerun them so they can upload the expected actual_* artifacts. If cancellation is expected, update the ruling-update workflow to skip artifact syncing when no artifacts were produced.
Code Review 👍 Approved with suggestions 🔴 High risk 1 closed / 4 findings

The selectable SemSitter frontend changes core Java parsing when enabled, so translation differences could alter findings across a project's Java files.

Adds a per-analysis parser provider so the Java frontend can be switched between ECJ and SemSitter behind a feature flag, with ECJ fallback on any failure. Approved with suggestions: close() can hang indefinitely on a stuck server because its quit exchange has no read timeout, so the forced-destroy fallback never runs. A cancelled file-by-file parse is also reported as a completed run instead of cancelled. And the no-arg provider always builds the ECJ parser while still logging that SemSitter is in use. An NPE when a type is looked up after the session closes was resolved.

💡 Edge Case: close() can hang forever on a server that stops answering

📄 java-frontend-semsitter/src/main/java/org/sonar/java/semsitter/SemSitterClient.java:180-194 📄 java-frontend-semsitter/src/main/java/org/sonar/java/semsitter/SemSitterClient.java:200-209

The close() javadoc says it "forc[es] it to stop when it does not answer". In fact close() first calls send(command("quit")), which writes the request and then blocks in readLine(fromServer). That read has no timeout. If the server process is still alive but stuck (deadlock, endless loop), close() never reaches waitFor(10, SECONDS) or destroyForcibly(), so the analysis hangs inside SemSitterParser.scan's try-with-resources. Every other exchange (getFile, getTypes) also blocks with no deadline. Fix: send quit without waiting for the answer, or read the answer on a separate thread with a timeout, so the destroyForcibly fallback always runs.

💡 Quality: Cancelled file-by-file parse is reported as a completed run

📄 java-frontend-semsitter/src/main/java/org/sonar/java/semsitter/SemSitterParser.java:109-123

When isCanceled turns true in FILE_BY_FILE mode, SemSitterParser.scan breaks out of the loop and then sets completed = true. The finally block therefore calls progressReport.stop(), which reports a finished run, instead of progressReport.cancel(). ECJ's JDTParser.FileByFile tracks a separate cancelled flag and calls cancel() in this case. The PR says cancellation is "reported like ECJ", but here a cancelled analysis is logged as complete. Fix: set completed only when the loop was not broken by cancellation.

💡 Quality: Provider claims SemSitter is used but always builds the ECJ parser

📄 java-frontend-semsitter/src/main/java/org/sonar/java/semsitter/SemSitterParserProvider.java:59 📄 java-frontend-semsitter/src/main/java/org/sonar/java/semsitter/SemSitterParserProvider.java:77-80 📄 java-frontend-semsitter/src/main/java/org/sonar/java/semsitter/SemSitterParserProvider.java:106-112

After this commit the public no-arg constructor passes Optional.empty() as the translator (SemSitterParserProvider.java:59), so create() always goes through orElseGet(defaultParser) and returns JDTParser (lines 109-111). configure() still sets enabled = true and logs "The Java analyzer will use the SemSitter frontend." (lines 77-80). With sonar.java.semsitter.enabled=true or the LaunchDarkly flag on, that log is false: ECJ parses every file, and the analysis still calls the flag service for nothing. The PR description still says SemSitterTranslator/JSema translate documents, but this commit deletes them. Suggested fix: when no translator is present, keep enabled false (and skip the flag lookup), or change the log so it doesn't claim SemSitter is in use.

Only enable (and only query the flag) when a translator is available
executable = Path.of(path.get());
if (translator.isEmpty()) {
  return;
}
enabled = configuration.getBoolean(ENABLED_PROPERTY).orElseGet(() -> isFlagOn(configuration));
✅ 1 closed
✅ Bug: NPE when a type is looked up after the SemSitter session closes

📄 java-frontend-semsitter/src/main/java/org/sonar/java/semsitter/JSema.java:2723-2729 📄 java-frontend-semsitter/src/main/java/org/sonar/java/semsitter/JSema.java:207-212 📄 java-frontend-semsitter/src/main/java/org/sonar/java/semsitter/JSema.java:381-385 📄 java-frontend-semsitter/src/main/java/org/sonar/java/semsitter/SemSitterParser.java:110 📄 java-frontend-semsitter/src/main/java/org/sonar/java/semsitter/SemSitterParser.java:190-202 📄 java-frontend-semsitter/src/main/java/org/sonar/java/semsitter/SemSitterParser.java:216-220
Session.types returns null once the session is closed, and its own doc says a type met after that "stays as it was". JSema.typeSymbol (line 383) and packageAnnotations (line 544) handle the null. JSema.declaresType (line 2726) does not: it passes the result straight to ingestTypes, which calls types.entrySet() and throws a NullPointerException. markUndeclared (line 211) has the same unguarded call.

This can happen in practice. JavaAstScanner.scan runs endOfAnalysis() after parse() returns, and by then SemSitterParser.scan has already closed the session in its try-with-resources. Any check that keeps symbols and queries them at the end of the analysis can reach declaresType and crash, for example through lookupGenericMember (line 1533) or servedConstant (line 1750). Fix: skip ingestTypes when the loader returns null, in both places.

🤖 Prompt for agents
Code Review: Adds a per-analysis parser provider so the Java frontend can be switched between ECJ and SemSitter behind a feature flag, with ECJ fallback on any failure. Approved with suggestions: `close()` can hang indefinitely on a stuck server because its `quit` exchange has no read timeout, so the forced-destroy fallback never runs. A cancelled file-by-file parse is also reported as a completed run instead of cancelled. And the no-arg provider always builds the ECJ parser while still logging that SemSitter is in use. An NPE when a type is looked up after the session closes was resolved.

1. 💡 Edge Case: close() can hang forever on a server that stops answering
   Files: java-frontend-semsitter/src/main/java/org/sonar/java/semsitter/SemSitterClient.java:180-194, java-frontend-semsitter/src/main/java/org/sonar/java/semsitter/SemSitterClient.java:200-209

   The `close()` javadoc says it "forc[es] it to stop when it does not answer". In fact `close()` first calls `send(command("quit"))`, which writes the request and then blocks in `readLine(fromServer)`. That read has no timeout. If the server process is still alive but stuck (deadlock, endless loop), `close()` never reaches `waitFor(10, SECONDS)` or `destroyForcibly()`, so the analysis hangs inside `SemSitterParser.scan`'s try-with-resources. Every other exchange (`getFile`, `getTypes`) also blocks with no deadline. Fix: send `quit` without waiting for the answer, or read the answer on a separate thread with a timeout, so the `destroyForcibly` fallback always runs.

2. 💡 Quality: Cancelled file-by-file parse is reported as a completed run
   Files: java-frontend-semsitter/src/main/java/org/sonar/java/semsitter/SemSitterParser.java:109-123

   When `isCanceled` turns true in `FILE_BY_FILE` mode, `SemSitterParser.scan` breaks out of the loop and then sets `completed = true`. The `finally` block therefore calls `progressReport.stop()`, which reports a finished run, instead of `progressReport.cancel()`. ECJ's `JDTParser.FileByFile` tracks a separate `cancelled` flag and calls `cancel()` in this case. The PR says cancellation is "reported like ECJ", but here a cancelled analysis is logged as complete. Fix: set `completed` only when the loop was not broken by cancellation.

3. 💡 Quality: Provider claims SemSitter is used but always builds the ECJ parser
   Files: java-frontend-semsitter/src/main/java/org/sonar/java/semsitter/SemSitterParserProvider.java:59, java-frontend-semsitter/src/main/java/org/sonar/java/semsitter/SemSitterParserProvider.java:77-80, java-frontend-semsitter/src/main/java/org/sonar/java/semsitter/SemSitterParserProvider.java:106-112

   After this commit the public no-arg constructor passes `Optional.empty()` as the translator (SemSitterParserProvider.java:59), so `create()` always goes through `orElseGet(defaultParser)` and returns `JDTParser` (lines 109-111). `configure()` still sets `enabled = true` and logs "The Java analyzer will use the SemSitter frontend." (lines 77-80). With `sonar.java.semsitter.enabled=true` or the LaunchDarkly flag on, that log is false: ECJ parses every file, and the analysis still calls the flag service for nothing. The PR description still says `SemSitterTranslator`/`JSema` translate documents, but this commit deletes them. Suggested fix: when no translator is present, keep `enabled` false (and skip the flag lookup), or change the log so it doesn't claim SemSitter is in use.

   Fix (Only enable (and only query the flag) when a translator is available):
   executable = Path.of(path.get());
   if (translator.isEmpty()) {
     return;
   }
   enabled = configuration.getBoolean(ENABLED_PROPERTY).orElseGet(() -> isFlagOn(configuration));

Verification Rules 2/2 met Repos 6 inspected · PRs 3 inspected

Functional validation No results

Rules 2 passed 2 not applicable

✅ Internal artifact registries
✅ Repository license compliance

Broader impact Repos 6 inspected PRs 3 inspected
Cross-repo 6 repositories inspected 0 open impacts

SonarSource/sonar-java-symbolic-execution Inspected
SonarSource/sonarlint-intellij Inspected
SonarSource/sonarlint-vscode Inspected
SonarSource/sonarlint-eclipse Inspected
SonarSource/sonar-iac Inspected

1 more repository

Cross-PR 3 PRs inspected 0 open impacts

PR 6541: SONARJAVA-7339 Add the SemSitter front-end module with its protobuf data model Inspected
PR 6542: SONARJAVA-7339 Add the SemSitter server client Inspected
PR 6501: SONARJAVA-7236 Experiment: let semsitter read sourcepath files that are not analyzed Inspected

Auto-approval Not enabled

Set up

For more verification details, comment gitar display:verbose.

Tip

Comment Gitar fix CI or enable auto-apply: gitar auto-apply:on

Options

Auto-apply is off → Gitar will not commit updates to this branch.
Display: compact → Showing compact verification results.

Comment with these commands to change the behavior for this request:

Auto-apply Compact
gitar auto-apply:on         
gitar display:verbose         

Was this helpful? React with 👍 / 👎 | Gitar

@sonarqube-next

Copy link
Copy Markdown
Contributor

Quality Gate failed Quality Gate failed

Failed conditions
8 New issues

See analysis details on SonarQube

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE SonarQube for IDE

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant