Skip to content

Commit 8a7e2b4

Browse files
authored
Merge branch '21.X.X' into abashev/Fix-imports-out-of-process
2 parents 7c38ef0 + 48ebc8a commit 8a7e2b4

20 files changed

Lines changed: 762 additions & 56 deletions

File tree

‎build.gradle‎

Lines changed: 7 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ buildscript {
77
dependencies {
88
classpath 'com.gradle.publish:plugin-publish-plugin:2.2.1'
99
// The released open-java-format formats this repository, not the one being built here.
10-
classpath 'dev.openjavaformat:gradle-open-java-format:2.98.0.6'
10+
classpath 'dev.openjavaformat:gradle-open-java-format:21.0.2'
1111
classpath 'me.champeau.jmh:jmh-gradle-plugin:0.7.3'
1212
}
1313
}
@@ -73,16 +73,19 @@ allprojects {
7373
}
7474

7575
repositories {
76-
mavenCentral() { metadataSources { mavenPom(); ignoreGradleMetadataRedirection() } }
77-
gradlePluginPortal()
76+
mavenCentral() {
77+
metadataSources {
78+
mavenPom(); ignoreGradleMetadataRedirection()
79+
}
80+
}
7881
}
7982
}
8083

8184
subprojects {
8285
apply plugin: 'java-library'
83-
// Java toolchains, javac and test settings; Error Prone. Both in buildSrc/src/main/groovy.
8486
apply plugin: 'open-java-format.java-conventions'
8587
apply plugin: 'open-java-format.error-prone-conventions'
88+
8689
// Bans Guava's immutable collections, but not in the SPI: FormatterService.getFormatReplacements keeps
8790
// returning ImmutableList, so that code compiled against an older SPI still links.
8891
if (path != ':open-java-format-spi') {
Lines changed: 77 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,77 @@
1+
# ADR-001: Two entry points, and one output for each on every surface
2+
3+
## Status
4+
Proposed
5+
6+
## Date
7+
2026-11-14
8+
9+
## Context
10+
11+
open-java-format runs on many surfaces: the command line and its native binary, the GitHub Action and the pre-commit hook (both run the native binary on whole files), the playground, the Maven plugin, our Gradle Spotless step, Spotless' own `palantirJavaFormat()` step, `formatDiff` (on the JVM or with the native binary), IntelliJ IDEA (Java-based or native) and Eclipse.
12+
13+
Each of them reaches the formatter through code of its own, and they had drifted apart (#219):
14+
15+
- `FormatFileCallable.formatFile`, behind the command line, lays out the code, then removes unused imports, sorts them and reflows long strings. `Formatter.formatSourceAndFixImports`, behind the Maven plugin and the Spotless step, fixes the imports first. Where a removed import has a comment, the two write different files: 1 of 3,865 real files.
16+
- Spotless' own step calls `ImportOrderer.reorderImports`, `RemoveUnusedImports.removeUnusedImports` and `Formatter.formatSource`, and never reflows long strings: 94 of the 3,865 files differ.
17+
- `formatDiff` on the JVM goes through `Formatter.getFormatReplacements` and never reflows; with the native binary, and in IntelliJ IDEA, `--output-replacements` reflows, but every long string in the file once one line in the ranges needs it.
18+
- `--lines` and `--offset` printing text format the ranges, then fix the imports and reflow the whole file.
19+
- Tests pinned the pieces, not the surfaces: the 915 cases of `FormatterCasesTest` went through `formatSourceAndFixImports`, and give the same output in both import orders, so none of this showed.
20+
21+
A file that one surface formats then fails the check of another, and nothing says which output is right.
22+
23+
## Decision
24+
25+
There are two entry points, and every surface goes through one of them:
26+
27+
| Entry | Does | Never does |
28+
|---|---|---|
29+
| **Whole file** | lays out the code, removes unused imports and sorts them, reflows long strings | — |
30+
| **Part of a file** (character or line ranges) | lays out the code the ranges cover, reflows the long strings in that code | touch the imports, or code outside what formatting the ranges rewrites |
31+
32+
For the same formatter version and input, every surface of an entry gives the same result: the same bytes, or the same diagnostics, for a whole file; the same replacements for a part of one. The command line is the reference for the whole file, since the GitHub Action and the pre-commit hook check against it.
33+
34+
- IntelliJ IDEA's Reformat is the part-of-a-file entry, also over a whole file, and leaves the imports alone. Optimize imports stays the IDE's own until the plugin's optimizer replaces it ([ADR-003](ADR-003-imports-in-intellij-idea.md), #222).
35+
- `--lines`, `--offset` and `--character-ranges` printing text are the part-of-a-file entry, and do not fix imports.
36+
- Eclipse is a surface of the part-of-a-file entry.
37+
38+
Accepted exceptions, each documented:
39+
40+
- **The order of fixing imports** stays as it is until #224 is decided: after the layout on the command line, before it in `formatSourceAndFixImports` and in Spotless' own step ([ADR-004](ADR-004-order-of-fixing-imports.md)).
41+
- **Spotless' own step does not reflow long strings.** `Formatter.formatSource` keeps its meaning; the docs site's Spotless page says so.
42+
- **Syntax of the runtime.** The formatter parses with the javac of the JVM it runs on; the native binary has GraalVM 25's. Code that needs a newer Java than the runtime fails there. The contract holds for code the runtime can parse, and tests that need a newer JDK run in classes of their own, with case files in a folder per JDK ([ADR-002](ADR-002-tests-by-jdk.md)).
43+
- **A file that does not parse** keeps today's behaviour: in-process services throw, out-of-process ones return no replacements.
44+
45+
## Alternatives Considered
46+
47+
### Leave each surface as it is
48+
- Pros: no change for anyone.
49+
- Cons: a file formatted on one surface fails another's check; no output is the right one.
50+
- Rejected: the point of a formatter shared by the build, the IDE and CI is that they agree.
51+
52+
### One entry: IntelliJ IDEA formats a whole file with imports, in one pass
53+
- Pros: Reformat of a whole file would give the command line's bytes with no IDE action in between.
54+
- Cons: Reformat would change imports even when Optimize imports is off.
55+
- Rejected: imports in IntelliJ IDEA go through the IDE's Optimize imports, replaced later by ours ([ADR-003](ADR-003-imports-in-intellij-idea.md)).
56+
57+
### Make `Formatter.formatSource` reflow long strings, for Spotless' own step
58+
- Pros: the only way from our side to give Spotless' step the command line's long strings.
59+
- Cons: it changes what a public method does, for every caller.
60+
- Rejected: Spotless' step stays an accepted exception.
61+
62+
### Let the part-of-a-file entry fix imports, as `--lines` does today
63+
- Pros: no change for users of `--lines`.
64+
- Cons: a range would change code outside it.
65+
- Rejected: a range touches only the code it covers.
66+
67+
### Decide the import order now
68+
- Pros: removes the last exception for whole files.
69+
- Cons: either order changes the output of some surface; google-java-format has just moved to fixing imports before the layout.
70+
- Deferred to #224 ([ADR-004](ADR-004-order-of-fixing-imports.md)).
71+
72+
## Consequences
73+
74+
- Each change that brings a surface to its entry is a pull request of its own, with a check through that surface which fails before the change: the in-process service returns what `--output-replacements` prints; the part-of-a-file entry reflows only in the ranges; `--lines` and `--offset` print what the replacements give; Eclipse reflows like the others; the out-of-process `fixImports` passes `--fix-imports-only` (#221).
75+
- Tests that pin what users get go through a surface, `Main` or a `FormatterService`, not through `ImportOrderer`, `RemoveUnusedImports` or `StringWrapper` on their own (#225).
76+
- Users of `--lines`, `--offset` and `--character-ranges` as text see imports left alone and long strings reflowed only in the ranges.
77+
- The exceptions stay until their own decisions: #224 for the import order; Spotless' step for as long as `formatSource` keeps its meaning.
Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
# ADR-002: Tests that need a newer JDK live in classes and folders of their own
2+
3+
## Status
4+
Proposed
5+
6+
## Date
7+
2026-11-14
8+
9+
## Context
10+
11+
The formatter parses with the javac of the JVM it runs on, so some of its tests can only pass on a newer JDK than the one the build compiles for (Java 21). CI runs the tests on JDK 21 in its build job and on 25, 26, 27 and 28-ea in its `jdk` jobs.
12+
13+
Today such tests skip themselves from inside, in three ways:
14+
15+
- `Assumptions.assumeTrue(Formatter.getRuntimeVersion() >= 23)`: the 7 tests of `ModuleImportTest`, in a `@BeforeAll`, and `moduleImportStaysAmongOrdinaryImports` in `RemoveUnusedImportsTest`.
16+
- `FileBasedTests.VERSIONED_TESTS`, a map from a minimum version to test names, read by `assumeJavaVersionForTest`: the `ModuleImport` pair of `FormatterIntegrationTest`, 4 runs.
17+
- `Assumptions.assumeTrue` on `Modifier.VALUE` existing: `valueClassAndRecord` in `FormatterTest` and `valueClass` in `ModifierOrdererTest`, for the value classes of JEP 401, a preview of JDK 28, whose early-access builds have the `value` modifier.
18+
19+
On JDK 21, where a local build runs, these 14 tests count as skipped among 1,422, and nothing says a change broke them. That is how the first push of #220 passed locally and failed CI's JDK 25 to 28 builds on `ModuleImportTest`. The case files of `FormatterCasesTest`, in `test-cases/`, have no way to say which JDK they need, so a case of newer syntax cannot be written at all.
20+
21+
## Decision
22+
23+
A test that needs a newer JDK says so on its class, and its case files live in a folder of that JDK:
24+
25+
- **Case files by JDK, side by side.** `test-cases/` becomes `test-cases-java21/`, for any JDK from 21; cases that need JDK 25 go to `test-cases-java25/` beside it, those that need JDK 28 to `test-cases-java28/`, and so on, one level deep.
26+
- **Classes by JDK.** A test class that needs a newer Java carries `@EnabledForJreRange` with that version, as `min = JRE.JAVA_25` (JUnit 6.1.3 has constants up to `JRE.JAVA_28`). On an older JDK it reports as disabled; on that JDK and later it runs with the JDK 21 tests.
27+
- **One class per JDK for the case folders, JDK 21 too.** Each names the folders its JDK parses, one, two or three of them, newest first, as `FormatterCasesJava25Test` runs `test-cases-java25` and `test-cases-java21`; each is enabled from its JDK on, `min = JRE.JAVA_21`, `JAVA_25` or `JAVA_28`, with no upper bound, so on a newer JDK the older folders run again in the newer classes. The code they share takes the folders as arguments.
28+
- **Module imports from JDK 25.** They parse from JDK 23 as a preview, and are final in 25; CI runs no 23 or 24.
29+
- **Preview features by the version that previews them.** Value classes are a preview of JDK 28: their formatting case goes to `test-cases-java28/`, and the `ModifierOrderer` test to a class with `@EnabledForJreRange(min = JRE.JAVA_28)`. CI's JDK 28 job runs early-access builds, which have the `value` modifier.
30+
- **No assumption on the runtime inside a test.** `assumeTrue` on the runtime and `VERSIONED_TESTS` go.
31+
32+
## Alternatives Considered
33+
34+
### Keep the assumptions inside the tests
35+
- Pros: no change.
36+
- Cons: three mechanisms; a skipped test inside a class that runs looks like a pass; case files cannot say what they need.
37+
- Rejected: a test that cannot run here should not run here, and say so.
38+
39+
### Folders inside `test-cases/`, as `test-cases/java25/`
40+
- Pros: one root for the cases.
41+
- Cons: a second level of nesting under folders that already have subfolders (`imports`, `jbang`, `stringwrapper`, `testdata`); the JDK 21 cases would have to be told apart from the JDK subfolders.
42+
- Rejected: one folder per JDK, side by side.
43+
44+
### Module imports from JDK 23
45+
- Pros: the tests start where the syntax first parses.
46+
- Cons: a preview there, and no CI job runs 23 or 24.
47+
- Rejected: JDK 25, where they are final and CI runs.
48+
49+
### A condition on the `value` modifier for value classes
50+
- Pros: the tests would run on any JDK build that has the modifier, whatever its version.
51+
- Cons: a second kind of condition, and a case file cannot carry it; the early-access builds of JDK 28, where the feature is previewed, have the modifier.
52+
- Rejected: preview features go by the version, as the rest.
53+
54+
## Consequences
55+
56+
- `FormatterCasesTest` keeps the code that runs the cases; `FormatterCasesJava21Test`, `FormatterCasesJava25Test` and `FormatterCasesJava28Test` run them. A new JDK folder means a new class that names it and the older ones. The comments that name `test-cases/`, in `Level.java` and `RemoveUnusedImportsTest`, change with it.
57+
- The `ModuleImport` pair of `FormatterIntegrationTest` becomes a case in `test-cases-java25/`; `ModuleImportTest` and the module-import test of `RemoveUnusedImportsTest` carry `@EnabledForJreRange(min = JRE.JAVA_25)`.
58+
- The value-class test of `FormatterTest` becomes a case in `test-cases-java28/`; the one of `ModifierOrdererTest` moves to a class with `@EnabledForJreRange(min = JRE.JAVA_28)`.
59+
- On JDK 23 and 24 the module-import tests no longer run.
60+
- Before a push, the tests run on JDK 21 and 25 locally: `./gradlew test -PjavaRuntime=25 -PtestkitGradleVersion=9.3.1`; a change to the JDK 28 tests also on an early-access build of 28, with `-PjavaRuntime=28 -Porg.gradle.java.installations.paths=<its home>`.
61+
- Comparing the native binary, which parses JDK 25 code whatever the test JVM, with an in-process surface on `test-cases-java25/` needs a test JVM of 25 or later.
Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
# ADR-003: In IntelliJ IDEA, Reformat leaves the imports, and Optimize imports is to become ours
2+
3+
## Status
4+
Proposed
5+
6+
## Date
7+
2026-11-14
8+
9+
## Context
10+
11+
With the plugin enabled, IntelliJ IDEA's Reformat runs open-java-format through the part-of-a-file entry ([ADR-001](ADR-001-one-output-per-entry-point.md)): `Main --character-ranges … --output-replacements`, which never touches imports. Imports are left to the IDE's Optimize imports, and that is the IDE's own `JavaImportOptimizer`, with the IDE's import layout and its thresholds for `import a.b.*;`. The command line, the GitHub Action and the build plugins then sort the imports again with `ImportOrderer`.
12+
13+
In the sources of IntelliJ IDEA 2024.2.6, the platform the plugin is built against:
14+
15+
- Every path that optimizes imports goes through `OptimizeImportsProcessor.collectOptimizers`: Optimize Imports; Reformat Code and Reformat Files with "Optimize imports"; Actions on Save; "Optimize imports" before a commit; the quick fix for an unused import; "Optimize imports on the fly". It asks `FormattingServiceUtil.findImportsOptimizingService(file)` for a service with `Feature.OPTIMIZE_IMPORTS`, and takes that service's `getImportOptimizers`.
16+
- The plugin's `PalantirJavaFormatFormattingService.getFeatures()` returns `FORMAT_FRAGMENTS` only, and `getImportOptimizers` comes from `AbstractDocumentFormattingService`, which returns none; so `CoreFormattingService` answers with the IDE's optimizer.
17+
- `processFile` of an optimizer runs in a background task under a read action (`OptimizeImportsProcessor.prepareTask`), so the time of one call counts.
18+
- With "Optimize imports" on, Reformat Code on files and Reformat in the editor over the whole file run the formatter first and the optimizer after it (`AbstractLayoutCodeProcessor.getAllProcessors`, `FileInEditorProcessor`); over "only VCS-changed text" the editor runs the optimizer first.
19+
20+
The plugin had an optimizer of its own for three weeks. google-java-format added one when it moved to the `FormattingService` API (google/google-java-format@84b2c9a2, 2023-03-02), running the formatter inside the IDE, for which users add `--add-exports` to the IDE's VM options. palantir/palantir-java-format#1078 followed it on 2024-04-26, and palantir/palantir-java-format#1089 removed the optimizer on 2024-05-21 as "slow". It called `FormatterService.fixImports`, which out of process runs `Main` without `--fix-imports-only`: a new JVM, and the whole file formatted, on every Optimize imports (#221). With the flag, one call on a small file takes about 20 ms with the native binary and 0.3 s with a new JVM.
21+
22+
## Decision
23+
24+
- **Reformat never changes imports**, over a part of a file or a whole one. It is the part-of-a-file entry.
25+
- **Imports change only through Optimize imports.** Until the plugin's optimizer exists, that is the IDE's own.
26+
- **The plugin's optimizer is to replace the IDE's** (#222): `getFeatures()` adds `OPTIMIZE_IMPORTS`, and `getImportOptimizers` returns an optimizer that calls `FormatterService.fixImports`. From then on it runs whenever Optimize imports is on, on every path listed above.
27+
- **`fixImports` out of process passes `--fix-imports-only` first** (#221).
28+
29+
## Alternatives Considered
30+
31+
### Reformat of a whole file fixes imports too, in one pass
32+
- Pros: Reformat of a whole file would give the command line's bytes, the imports included, without Optimize imports.
33+
- Cons: Reformat would change imports with Optimize imports off, which the IDE's users do not expect of it.
34+
- Rejected: imports belong to Optimize imports.
35+
36+
### Keep the IDE's optimizer, and set its import settings to open-java-format's
37+
- Pros: no optimizer of ours; a `CodeStyleSettingsModifier` can set the Java import settings for each file, as EditorConfig does.
38+
- Cons: the IDE's sorting is still the IDE's, not `ImportOrderer`'s.
39+
- Not chosen as the answer; a settings modifier may still help the imports the IDE inserts itself, through auto-import and completion, which no optimizer sees.
40+
41+
### An optimizer running the formatter inside the IDE, as google-java-format does
42+
- Pros: no process per call.
43+
- Cons: the javac packages have to be reachable from the IDE's JVM.
44+
- Left to how the plugin runs the formatter (#223); the optimizer calls `FormatterService.fixImports` whichever way that is.
45+
46+
## Consequences
47+
48+
- Until #222, imports in IntelliJ IDEA follow the IDE's settings, and the command line or CI may sort them again.
49+
- #222 depends on #221.
50+
- The optimizer's time counts inside a read action: about 20 ms a call with the native binary, about 0.3 s with a new JVM, on a small file. "Optimize imports on the fly" runs while typing.
51+
- The plugin's formatting service is asynchronous, and google-java-format's optimizer notes that it "can overwrite the results from the main formatter": the optimizer has to be checked in a running IDE.

0 commit comments

Comments
 (0)