Coder Legion - Recent posts tagged openrewrite

Testing an OpenRewrite recipe

Thu, 26 Jun 2025 20:18:12 +0000

For the last two weeks, I've kicked the tires of OpenRewrite. At first, I created a recipe to move Kotlin source files as per the official recommendations with a set package name. I then improved the recipe to compute the root automatically. In both versions, I thoroughly tested the recipe. However, my testing approach was wrong. In this post, I want to describe my mistakes, and how I fixed them.

The naive approach

I originally approached the testing of the recipe in a very naive way, to say the least. As explained in the first post, I used OpenRewrite's low-level APIs. Here's what I wrote:

// Given
val parser = KotlinParser.builder().build()                                      //1
val cu = parser.parse(
    InMemoryExecutionContext(),                                                  //2
    sourceCode
).findFirst()                                                                    //3
 .orElseThrow { IllegalStateException("Failed to parse Kotlin file") }           //3
val originalPath = Paths.get(originalPath)
val modifiedCu = (cu as K.CompilationUnit).withSourcePath(originalPath)          //4

// When
val recipe = FlattenStructure(configuredRootPackage)
val result = recipe.visitor.visit(modifiedCu, InMemoryExecutionContext())        //5

// Then
val expectedPath = Paths.get(expectedPath)
assertEquals(expectedPath, (result as SourceFile).sourcePath)                    //6

Build the Kotlin parser
Set an execution context; I had to choose and the in-memory one was the easiest.
Boilerplate to get the single compilation unit from the stream
Cast to a K.CompilationUnit because we know better
Explicitly call the visitor to visit
Assert the recipe moved the file

The above works, but requires a deep understanding of how OpenRewrite works. I didn't have that understanding, but it was good enough. It came back to bite me when I improved the recipe to compute the root.

As explained in the last post, I switched from a regular recipe to a scanning recipe. I had to provide at least two source files to test the new capability. I came up with the following:

// When
val recipe = FlattenStructure()
val context = InMemoryExecutionContext()
val acc = AtomicReference//String?(null)
recipe.getScanner(acc).visit(modifiedCu1, context)                               //1
recipe.getScanner(acc).visit(modifiedCu2, context)                               //1
val result1 = recipe.getVisitor(acc).visit(modifiedCu1, context)                 //2
val result2 = recipe.getVisitor(acc).visit(modifiedCu2, context)                 //2

Get the scanner and visit source files to compute the root
Get the visitor and visit source files to move the file

It worked, but I admit it was a lucky guess. More involved recipes would require a deeper knowledge of how OpenRewrite works, with more potential bugs. Fortunately, OpenRewrite provides the means to keep the testing code at the right level of abstraction.

The nominal approach

The nominal approach involves a couple of out-of-the-box classes; it requires a new dependency. I didn't do it before, so now is a good time: let's introduce a BOM to align all of OpenRewrite's dependencies:

<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>org.openrewrite.recipe</groupId>
            <artifactId>rewrite-recipe-bom</artifactId>
            <version>3.9.0</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
    </dependencies>
</dependencyManagement>

It's now possible to add the dependency without a version, as Maven resolves it from the above BOM.

<dependency>
    <groupId>org.openrewrite</groupId>
    <artifactId>rewrite-test</artifactId>
    <scope>test</scope>
</dependency>

This brings a couple of new classes to the project:

The documentation states your test class should inherit from RewriteTest, which provides rewriteRun. The latter runs the recipe, without any need to know about its inner workings, e.g., the above in-memory execution context. It's the abstraction level that we want. Assertions offers static methods to assert. OpenRewrite also advises using Assertion4J, which I fully endorse. Yet, I didn't do it to keep the comparison simpler.

We can rewrite the previous snippet to:

rewriteRun(                                                                     //1
    kotlin(sourceCode1) { spec ->                                               //2-3-4
        spec.path("src/main/kotlin/ch/frankel/blog/foo/Foo.kt")                 //5
        spec.afterRecipe {                                                      //6
            assertEquals(                                                       //7
                Paths.get("src/main/kotlin/foo/Foo.kt"),
                it.sourcePath
            )
        }
    },
    kotlin(sourceCode2) { spec ->                                               //2-3-4
        spec.path("src/main/kotlin/org/frankel/blog/bar/Bar.kt")                //5
        spec.afterRecipe {                                                      //6
            assertEquals(                                                       //7
                Paths.get("src/main/kotlin/bar/Bar.kt"),
                it.sourcePath
            )
        }
    },
)

Run the recipe
kotlin transform the string into a SourceSpecs
I'm using Kotlin, but java does the same for regular Java projects
Allow customizing the source specification
Customize the path
Hook after running the recipe
Assert that the recipe updated the path according to the expectations

You may have noticed that the rewritten code doesn't specify which recipe it's testing. That's the responsibility of the RewriteTest.defaults() method.

class FlattenStructureComputeRootPackageTest : RewriteTest {

    override fun defaults(spec: RecipeSpec) {
        spec.recipe(FlattenStructure())
    }

    // Rest of the class
}

Don't forget cycles

If you followed the above instructions, there's a high chance your test fails with this error message:

java.lang.AssertionError: Expected recipe to complete in 0 cycle, but took 1 cycle. This usually indicates the recipe is making changes after it should have stabilized.

We need to turn to the documentation to understand this cryptic message:

>The recipes in the execution pipeline may produce changes that in turn cause another recipe to do further work. As a result, the pipeline may perform multiple passes (or cycles) over all the recipes in the pipeline again until either no changes are made in a pass or some maximum number of passes is reached (by default 3). This allows recipes to respond to changes made by other recipes which execute after them in the pipeline.

>-- Execution Cycles

Because the recipe doesn't rely on any other and no other recipe depends on it, we can set the cycle to 1.

override fun defaults(spec: RecipeSpec) {
    spec.recipe(FlattenStructure())
        .cycles(1)                                                              //1
        .expectedCyclesThatMakeChanges(1)                                       //2
}

Set how many cycles the recipe should run
Set to 0 if the recipe isn't expected to make changes

Criticisms

I like what the OpenRewrite testing classes bring, but I have two criticisms.

First and foremost, why does OpenRewrite assert the number of cycles by default? It bit me in the back for no good reason. I had to dig into the documentation and understand how OpenRewrite works, although the testing API is supposed to shield users from its inner workings. I also can't help but wonder about the defaults.

public class RecipeSpec {

    @Nullable
    Integer cycles;

    int getCycles() {
        return cycles == null ? 2 : cycles;                                     //1
    }

    int getExpectedCyclesThatMakeChanges(int cycles) {
        return expectedCyclesThatMakeChanges == null ? cycles - 1 :             //2
                expectedCyclesThatMakeChanges;
    }

    // Rest of the class body
}

Why two cycles by default? Shouldn't one be enough in most cases?
Why cycles - 1 by default?

My second criticism is about how the provided testing classes make you structure your tests. I like to structure them into three parts:

Given: describe the initial state
When: execute the to-be-tested code
Then: assert the final state conforms to what I expect.

With OpenRewrite's abstractions, the structure is widely different from the above.

Conclusion

In this post, I migrated my ad hoc test code to rely on OpenRewrite's provided classes. Even though they are not exempt from criticism, they offer a solid abstraction layer and make tests more maintainable.

The complete source code for this post can be found on GitHub.

To go further:

Originally published at A Java Geek on June 22^nd, 2025

Improving my previous OpenRewrite recipe

Thu, 19 Jun 2025 09:26:47 +0000

I started discovering OpenRewrite last week by writing a Kotlin recipe that moves Kotlin files according to the official directory structure recommendation. I mentioned some future works, and here they are. In this post, I want to describe how to compute the root package instead of letting the user set it.

Reminder

I developed last week a recipe to follow the Kotlin recommendation regarding directory structure:

>In pure Kotlin projects, the recommended directory structure follows the package structure with the common root package omitted. For example, if all the code in the project is in the org.example.kotlin package and its subpackages, files with the org.example.kotlin package should be placed directly under the source root, and files in org.example.kotlin.network.socket should be in the network/socket subdirectory of the source root.

>-- Directory structure

In a Java project, if you have packages ch.frankel.foo, ch.frankel.bar, and ch.frankel.baz, you'll get the following structure:

src
 |__ main
    |__ java
       |__ ch
          |__ frankel
             |__ foo
             |__ bar
             |__ baz

In Kotlin projects, many developers follow the same structure as above, but it can be flattened as:

src
 |__ main
    |__ kotlin
       |__ foo
       |__ bar
       |__ baz

The work

My recipe's original version mandated that you configure the root package yourself, e.g., ch.frankel for the above example. However, it should be possible to compute it automatically, from looking at the source files. It adds an extra step to the process: before moving the file to the root, the recipe should look at each source file, get the package, compute the longest prefix with the existing root, make it the root, and go to the next source file. The regular Recipe doesn't work in this case. We need to switch to a ScanningRecipe:

>If a recipe needs to generate new source files or needs to see all source files before making changes, it must be a ScanningRecipe. A ScanningRecipe extends the normal Recipe and adds two key objects: an accumulator and a scanner. The accumulator object is a custom data structure defined by the recipe itself to store any information the recipe needs to function. The scanner object is a visitor which populates the accumulator with data.

>-- Scanning recipes

Scanning recipes offer two steps: the first to gather data, the second to do the work.

We must design our algorithm within the constraints of OpenRewrite, and they are the following: in the first phase, for each source file, OpenRewrite will call the getScanner() method that returns a visitor of our choice. In turn, OpenRewrite calls the visitor's methods, which can access the accumulator.

My first naive approach was to use a collection as the accumulator, but it's not necessary. The algorithm is much simpler if we set a mutable placeholder that holds the package root and update it if necessary during each visit. The initial value should be null.

If the value is null, which happens on the first visitor, set the package root to the source file's package.
If the value is an empty string, skip–see below.
In any other case, compute the new package root by finding the longest prefix between the existing package root and the source file's package.

It might result in an empty string, indicating that packages have no common root, e.g., ch.frankel.foo and org.frankel.foo.

Here's the updated code:

class FlattenStructure(private val rootPackage: String?) : ScanningRecipe<AtomicReference<String?>>() { //1-2

  constructor() : this(null)                                                   //3

  override fun getDisplayName(): String = "Flatten Kotlin package directory structure"
  override fun getDescription(): String =
    "Move Kotlin files to match idiomatic layout by omitting the root package according to the official recommendation."

  override fun getInitialValue(ctx: ExecutionContext) = AtomicReference<String?>(null) //4

  override fun getScanner(acc: AtomicReference<String?>): TreeVisitor<*, ExecutionContext> {
    if (rootPackage != null) return TreeVisitor.noop<Tree, ExecutionContext>() //5
    return object : KotlinIsoVisitor<ExecutionContext>() {
      override fun visitCompilationUnit(cu: K.CompilationUnit, ctx: ExecutionContext): K.CompilationUnit {
        val packageName = cu.packageDeclaration?.packageName ?: return cu
        val computedPackage = acc.get()
        when (computedPackage) {
          null -> acc.set(packageName)                                         //6
          "" -> {}                                                             //7
          else -> {
            val commonPrefix = packageName.commonPrefixWith(computedPackage).removeSuffix(".") //8
            acc.set(commonPrefix)
          }
        }
        return cu
      }
    }
  }

  override fun getVisitor(acc: AtomicReference<String?>): TreeVisitor<*, ExecutionContext> {
    return object : KotlinIsoVisitor<ExecutionContext>() {
      override fun visitCompilationUnit(cu: K.CompilationUnit, ctx: ExecutionContext): K.CompilationUnit {
        val packageName = cu.packageDeclaration?.packageName ?: return cu
        val packageToSet: String? = rootPackage ?: acc.get()                   //9
        if (packageToSet == null || packageToSet.isEmpty()) return cu
        val relativePath = packageName.removePrefix(packageToSet).removePrefix(".")
          .replace('.', '/')
        val filename = cu.sourcePath.fileName.toString()
        val newPath: Path = Paths.get("src/main/kotlin")
          .resolve(relativePath)
          .resolve(filename)
        return cu.withSourcePath(newPath)
      }
    }
  }
}

Inherit from ScanningRecipe instead of directly from Recipe
Set the accumulator type to be an AtomicReference<String?>
Make the configuration easier when you don't override the package root
The initial root is uninitialized
Skip the computation if the root package is manually set
If it's the first file visited, the accumulator holds null, and we can set the (temporary) root as the current package
One of the previous computations returned no common root–do nothing
Find the longest common prefix between the held package root and the current package
The only difference with the original code:
we check if the root package has been set manually otherwise, we use the one computed in the first pass

Optimizing the recipe

You may have noticed that when there is no common root, e.g., ch.frankel.foo and org.frankel.foo, we scan all files anyway. In a small codebase, it's not a big issue, but when scanning millions of source files, that's a huge waste of CPU cycles and time. If you run the recipe in the Cloud, it directly translates to money. We should stop scanning as soon as we detect the computed package root is an empty string to optimize the recipe.

Here's the updated code:

override fun getScanner(acc: AtomicReference<String?>): TreeVisitor<*, ExecutionContext> {
  if (rootPackage != null) return TreeVisitor.noop<Tree, ExecutionContext>()   //1
  val currentPackage = acc.get()
  if (currentPackage == "") return TreeVisitor.noop<Tree, ExecutionContext>()  //2
  return object : KotlinIsoVisitor<ExecutionContext>() {
    override fun visitCompilationUnit(cu: K.CompilationUnit, ctx: ExecutionContext): K.CompilationUnit {
      val packageName = cu.packageDeclaration?.packageName ?: return cu
      // Different call than the one above!
      val currentPackage = acc.get()
      // First scanned file
      if (currentPackage == null) acc.set(packageName)                         //3
      else {
        // Find the longest common prefix between the stored package and the current one
        val commonPrefix = packageName.commonPrefixWith(currentPackage).removeSuffix(".")
        acc.set(commonPrefix)
      }
      return cu
    }
  }
}

If the root package has been set, skip visiting
If one of the previous computations sets an empty string, there isn't any common root package: skip visiting
Simplify the visitor by removing the clause when the accumulator is an empty string since it can't happen anymore

Note that OpenRewrite still scans each file, but at least doesn't visit it thanks to the no-op visitor.

Counting visits

I made a couple of attempts before finding the right approach to the above. To ensure that I got it right, I wanted to display the number of visits by the scanner. We can use the accumulator to increment the visit count. Here are the changes I made:

Migrated from an AtomicReference<String?> to an AtomicReference<Pair<Int, String?>>
The getInitialValue() function returns AtomicReference<Pair<Int, String?>>(0 to null)
During each visit:
- Get the visits count from the accumulator
- Increment it
- Print it
- Store it back in the accumulator
Update the tests accordingly

With packages ch.frankel.blog.foo, org.frankel.blog.bar, and org.frankel.blog.baz, the log shows:

[INFO] [stdout] FlattenStructure: 1 file visited
[INFO] [stdout] FlattenStructure: 2 files visited

By changing ch.frankel.blog.foo to org.frankel.blog.foo, the log changes to:

[INFO] [stdout] FlattenStructure: 1 files visited
[INFO] [stdout] FlattenStructure: 2 files visited
[INFO] [stdout] FlattenStructure: 3 files visited

Because I made these changes just to validate my understanding of how OpenRewrite works, I put them in the visits_count branch on GitHub. To see the differences, execute git diff master visits_count.

Conclusion

In this post, I added the automatic computation of the root package. I had to change my design and understand how scanning recipes work. Then, I skipped further visits when there wasn't any common root package to optimize performance.

The recipe is still not serializable, though it's a recommendation. I also noticed that my tests didn't leverage OpenRewrite's testing API. There's still a lot of work to do!

The complete source code for this post can be found on GitHub.

To go further:

Originally published at A Java Geek on June 15^th, 2025