Skip to content

Linkage Checker Gradle plugin to work with circular dependencies #2174

Closed

Description

@suztomo

Linkage Checker Gradle plugin to work with circular dependencies.

  • Find an artifact that has circular dependency jdom?
  • Create a test
  • Fix the issue

Activity

  1. suztomo commented on Aug 11, 2021

    @suztomo
    ContributorAuthor

    I thought dom4j:dom4j:1.6.1 (compile) / jaxen:jaxen:1.1-beta-6 (compile, optional) would help but because it's an optional dependency, it doesn't work.

    Let's try creating such relationship in local artifacts.

  2. suztomo commented on Aug 11, 2021

    @suztomo
    ContributorAuthor

    Created:

    suztomo-macbookpro44% mvn dependency:tree -Dverbose
    [INFO] Scanning for projects...
    [INFO] 
    [INFO] ------------------------< org.example:moduleB >-------------------------
    [INFO] Building moduleB 2.0-SNAPSHOT
    [INFO] --------------------------------[ jar ]---------------------------------
    [INFO] 
    [INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ moduleB ---
    [INFO] org.example:moduleB:jar:2.0-SNAPSHOT
    [INFO] +- (org.example:moduleB:jar:1.0-SNAPSHOT:compile - omitted for cycle)
    [INFO] \- org.example:moduleA:jar:1.0-SNAPSHOT:compile
    [INFO] ------------------------------------------------------------------------
    [INFO] BUILD SUCCESS
    [INFO] ------------------------------------------------------------------------
    [INFO] Total time:  1.194 s
    [INFO] Finished at: 2021-08-11T09:49:07-04:00
    [INFO] ------------------------------------------------------------------------
    

    With the following pom file installed:

    suztomo-macbookpro44% cat ~/.m2/repository/org/example/moduleB/2.0-SNAPSHOT/moduleB-2.0-SNAPSHOT.pom
    <project xmlns="http://maven.apache.org/POM/4.0.0"
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
      <parent>
        <artifactId>circular-deps</artifactId>
        <groupId>org.example</groupId>
        <version>2.0-SNAPSHOT</version>
      </parent>
      <modelVersion>4.0.0</modelVersion>
    
      <artifactId>moduleB</artifactId>
    
      <dependencies>
        <dependency>
          <artifactId>moduleA</artifactId>
          <groupId>org.example</groupId>
          <version>1.0-SNAPSHOT</version>
        </dependency>
      </dependencies>
      <properties>
        <maven.compiler.source>8</maven.compiler.source>
        <maven.compiler.target>8</maven.compiler.target>
      </properties>
    
    </project>
    
     suztomo-macbookpro44% cat ~/.m2/repository/org/example/moduleA/1.0-SNAPSHOT/moduleA-1.0-SNAPSHOT.pom
    <project xmlns="http://maven.apache.org/POM/4.0.0"
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
      <parent>
        <artifactId>circular-deps</artifactId>
        <groupId>org.example</groupId>
        <version>1.0-SNAPSHOT</version>
      </parent>
      <modelVersion>4.0.0</modelVersion>
    
      <artifactId>moduleA</artifactId>
    
      <dependencies>
        <dependency>
          <artifactId>moduleB</artifactId>
          <groupId>org.example</groupId>
          <version>1.0-SNAPSHOT</version>
        </dependency>
      </dependencies>
    
      <properties>
        <maven.compiler.source>8</maven.compiler.source>
        <maven.compiler.target>8</maven.compiler.target>
      </properties>
    
    </project>
    

    This Gradle build succeeds:

    suztomo-macbookpro44% cat build.gradle 
    plugins {
        id 'java'
        id 'com.google.cloud.tools.linkagechecker' version "1.5.9"
    }
    
    group 'org.example'
    version '1.0-SNAPSHOT'
    
    repositories {
        mavenCentral()
        mavenLocal()
    }
    
    // This module has cycle
    // [INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ moduleB ---
    // [INFO] org.example:moduleB:jar:2.0-SNAPSHOT
    // [INFO] +- (org.example:moduleB:jar:1.0-SNAPSHOT:compile - omitted for cycle)
    // [INFO]  \- org.example:moduleA:jar:1.0-SNAPSHOT:compile
    dependencies {
        compile 'org.example:moduleB:2.0-SNAPSHOT'
    }
    
    linkageChecker {
        configurations = ['compile']
    }
    
    suztomo-macbookpro44%  ./gradlew  linkageCheck
    
    Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
    Use '--warning-mode all' to show the individual deprecation warnings.
    See https://docs.gradle.org/6.8/userguide/command_line_interface.html#sec:command_line_warnings
    
    BUILD SUCCESSFUL in 1s
    1 actionable task: 1 executed
    

    A test case that uses local artifacts. https://gist.github.com/suztomo/473ea281f6deebf8804197bec53a873c

  3. suztomo commented on Aug 11, 2021

    @suztomo
    ContributorAuthor

    The code from LinkageCheckTask.java

      private static ClassPathResult createClassPathResult(ResolvedConfiguration configuration) {
        DependencyGraph dependencyGraph = createDependencyGraph(configuration);
    

    It's either

    • Gradle failed to give "configuration" (list of dependencies) to the plugin (very unlikely), or
    • Our createDependencyGraph fails to record visited node while iterating the dependencies from the "configuration".

    Maybe we could add debug output in the plugin.

  4. suztomo commented on Aug 11, 2021

    @suztomo
    ContributorAuthor

    Found a real-world example:

    In the dependency tree of com.google.cloud:google-cloud-bigquery:1.137.1:

    |    +--- com.fasterxml.jackson.core:jackson-core:2.12.3
    |    |    \--- com.fasterxml.jackson:jackson-bom:2.12.3
    |    |         \--- com.fasterxml.jackson.core:jackson-core:2.12.3 (c)
    

    None of the artifacts above define dependency on jackson-bom. Why does Gradle think com.fasterxml.jackson.core:jackson-core:2.12.3 depends on com.fasterxml.jackson:jackson-bom?

    Maven also thinks that jackson-core doesn't depend on jackson-bom.

    suztomo-macbookpro44% mvn dependency:tree -Dverbose
    [INFO] Scanning for projects...
    [INFO] 
    [INFO] ---------------< org.example:circular-dependency-gradle >---------------
    [INFO] Building circular-dependency-gradle 1.0-SNAPSHOT
    [INFO] --------------------------------[ jar ]---------------------------------
    [INFO] 
    [INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ circular-dependency-gradle ---
    [INFO] org.example:circular-dependency-gradle:jar:1.0-SNAPSHOT
    [INFO] \- com.fasterxml.jackson.core:jackson-core:jar:2.12.3:compile
    [INFO] ------------------------------------------------------------------------
    
  5. suztomo commented on Aug 11, 2021

    @suztomo
    ContributorAuthor

    Now, it outputs too many circulear dependencies message for artifacts with large dependencies.

    Circular dependency for: com.fasterxml.jackson:jackson-bom:2.12.1
     The stack is: [project :, org.apache.beam:beam-runners-direct-java:2.30.0, org.apache.beam:beam-sdks-java-core:2.30.0, com.fasterxml.jackson.core:jackson-annotations:2.12.1, com.fasterxml.jackson:jackson-bom:2.12.1, com.fasterxml.jackson.core:jackson-core:2.12.1]
    Circular dependency for: com.fasterxml.jackson.core:jackson-annotations:2.12.1
     The stack is: [project :, org.apache.beam:beam-runners-direct-java:2.30.0, org.apache.beam:beam-sdks-java-core:2.30.0, com.fasterxml.jackson.core:jackson-annotations:2.12.1, com.fasterxml.jackson:jackson-bom:2.12.1, com.fasterxml.jackson.core:jackson-databind:2.12.1]
    Circular dependency for: com.fasterxml.jackson:jackson-bom:2.12.1
     The stack is: [project :, org.apache.beam:beam-runners-direct-java:2.30.0, org.apache.beam:beam-sdks-java-core:2.30.0, com.fasterxml.jackson.core:jackson-annotations:2.12.1, com.fasterxml.jackson:jackson-bom:2.12.1, com.fasterxml.jackson.core:jackson-databind:2.12.1, com.fasterxml.jackson.core:jackson-core:2.12.1]
    Circular dependency for: com.fasterxml.jackson:jackson-bom:2.12.1
     The stack is: [project :, org.apache.beam:beam-runners-direct-java:2.30.0, org.apache.beam:beam-sdks-java-core:2.30.0, com.fasterxml.jackson.core:jackson-annotations:2.12.1, com.fasterxml.jackson:jackson-bom:2.12.1, com.fasterxml.jackson.core:jackson-databind:2.12.1]
    Circular dependency for: com.fasterxml.jackson.core:jackson-annotations:2.12.1
     The stack is: [project :, org.apache.beam:beam-runners-direct-java:2.30.0, org.apache.beam:beam-sdks-java-core:2.30.0, com.fasterxml.jackson.core:jackson-databind:2.12.1, com.fasterxml.jackson.core:jackson-annotations:2.12.1, com.fasterxml.jackson:jackson-bom:2.12.1]
    Circular dependency for: com.fasterxml.jackson:jackson-bom:2.12.1
     The stack is: [project :, org.apache.beam:beam-runners-direct-java:2.30.0, org.apache.beam:beam-sdks-java-core:2.30.0, com.fasterxml.jackson.core:jackson-databind:2.12.1, com.fasterxml.jackson.core:jackson-annotations:2.12.1, com.fasterxml.jackson:jackson-bom:2.12.1, com.fasterxml.jackson.core:jackson-core:2.12.1]
    Circular dependency for: com.fasterxml.jackson.core:jackson-databind:2.12.1
     The stack is: [project :, org.apache.beam:beam-runners-direct-java:2.30.0, org.apache.beam:beam-sdks-java-core:2.30.0, com.fasterxml.jackson.core:jackson-databind:2.12.1, com.fasterxml.jackson.core:jackson-annotations:2.12.1, com.fasterxml.jackson:jackson-bom:2.12.1]
    Circular dependency for: com.fasterxml.jackson:jackson-bom:2.12.1
     The stack is: [project :, org.apache.beam:beam-runners-direct-java:2.30.0, org.apache.beam:beam-sdks-java-core:2.30.0, com.fasterxml.jackson.core:jackson-databind:2.12.1, com.fasterxml.jackson.core:jackson-core:2.12.1, com.fasterxml.jackson:jackson-bom:2.12.1, com.fasterxml.jackson.core:jackson-annotations:2.12.1]
    Circular dependency for: com.fasterxml.jackson.core:jackson-core:2.12.1
     The stack is: [project :, org.apache.beam:beam-runners-direct-java:2.30.0, org.apache.beam:beam-sdks-java-core:2.30.0, com.fasterxml.jackson.core:jackson-databind:2.12.1, com.fasterxml.jackson.core:jackson-core:2.12.1, com.fasterxml.jackson:jackson-bom:2.12.1]
    Circular dependency for: com.fasterxml.jackson.core:jackson-databind:2.12.1
     The stack is: [project :, org.apache.beam:beam-runners-direct-java:2.30.0, org.apache.beam:beam-sdks-java-core:2.30.0, com.fasterxml.jackson.core:jackson-databind:2.12.1, com.fasterxml.jackson.core:jackson-core:2.12.1, com.fasterxml.jackson:jackson-bom:2.12.1]
    Circular dependency for: com.fasterxml.jackson:jackson-bom:2.12.1
     The stack is: [project :, org.apache.beam:beam-runners-direct-java:2.30.0, org.apache.beam:beam-sdks-java-core:2.30.0, com.fasterxml.jackson.core:jackson-databind:2.12.1, com.fasterxml.jackson:jackson-bom:2.12.1, com.fasterxml.jackson.core:jackson-annotations:2.12.1]
    Circular dependency for: com.fasterxml.jackson:jackson-bom:2.12.1
     The stack is: [project :, org.apache.beam:beam-runners-direct-java:2.30.0, org.apache.beam:beam-sdks-java-core:2.30.0, com.fasterxml.jackson.core:jackson-databind:2.12.1, com.fasterxml.jackson:jackson-bom:2.12.1, com.fasterxml.jackson.core:jackson-core:2.12.1]
    Circular dependency for: com.fasterxml.jackson.core:jackson-databind:2.12.1
     The stack is: [project :, org.apache.beam:beam-runners-direct-java:2.30.0, org.apache.beam:beam-sdks-java-core:2.30.0, com.fasterxml.jackson.core:jackson-databind:2.12.1, com.fasterxml.jackson:jackson-bom:2.12.1]
    Circular dependency for: com.fasterxml.jackson.core:jackson-annotations:2.12.1
     The stack is: [project :, org.apache.beam:beam-runners-direct-java:2.30.0, com.fasterxml.jackson.core:jackson-databind:2.12.1, com.fasterxml.jackson.core:jackson-annotations:2.12.1, com.fasterxml.jackson:jackson-bom:2.12.1]
    Circular dependency for: com.fasterxml.jackson:jackson-bom:2.12.1
     The stack is: [project :, org.apache.beam:beam-runners-direct-java:2.30.0, com.fasterxml.jackson.core:jackson-databind:2.12.1, com.fasterxml.jackson.core:jackson-annotations:2.12.1, com.fasterxml.jackson:jackson-bom:2.12.1, com.fasterxml.jackson.core:jackson-core:2.12.1]
    Circular dependency for: com.fasterxml.jackson.core:jackson-databind:2.12.1
     The stack is: [project :, org.apache.beam:beam-runners-direct-java:2.30.0, com.fasterxml.jackson.core:jackson-databind:2.12.1, com.fasterxml.jackson.core:jackson-annotations:2.12.1, com.fasterxml.jackson:jackson-bom:2.12.1]
    Circular dependency for: com.fasterxml.jackson:jackson-bom:2.12.1
     The stack is: [project :, org.apache.beam:beam-runners-direct-java:2.30.0, com.fasterxml.jackson.core:jackson-databind:2.12.1, com.fasterxml.jackson.core:jackson-core:2.12.1, com.fasterxml.jackson:jackson-bom:2.12.1, com.fasterxml.jackson.core:jackson-annotations:2.12.1]
    Circular dependency for: com.fasterxml.jackson.core:jackson-core:2.12.1
     The stack is: [project :, org.apache.beam:beam-runners-direct-java:2.30.0, com.fasterxml.jackson.core:jackson-databind:2.12.1, com.fasterxml.jackson.core:jackson-core:2.12.1, com.fasterxml.jackson:jackson-bom:2.12.1]
    Circular dependency for: com.fasterxml.jackson.core:jackson-databind:2.12.1
     The stack is: [project :, org.apache.beam:beam-runners-direct-java:2.30.0, com.fasterxml.jackson.core:jackson-databind:2.12.1, com.fasterxml.jackson.core:jackson-core:2.12.1, com.fasterxml.jackson:jackson-bom:2.12.1]
    Circular dependency for: com.fasterxml.jackson:jackson-bom:2.12.1
     The stack is: [project :, org.apache.beam:beam-runners-direct-java:2.30.0, com.fasterxml.jackson.core:jackson-databind:2.12.1, com.fasterxml.jackson:jackson-bom:2.12.1, com.fasterxml.jackson.core:jackson-annotations:2.12.1]
    
    
  6. suztomo commented on Aug 11, 2021

    @suztomo
    ContributorAuthor

    This also shows missing references from the artifact.

    Class org.apache.beam.model.fnexecution.v1.BeamFnApi$WorkerStatusResponse is not found;
      referenced by 2 class files
        org.apache.beam.repackaged.direct_java.runners.fnexecution.status.WorkerStatusClient (org.apache.beam:beam-runners-direct-java:2.30.0)
        org.apache.beam.repackaged.direct_java.runners.fnexecution.status.BeamWorkerStatusGrpcService (org.apache.beam:beam-runners-direct-java:2.30.0)
      Cause:
        Unknown
    
  7. suztomo commented on Aug 12, 2021

    @suztomo
    ContributorAuthor

    Fixed in #2181

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions