How to publish modules which are actually changed in gradle instead of all(not changed) to setup CI?

5.8k views Asked by At

I am setting up the CI model for one project.This project is almost having 500 modules approx.We just update the workspace everytime through our CI tool and build modules which is actually changed.We are using gradle to build all modules my requirement is to publish only those modules which are changed in the current build to nexus snapshot repository.I know there is gradle task to publish the artifacts but only to publish to changed modules is the requirement.

Below is the example.

A
B
C
D
E
F

If there is a change in B and F then I want to publish only B and F modules in nexus and if change is in A and F then publish only A and F module.

Something similar to

 class IncrementalReverseTask extends DefaultTask {
      @InputDirectory
      def File inputDir

      @OutputDirectory
      def File outputDir

      @TaskAction
      void execute(IncrementalTaskInputs inputs) {
          if (!inputs.incremental)
              project.delete(outputDir.listFiles())

          inputs.outOfDate { change ->
              def targetFile = project.file("$outputDir/${change.file.name}")
              targetFile.text = change.file.text.reverse()
          }

          inputs.removed { change ->
              def targetFile = project.file("$outputDir/${change.file.name}")
              if (targetFile.exists()) {
                  targetFile.delete()
              }
          }
      }
  }

I tried in below way and getting this issue

publishing {
    publications {
        maven(MavenPublication) {
            groupId 'com.example'
            artifactId 'core'
            version '1.0-SNAPSHOT'
            from components.java

    }
    }
    repositories {
    maven {
        credentials {
            username "abcde"
            password "***********"
            }
            url "https://nexus.test.com/content/repositories/snapshots"
            }
            }
    }

task incrementalPublishToMavenRepository(type: IncrementalPublishToMavenRepository) {
  inputDir = file('.')
  publication = project.tasks.getByPath(":${project.name}:publishMavenPublicationToMavenRepository").publication
}

class IncrementalPublishToMavenRepository extends org.gradle.api.publish.maven.tasks.PublishToMavenRepository {
    @InputDirectory
    def File inputDir

    @OutputDirectory
    File generatedFileDir = project.file("${project.buildDir}/libs")

    @TaskAction
    void perform(IncrementalTaskInputs inputs) {
        println 'hello this should be executed ones'
    }
}

and getting below error

gradle jar incrementalPublishToMavenRepository

Configuration on demand is an incubating feature.
:core:compileJava UP-TO-DATE
:core:processResources NO-SOURCE
:core:classes UP-TO-DATE
:core:jar UP-TO-DATE
:app:compileJava UP-TO-DATE
:app:processResources UP-TO-DATE
:app:classes UP-TO-DATE
:app:jar UP-TO-DATE
:app:generatePomFileForMavenPublication
:app:incrementalPublishToMavenRepository FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':app:incrementalPublishToMavenRepository'.
> The 'repository' property is required

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.
3

There are 3 answers

10
webdizz On

Each task in Gradle has inputs and outputs properties to enable incremental task capabilities, so this capability could be configured for publishing task to publish artifacts to repository.

Update:

Well, to implement full solution more effort is required, however here is an idea how you can use mentioned capabilities:

apply plugin: 'maven-publish'

publishing {
    publications {
        mavenJava(MavenPublication) {
            from components.java
        }
    }
}

class IncrementalPublishToMavenLocal extends org.gradle.api.publish.maven.tasks.PublishToMavenLocal {
    @Input
    int fileCount = 10

    @OutputDirectory
    File generatedFileDir = project.file("${project.buildDir}/libs")

    @TaskAction
    void perform(IncrementalTaskInputs inputs) {
        println 'hello this should be executed ones'
    }
}

task incrementalPublishToMavenLocal(type: IncrementalPublishToMavenLocal) {
  publication = project.tasks.getByPath(":${project.name}:publishMavenJavaPublicationToMavenLocal").publication
}

Now you can use ./gradlew incrementalPublishToMavenLocal to perform incremental publishing. Hope this will help to move forward.

0
unknown On

I tried to skip the uploadArchives method in below way when it is not creating the jar file

subprojects {
    apply plugin: 'java'
    apply plugin: 'maven'

    repositories {
        mavenCentral()
    }

    uploadArchives {
    repositories {
       mavenDeployer {
             repository(url: "https://repo.test.com/nexus/repos/snapshots") {
             authentication(userName: "abcdef", password: "******")
}
             pom.version = "1.1-SNAPSHOT"
             pom.groupId = "com.test"
       }
    }
}
uploadArchives {
  onlyIf { jar.didWork }
}

}

Below is the output when I run with clean

:app:clean
:core:clean
:core:compileJava
:core:processResources NO-SOURCE
:core:classes
:core:jar
:app:compileJava
:app:processResources
:app:classes
:app:jar
:app:copyLicense
:app:startScripts
:app:distTar
:app:distZip
:app:uploadArchives
:core:uploadArchives

Below is the output when run without clean

:core:compileJava UP-TO-DATE
:core:processResources NO-SOURCE
:core:classes UP-TO-DATE
:core:jar UP-TO-DATE
:app:compileJava UP-TO-DATE
:app:processResources UP-TO-DATE
:app:classes UP-TO-DATE
:app:jar UP-TO-DATE
:app:copyLicense UP-TO-DATE
:app:startScripts UP-TO-DATE
:app:distTar UP-TO-DATE
:app:distZip UP-TO-DATE
:app:uploadArchives SKIPPED
:core:uploadArchives SKIPPED
0
unknown On

I found another way to do it with maven-publish plugin

apply plugin: 'maven-publish'

publishing {
    publications {
        maven(MavenPublication) {
            groupId 'com.example'
            artifactId 'core'
            version '1.0-SNAPSHOT'
            from components.java
        }
    }
}

publishing {
    repositories {
        maven {
        credentials {
            username "abcedf"
            password "***********"
            }
            // change to point to your repo, e.g. http://my.org/repo
            url "https://repo.test.com/content/repositories/snapshots"
        }
    }
}


task incrementalPublishToMavenRepository(type: IncrementalPublishToMavenRepository) {
  inputDir = file('src')
  publication = project.tasks.getByPath(":${project.name}:publishMavenPublicationToMavenRepository").publication
  repository = project.tasks.getByPath(":${project.name}:publishMavenPublicationToMavenRepository").repository
}

class IncrementalPublishToMavenRepository extends org.gradle.api.publish.maven.tasks.PublishToMavenRepository {
    @InputDirectory
    def File inputDir

    @OutputDirectory
    File generatedFileDir = project.file("${project.buildDir}/libs")

    @TaskAction
    void perform(IncrementalTaskInputs inputs) {
        println 'hello this should be executed ones'
    }
}

Here is the output

gradle jar incrementalPublishToMavenRepository
Configuration on demand is an incubating feature.
:core:compileJava UP-TO-DATE
:core:processResources NO-SOURCE
:core:classes UP-TO-DATE
:core:jar UP-TO-DATE
:app:compileJava UP-TO-DATE
:app:processResources UP-TO-DATE
:app:classes UP-TO-DATE
:app:jar UP-TO-DATE
:app:generatePomFileForMavenPublication
:app:incrementalPublishToMavenRepository UP-TO-DATE
:core:generatePomFileForMavenPublication
:core:incrementalPublishToMavenRepository UP-TO-DATE

Here is the output if I make a change in app module the it just upload the app and say UP-TO-DATE for core module

gradle jar incrementalPublishToMavenRepository
Configuration on demand is an incubating feature.
:core:compileJava UP-TO-DATE
:core:processResources NO-SOURCE
:core:classes UP-TO-DATE
:core:jar UP-TO-DATE
:app:compileJava
:app:processResources UP-TO-DATE
:app:classes
:app:jar
:app:generatePomFileForMavenPublication
:app:incrementalPublishToMavenRepository
Upload https://repo.test.com/content/repositories/snapshots/com/example/app/1.0-SNAPSHOT/app-1.0-20170609.180436-15.jar
Upload https://repo.test.com/content/repositories/snapshots/com/example/app/1.0-SNAPSHOT/app-1.0-20170609.180436-15.jar.sha1
Upload https://repo.test.com/content/repositories/snapshots/com/example/app/1.0-SNAPSHOT/app-1.0-20170609.180436-15.jar.md5
Upload https://repo.test.com/content/repositories/snapshots/com/example/app/1.0-SNAPSHOT/app-1.0-20170609.180436-15.pom
Upload https://repo.test.com/content/repositories/snapshots/com/example/app/1.0-SNAPSHOT/app-1.0-20170609.180436-15.pom.sha1
Upload https://repo.test.com/content/repositories/snapshots/com/example/app/1.0-SNAPSHOT/app-1.0-20170609.180436-15.pom.md5
Upload https://repo.test.com/content/repositories/snapshots/com/example/app/1.0-SNAPSHOT/maven-metadata.xml
Upload https://repo.test.com/content/repositories/snapshots/com/example/app/1.0-SNAPSHOT/maven-metadata.xml.sha1
Upload https://repo.test.com/content/repositories/snapshots/com/example/app/1.0-SNAPSHOT/maven-metadata.xml.md5
Upload https://repo.test.com/content/repositories/snapshots/com/example/app/maven-metadata.xml
Upload https://repo.test.com/content/repositories/snapshots/com/example/app/maven-metadata.xml.sha1
Upload https://repo.test.com/content/repositories/snapshots/com/example/app/maven-metadata.xml.md5
hello this should be executed ones
:core:generatePomFileForMavenPublication
:core:incrementalPublishToMavenRepository UP-TO-DATE