Grouper Build/Dependency Management

Grouper currently has no facility to manage third party library dependencies. Various JARs exist in the codebase itself that are pulled together via build scripts to execute compilation, packaging and deployment tasks. This approach has a number of drawbacks:

• Difficult to upgrade a JAR, should a new version be required
• Difficult to add new capabilities to Grouper which may depend on new JARs
• Grouper codebase size is dramatically increased

We would want to devise a strategy by which such third party libraries can be pulled from public artifact repositories and integrated with the build. The following dependency management solutions may be considered.

Apache Ivy

The Grouper's build is managed via Apache Ant. Apache Ivy simply adds dependency management to Ant without replacing the existing build. Dependencies can be pulled from public repositories, conflicts managed and profiles created, etc. The advantage of using this option would be strong and tight integration with existing Ant scripts with relatively minimal effort. 

Notes to consider, if Ivy is chosen:

1. Existing ant build scripts can be kept. Ivy has excellent integration with Ant. 
2. Consider that Ant is somewhat of a deprecated tool and is largely being replaced by the likes of Maven, and specially Gradle.
3. All configuration is via XML. May or may not be desirable. 
4. Ant installation, and configuration is required of a grouper developer.
5. IDE integration is decent

Apache Maven

Apache Maven is a build AND dependency management tool. While Ant is a bit more task-based and can provide script-like tasks and targets to execute builds without requiring any particular project structure or layout, Maven introduces a new standard layout for projects' resources and additionally allows support for managing and pulling in dependencies. Maven is also, unlike Ant, lifecycle based and so various phases in the overall build cycle are introduced by default that take care of compilation, packaging and deployment of artifacts without requiring much custom work. 

Notes to consider, if Maven is chosen:

1. Existing ant build scripts need to be rewritten for Maven. Both are XML, but the syntax and format is totally different and it's not possible to use Maven's dependency management features without using its build features.
2. Of course, it's all going to be XML. Configuration of builds, dependencies, profiles, plugins and such can be very verbose.
3. Maven is not particularly good at running custom build scripts. All tasks need to fit into its own phases and the lifecycle cannot easily be intercepted or modified without writing maven plugins.
4. A Maven installation and configuration is required of a grouper developer.
5. Plenty of plugins exist for Maven that provide integration with all sorts of systems and capabilities.
6. Grouper can take advantage of Maven's release plugin, by publishing artifacts to Maven central, etc when needed.
7. JARs that are not found in any public repositories can be added to a maven build. While Maven has a concept of System dependencies, it would however be best if such JARs are put into a repository that is perhaps owned by Internet2. 
8. Maven has a concept for Profiles, which allow specific settings to be activated per environment (dev, QA, etc)
9. IDE integration is decent. Eclipse may be problematic though. 
10. Custom artifact repositories can be declared with no problem at all. 
11. Maven's dependency management can result into a "Dependency Hell" type of problem for very large and complicated projects. Managing transitive dependencies and resolving conflicts may not be immediately trivial. 

Gradle/Grape

Gradle is also an entirely new build tool with a dependency management feature called Grape. Gradle is the best of Ant and Maven combined. It has support for Maven project layouts and Maven repositories for managing dependencies. Build scripts are entirely based on Groovy and so can be heavily customized to provide Ant-like features. Gradle also introduces a standard set of phases and goals, and additional tasks can be added to the build script. The dependency management of Gradle, Grape, also has the advantage of declaring dependencies directly in the code via annotations, can work with Maven and Gradle repositories:

Notes to consider, if Gradle is chosen:

1. Existing ant build scripts need to be rewritten for Gradle. It's not possible to use Gradle's dependency management features without using its build features.
2. Of course, the build script is written in Groovy. May or may not be desirable.
3. Gradle by far is the most modern and sought after build tool, today. 
4. A Gradle installation is NOT required of a grouper developer. Gradle can download and configure itself.
5. Since the build script is written in Groovy, all manners of custom tasks, profiles, configurations and such can be added.
6. Gradle's concurrent and parallel builds with caching and daemons can be a HUGE benefit when running builds for large projects.
7. Gradle supports all Maven-like artifact repositories.
8. Plenty of plugins exist for Gradle that allow integration and customization of various build features.

Considerations for Grouper

1. Build/Dependency tools must be configured to retrieve the source artifact for JARs.
2. Build/Dependency tools must be easily able to create and update javadocs.
3. Build/Dependency tools must have the ability to support environment profiles.
4. Build/Dependency tools must have support for local dependencies.
5. Consider what may be the most natural and easiest option for a Grouper contributor to get started.
6. There needs to be a way for Grouper to determine the version of its JARs used:
   1. Can be use/reference the Maven build maintained in parallel with ant.
   2. Can use the JAR manifest itself.
7. IDE integration is important. While IntelliJ/NetBeans have no issues with either of these tools, Eclipse may be problematic for Maven or Gradle.