How modules are loaded

By default, ml-gradle loads modules from src/main/ml-modules. This directory aligns with the Maven convention of placing all application code under "src/main", and "ml-modules" was chosen as a MarkLogic-specific path.

To accommodate storing both REST API specific modules (such as options, services, and transforms) and "regular" application modules under the same parent directory, the following directory structure is used for identifying where different kinds of modules should be stored:

1. /root - You store any modules here that aren't REST API options, services, transforms, or namespaces. The module will be loaded with a URI relative to "/root" (it won't have "/root" in the URI).
2. /ext - You can also store any modules here too, just like "/root". What's the difference? "/ext" was actually supported first in an effort to mirror what the REST API suggests with its /v1/ext endpoint. But in practice, there's little value in storing modules under "/ext", so it's easier just to toss everything under "/root".
3. /options - REST API search options are stored here. The name of the file (minus the extension) is used to name the search options loaded into ML.
4. /services - REST API resource extensions are stored here. The name of the file (minus the extension) is used to name the resource extension.
5. /transforms - REST API transforms are stored here. The name of the file (minus the extension) is used to name the transform.
6. /namespaces - A REST API namespace namespace can be registered via a file containing the namespace URI, and the name of the file (minus the extension, which can be anything) is used for the namespace prefix.
7. In addition, any "unrecognized" directory under ml-modules - i.e. a directory that isn't one of the above - will have its contents loaded into the modules database. Prior to ml-gradle 3.0.0, the URI of each such document would include the name of the unrecognized directory. But with version 3.0.0 and greater, the URI does not have the unrecognized directory name. Best practice is to stick with "ext" and "root" for modules and not use this feature!

Modules are loaded when you run "mlDeploy". They can also be loaded by themselves:

gradle mlLoadModules

It's often helpful to use info-level logging so you can see which modules are loaded:

gradle -i mlLoadModules

And debug-level logging will show all the directories that ml-gradle looks in for modules:

gradle -d mlLoadModules

You can also reload modules - i.e. clear the modules database first, and then load modules:

gradle mlReloadModules

If you run into any problems, please see Debugging module loading.

Ports used for loading modules

ml-gradle has to handle REST modules - options, transforms, services, and namespaces - differently from non-REST modules, as REST modules must be loaded via an app-specific REST server. However, not every MarkLogic app has a REST server. For this reason, ml-gradle takes the following approach by default:

1. Non-REST modules are loaded via the port defined by mlAppServicesPort, which defaults to 8000. This port is nearly guaranteed to exist in every ML cluster, making it a safe choice.
2. REST modules are loaded via the port defined by mlRestPort, which does not have a default value.

See the Property reference for all of the properties that are used for connecting to these ports.

This is an important distinction for when you run into an error while loading modules, as your non-REST modules may be loading fine, but your REST modules are not.

Replacing tokens in modules

As described in the Configuring resources page, a "custom tokens" map can be populated to specify tokens in resource payloads that will be replaced. These tokens will also be replaced in module files - but as of version 3.2.0, this is only done in "asset" modules, not yet REST API modules (services, options, and transforms).

Two properties control this behavior that are worth noting:

1. mlReplaceTokensInModules - controls whether tokens are replaced or not, defaults to true
2. mlUseRoxyTokenPrefix - (see the note below about how this property changes in 3.2.0) expects tokens to start with "@ml.", mirroring Roxy's behavior. This defaults to true; if you don't need these prefixes in place, be sure to set this property to false.

NEW in version 3.2.0 - as described on the Configuring resources page, all Gradle properties will be added to the tokens map by default. And, mlUseRoxyTokenPrefix now defaults to false, effectively deprecating that feature. You can emulate that behavior by setting the following properties to mirror Roxy's behavior:

mlTokenPrefix=@ml.
mlTokenSuffix=

Thus, in 3.2.0, with mlPropsAsTokens and mlReplaceTokensInModules both defaulting to true, the default behavior is that all Gradle properties are used as tokens (with "%%" as the default prefix and suffix), and these tokens will be replaced in modules if they exist.

Only loading modules that need loading

See Watching for module changes for more information on this topic.

The /v1/ext endpoint in the ML Client REST API only allows for loading one module at a time, and thus loading hundreds of modules or more can take minutes. This isn't too painful for initializing an empty modules database, but it's unacceptable during a code-test cycle.

To address this, ml-gradle keeps track of when it last loaded each module. This is done via a properties file in the "build" directory of the project (so that the properties file is not in version control). The path of the file is build/ml-last-installed-timestamp.properties. When ml-gradle tries to load a module, it first checks to see if the module's last-modified timestamp is greater than what is recorded in the properties file. If so, the module is loaded, and the properties file is updated.

As of 3.3.1, this feature can be disabled by setting the mlModuleTimestampsPath property to an empty string. Prior to 3.3.1, it can be disabled in the following manner in build.gradle:

ext {
  mlAppConfig.setModuleTimestampsPath(null)
}