Partial Incremental Tasks

(See the links to example scripts Markdown.tasks.ps1 at the bottom)

---

Partial incremental tasks are used in order to improve build performance in scenarios with one-to-one correspondence between input and output files. The build engine checks for output files existence and compares their timestamps with input timestamps. Finally, only input files with missing or out-of-date outputs are passed in task scripts.

If all outputs are up-to-date then task scripts are not invoked at all. Note: referenced dependent tasks, if any, are invoked anyway as usual.

Here is a partial incremental task template:

task Name -Partial @{<inputs> = <outputs>} [-Jobs] {process{
    ...
}}

It normally defines at least these three pieces: the inputs and outputs expressions and the script, often with the process block in it.

The Inputs

It is a list of input file items or paths or a script block which gets them. For example, for all markdown files in the build directory it can be:

-Partial @{ (Get-Item *.md, *.markdown) = ... }

A fixed list of known absolute or relative file paths will do as well:

-Partial @{ @('README.md', 'Release-Notes.md') = ... }

The inputs are finally resolved by the engine into the full paths piped to the outputs if it is a script block (all of them) and to the task script (some of them). All input files must exist, otherwise the task fails.

The Outputs

It is a list of output file paths or a script block which gets them. There must be one-to-one correspondence between input and output files taking their order into account.

If the argument is a script block then it is invoked with the inputs piped to it. Its goal is to transform each input path into its output pair. For example, on converting markdown files to HTML files it might be:

-Partial @{ ... = {process{ [System.IO.Path]::ChangeExtension($_, 'htm') }} }

Note use of the process block. It is called for every input path represented by the automatic variable $_ in it. The script processes the current $_ and outputs the corresponding output path.

A list of known file names will do as well (and the process is not needed):

-Partial @{ ... = @('README.htm', 'Release-Notes.htm') }

The Script

The script is invoked with filtered input paths piped to it. Only input paths with missing or out-of-date output are piped.

Though it is not mandatory, it is typical for partial incremental task scripts to have the process block. This block is called for each input path. The current input and output paths are represented by the automatic variables $_ and $$ respectively. The script processes the input file $_ and creates or updates the output file $$:

{process{
    # convert the file $_ into the file $$
    ...
}}

The script may also have the begin and/or end blocks called once before and after the process iterations. They can use the automatic variables $Inputs (array of resolved full input paths) and $Outputs (exactly as it was defined):

{
    begin {
        # init: load assemblies, import modules, dot-source scripts, etc.
        ...
    }
    process {
        # convert the file $_ into the file $$
        ...
    }
    end {
        # clean-up
        ...
    }
}

Note: in PowerShell if a script block is just code with no begin, process, and end blocks then it is treated as the end block. A partial incremental task can be defined in this way but the code has to deal with entire $Inputs and $Outputs instead of $_ and $$.

{
    # convert all $Inputs files into $Outputs files
    ...
}

Examples

The build script .build.ps1 of this project uses the task ConvertMarkdown which converts markdown files in the current location to HTML files (for example, in order to be included into the released package instead of the original markdown files).

The task script Markdown.tasks.ps1 is not included into the project. There are choices:

• Task script using MarkdownDeep
• Task script using MarkdownSharp
• Task script using Markdown.pl

MarkdownDeep and MarkdownSharp versions work fine with UTF8 and Unicode but one has to obtain the assembly (see links for details).

Markdown.pl (here) requires perl. Note: msysgit includes perl but an alias perl has to be defined if it is not in the path.

---