Skip to content

Commit

Permalink
Merge pull request #350 from djarecka/doc/user_guide_upd
Browse files Browse the repository at this point in the history
editing input_spec, adding a short output_spec section
  • Loading branch information
djarecka authored Sep 22, 2020
2 parents 8fba851 + 0b0b68c commit 832118a
Show file tree
Hide file tree
Showing 3 changed files with 107 additions and 5 deletions.
30 changes: 25 additions & 5 deletions docs/input_spec.rst
Original file line number Diff line number Diff line change
Expand Up @@ -35,14 +35,14 @@ Let's start from the previous example:
In order to create an input specification, a new `SpecInfo` object has to be created.
The field `name` specifiest the typo of the spec and it should be always "Input" for
The field `name` specifies the type of the spec and it should be always "Input" for
the input specification.
The field `bases` specifies the "base specification" you want to use (can think about it as a
`parent class`) and it will usually contains `ShellSpec` only, unless you want to build on top of
your other specification (this will not be cover in this section).
The part that should be always customised is the `fields` part.
Each element of the `fields` is a separate input field that is added to the specification.
In this example, a three-elements tuples - with name, type and dictionary with additional
In this example, three-elements tuples - with name, type and dictionary with additional
information - are used.
But this is only one of the supported syntax, more options will be described below.

Expand Down Expand Up @@ -86,9 +86,25 @@ However, we allow for shorter syntax, that does not include `attr.ib`:
Each of the shorter versions will be converted to the `(name, attr.ib(...)`.
Types
-----
Type can be provided as a simple python type (e.g. `str`, `int`, `float`, etc.)
or can be more complex by using `typing.List`, `typing.Dict` and `typing.Union`.
There are also special types provided by Pydra:
- `File` and `Directory` - should be used in `input_spec` if the field is an existing file
or directory.
Pydra checks if the file or directory exists, and returns an error if it doesn't exist.
- `MultiInputObj` - a special type that takes a any value and if the value is not a list it
converts value to a 1-element list (it could be used together with `MultiOutputObj`
in the `output_spec` to reverse the conversion of the output values).
Metadata
--------
Expand Down Expand Up @@ -126,9 +142,6 @@ In the example we used multiple keys in the metadata dictionary including `help_
`xor` (`list`):
List of field names that are mutually exclusive with the field.
`keep_extension` (`bool`, default: `True`):
A flag that specifies if the file extension should be removed from the field value.
`copyfile` (`bool`, default: `False`):
If `True`, a hard link is created for the input file in the output directory.
If hard link not possible, the file is copied to the output directory.
Expand All @@ -139,9 +152,16 @@ In the example we used multiple keys in the metadata dictionary including `help_
`output_file_template` (`str`):
If provided, the field is treated also as an output field and it is added to the output spec.
The template can use other fields, e.g. `{file1}`.
Used in order to create an output specification.
`output_field_name` (`str`, used together with `output_file_template`)
If provided the field is added to the output spec with changed name.
Used in order to create an output specification.
`keep_extension` (`bool`, default: `True`):
A flag that specifies if the file extension should be removed from the field value.
Used in order to create an output specification.
`readonly` (`bool`, default: `False`):
If `True` the input field can't be provided by the user but it aggregates other input fields
Expand Down
81 changes: 81 additions & 0 deletions docs/output_spec.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
.. _Output Specification section:

Output Specification
====================

As it was mentioned in :ref:`shell_command_task`, the user can customize the input and output
for the `ShellCommandTask`.
In this section, the output specification will be covered.


Instead of using field with `output_file_template` in the customized `input_spec` to specify an output field,
a customized `output_spec` can be used, e.g.:


.. code-block:: python
output_spec = SpecInfo(
name="Output",
fields=[
(
"out1",
attr.ib(
type=File,
metadata={
"output_file_template": "{inp1}",
"help_string": "output file",
"requires": ["inp1", "inp2"]
},
),
)
],
bases=(ShellOutSpec,),
)
ShellCommandTask(executable=executable,
output_spec=output_spec)
Similarly as for `input_spec`, in order to create an output specification,
a new `SpecInfo` object has to be created.
The field `name` specifies the type of the spec and it should be always "Output" for
the output specification.
The field `bases` specifies the "base specification" you want to use (can think about it as a
`parent class`) and it will usually contains `ShellOutSpec` only, unless you want to build on top of
your other specification (this will not be cover in this section).
The part that should be always customised is the `fields` part.
Each element of the `fields` is a separate output field that is added to the specification.
In this example, a three-elements tuple - with name, type and dictionary with additional
information - is used.
See :ref:`Input Specification section` for other recognized syntax for specification's fields
and possible types.



Metadata
--------

The metadata dictionary for `output_spec` can include:

`help_string` (`str`, mandatory):
A short description of the input field. The same as in `input_spec`.

`output_file_template` (`str`):
If provided, the field is treated also as an output field and it is added to the output spec.
The template can use other fields, e.g. `{file1}`. The same as in `input_spec`.

`output_field_name` (`str`, used together with `output_file_template`)
If provided the field is added to the output spec with changed name.
The same as in `input_spec`.

`keep_extension` (`bool`, default: `True`):
A flag that specifies if the file extension should be removed from the field value.
The same as in `input_spec`.


`requires` (`list`):
List of field names that are required to create a specific output.
The fields do not have to be a part of the `output_file_template` and
if any field from the list is not provided in the input, a `NOTHING` is returned for the specific output.
This has a different meaning than the `requires` form the `input_spec`.
1 change: 1 addition & 0 deletions docs/user_guide.rst
Original file line number Diff line number Diff line change
Expand Up @@ -9,3 +9,4 @@ User Guide
state
combiner
input_spec
output_spec

0 comments on commit 832118a

Please sign in to comment.