Update docs

README.md

@@ -1,3 +1,5 @@
+<img alt="FastCS Logo" align="right" width="100" height="100" src="https://raw.githubusercontent.com/epics-containers/pvi/main/docs/images/pvi-logo.svg" target=https://github.com/epics-containers/pvi>
+
 [![CI](https://github.com/epics-containers/pvi/actions/workflows/ci.yml/badge.svg)](https://github.com/epics-containers/pvi/actions/workflows/ci.yml)
 [![Coverage](https://codecov.io/gh/epics-containers/pvi/branch/main/graph/badge.svg)](https://codecov.io/gh/epics-containers/pvi)
 [![PyPI](https://img.shields.io/pypi/v/pvi.svg)](https://pypi.org/project/pvi)
@@ -6,27 +8,23 @@
 # PVI
 
 PVI (PV Interface) is a framework for specifying the interface to an EPICS
-driver in a single YAML file. The initial target is asyn port driver based
-drivers, but it could be extended to streamDevice and other driver types at a
-later date.
-
-It allows the asyn parameter interface to be specified in a single place,
-and removes boilerplate code in the driver CPP, template files, documentation,
-and low level opis.
+driver. PVI can be used either as a library or an application. PVI Devices can be
+defined either in code or a YAML file. It can be used to generate UIs (adl, edl, bob) or
+a template appending info tags to existing records to define an NTTable of the PVs in an
+IOC.
 
 Source          | <https://github.com/epics-containers/pvi>
 :---:           | :---:
 PyPI            | `pip install pvi`
 Documentation   | <https://epics-containers.github.io/pvi>
 Releases        | <https://github.com/epics-containers/pvi/releases>
 
----
-
-Note: This module is currently a proposal only, so all details are subject to
-change at any point. The documentation is written in the present tense, but only
-prototype code is written.
+## Projects Using PVI
 
----
+- [ibek](https://github.com/epics-containers/ibek) - IOC Builder for EPICS and
+  Kubernetes
+- [FastCS](https://github.com/DiamondLightSource/FastCS) - Control system agnostic
+  framework for building device support in Python for both EPICS and Tango
 
 <!-- README only content. Anything below this line won't be included in index.md -->
 

docs/explanations.md

@@ -6,5 +6,7 @@ Explanations of how it works and why it works that way.
 :maxdepth: 1
 :glob:
 
+explanations/pvi-pv
+explanations/original-design
 explanations/*
 ```

docs/explanations/original-design.md

@@ -176,7 +176,7 @@ I suggest creating adl and edl files initially, following the example of
 makeAdl.py in ADGenICam, then expanding to support opi, bob and ui files
 natively. This would avoid needing screen converters installed
 
-# Drivers
+## Drivers
 
 The generated header file contains the string parameters, and defines the parameters to
 make the interface. In this example we have a header file pilatusDetectorParamSet.h:

docs/explanations/pvi-pv.md

@@ -0,0 +1,93 @@
+# PVI IOC Introspection
+
+PVI can be used to add info tags to existing EPICS database to create a V4 PV with an
+NTTable of the PVs within the IOC. This can be done with the `format_template` function
+by passing a `Device` instance, a PV prefix and an output path to write the template to.
+Or, using the CLI command `pvi generate-template` with a `pvi.device.yaml`.  The PV
+prefix is the prefix of the PVI PV itself. Usually this would match the prefix of the
+actual PVs, as defined in the `Device` signals, but it doesn't have to.
+
+## Template Format
+
+This will generate a new template containing records for all of the signals in the
+`Device` that inserts an info tag to the existing record.
+
+```
+record("*", "$(P)$(R)Gain") {
+    info(Q:group, {
+        "$(P)$(R)PVI": {
+            "pvi.Gain.w": {
+                "+channel": "NAME",
+                "+type": "plain",
+            }
+        }
+    })
+}
+
+record("*", "$(P)$(R)Gain_RBV") {
+    info(Q:group, {
+        "$(P)$(R)PVI": {
+            "pvi.Gain.r": {
+                "+channel": "NAME",
+                "+type": "plain",
+            }
+        }
+    })
+}
+...
+record("*", "$(P)$(R)UniqueId_RBV") {
+    info(Q:group, {
+        "$(P)$(R)PVI": {
+            "pvi.UniqueId.r": {
+                "+channel": "NAME",
+                "+type": "plain",
+            }
+        }
+    })
+}
+...
+record("*", "$(P)$(R)WaitForPlugins") {
+    info(Q:group, {
+        "$(P)$(R)PVI": {
+            "pvi.WaitForPlugins.w": {
+                "+channel": "NAME",
+                "+type": "plain",
+            }
+        }
+    })
+}
+```
+
+These info tags are then collected and served as a V4 PV by QSRV. This info tag adds an
+entry into the V4 PV `$(P)$(R)PVI` with the name `pvi.GainX.w` where `w` is the access
+mode (`r`, `w`, `rw`, `x`). Each `Device` in the IOC will produce its own PVI PV,
+differentiated by the `R` macro in this case.
+
+For more information on the syntax of the info tags, see the [QSRV documentation][QSRV].
+
+## PVI NTTable
+
+With an IOC running the PVI PV can be accessed using PVAccess, for example the `pvget`
+CLI tool in [PVAccessCPP], or the [p4p] python library.
+
+```shell
+❯ pvget SIM:DET:PVI
+SIM:DET:PVI structure
+    structure record
+        structure _options
+            boolean atomic true
+    structure pvi
+        structure Gain
+            string r SIM:DET:Gain_RBV
+            string w SIM:DET:Gain
+...
+        structure UniqueId
+            string r SIM:DET:UniqueId_RBV
+...
+        structure WaitForPlugins
+            string w SIM:DET:WaitForPlugins
+```
+
+[QSRV]: https://epics-base.github.io/pva2pva/qsrv_page.html
+[PVAccessCPP]: https://github.com/epics-base/pvAccessCPP
+[p4p]: https://github.com/mdavidsaver/p4p

docs/how-to.md

@@ -6,5 +6,6 @@ Practical step-by-step guides for the more experienced user.
 :maxdepth: 1
 :glob:
 
+how-to/write-a-formatter
 how-to/*
 ```

docs/how-to/write-a-formatter.md

@@ -5,53 +5,59 @@ your own use cases.
 
 ## Overview
 
-The formatters role is to take a device.yaml file and turn this into a screen file that
-can be used by the display software. Inside of the device.yaml file is a list of
-components that specify its name, a widget type and any additional properties that can be
-assigned to that widget (such as a pv name). During formatting, the device.yaml file is
-deserialised into component objects, which are later translated into widgets:
+The formatters role is to take a `Device` - defined either in code or in a
+`pvi.device.yaml` file - and turn this into a screen file that can be used by the
+display software. The `Device` has a list of components that specify its name, a widget
+type and any additional properties that can be assigned to that widget (such as a pv
+name). During formatting, `Component` objects of the `Device` are translated into
+widgets to be written to a UI file.
 
 ```{literalinclude} ../../src/pvi/device.py
 :pyobject: Component
 ```
 
+There are various types of `Component`. The simplest is a read-only signal.
+
 ```{literalinclude} ../../src/pvi/device.py
 :pyobject: SignalR
 ```
 
-To make a screen from this, we need a template file. This contains a blank representation
-of each supported widget for each of the supported file formats (bob, edl etc...). Below
-is an example of a 'text entry' widget for a .bob file:
+To add structure, there is a `Group` component, which itself has a list of `Components`.
+
+```{literalinclude} ../../src/pvi/device.py
+:pyobject: Group
+```
+
+To make a screen from this, we need a template UI file. This contains a blank
+representation of each supported widget for each of the supported file formats (bob, edl
+etc...). Below is an example of a `textentry` widget for a .bob file.
 
 ```{literalinclude} ../../src/pvi/_format/dls.bob
-:lines: 57-73
+:language: xml
+:lines: 46-54
 ```
 
 By extracting and altering the template widgets with the information provided by the
 components, we can create a screen file.
 
-## Create a formatter subclass
+## Create a Formatter subclass
 
 To start, we will need to create our own formatter class. These inherit from an abstract
-'Formatter' class that is defined in base.py. Inside, we need to define one mandatory
-'format' function, which will be used to create our screen file:
+`Formatter` class that is defined in base.py. Inside, we need to define one mandatory
+`format` function, which will be used to create our screen file:
 
 ```{literalinclude} ../../src/pvi/_format/base.py
-:pyobject: Formatter
+:pyobject: Formatter.format
 ```
 
-The format function takes in a device: a list of components obtained from our
-deserialised device.yaml file, A prefix: the pv prefix of the device, and a path: the
-output destination for the generated screen file.
-
 With a formatter defined, we now can start to populate this by defining the screen
 dependencies.
 
-## Define the Screen Layout Properties
+## Define the ScreenLayout properties
 
 Each screen requires a number of layout properties that allow you to customise the size
-and placement of widgets. These are stored within a 'ScrenLayout' dataclass that can
-be imported from utils.py. Within the dataclass are the following configurable parameters:
+and placement of widgets. These are stored within `ScreenLayout` dataclass with the
+following configurable parameters:
 
 ```{literalinclude} ../../src/pvi/_format/screen.py
 :pyobject: ScreenLayout
@@ -64,113 +70,95 @@ screen format function) will be available to adjust inside of the formatter.yaml
 Anything else, should be considered as defaults for the formatter:
 
 ```{literalinclude} ../../src/pvi/_format/dls.py
-:end-before: SW DOCS REF
+:language: python
 :start-after: LP DOCS REF
+:end-before: SW DOCS REF
 ```
 
 In the example above, everything has been made adjustable from the formatter.yaml except
-the properties relating to groups. This is becuase they are more dependant on the file
+the properties relating to groups. This is because they are more dependant on the file
 format used rather than the users personal preference.
 
 For clarity, the example below shows how the formatter.yaml can be used to set the
 layout properties. Note that these are optional as each property is defined with a
-default value:
+default value.
 
 ```{literalinclude} ../../formatters/dls.bob.pvi.formatter.yaml
 ```
 
-## Assign a Template File
+## Assign a template file
 
 As previously stated, a template file provides the formatter with a base model of all
 of the supported widgets that it can then overwrite with component data. Currently,
 pvi supports templates for edl, adl and bob files, which can be referenced from the
 \_format directory with the filename 'dls' + the file formats suffix (eg. dls.bob).
 
 Inside of the format function, we need to provide a reference to the template file that
-can then be used to identify what each widget should look like:
+can then be used to identify what each widget should look like.
 
-```python3
+```python3 notest
 template = BobTemplate(str(Path(__file__).parent / "dls.bob"))
 ```
 
-% Documentation does not explain what the WidgetTemplate function does,
-% nor its subclasses BobTemplate, EdlTemplate & AdlTemplate.
+## Divide the template into widgets
 
-## Divide the Template into Widgets
+With a template defined, we now need to assign each part of it to a supported widget
+formatter. This is achieved by instantiating a WidgetFormatterFactory composed of
+WidgetFormatters created from the UI template. WidgetFormatters are created by searching
+the UI template for the given search term and a set of properties in the template to
+replace with widget fields.
 
-With a template defined, we now need to assign each part of it to a supported widget.
-This is achieved using the ScreenWidgets dataclass (from utils.py). With this, we can
-assign each of the widget classes to a snippet of the template using the
-WidgetFactory.from_template method:
+:::{note}
+The `WidgetFormatter`s are generic types that must be parameterised depending on the
+specific UI. Commonly this would use `str` for formatting text to a file directly. In
+this case we use `_Element`, which will serialised to text with the `lxml` library.
+:::
 
 ```{literalinclude} ../../src/pvi/_format/dls.py
-:end-before: MAKE_WIDGETS DOCS REF
+:language: python
 :start-after: SW DOCS REF
+:end-before: MAKE_WIDGETS DOCS REF
 ```
 
+```{warning}
 This function uses a unique search term to locate and extract a widget from the template.
-As such, the search term MUST be unique to avoid extracing multiple or irrelevant
+As such, the search term MUST be unique to avoid extracting multiple or irrelevant
 widgets from the template.
+```
 
 ## Define screen and group widget functions
 
-Two widgets that are not handled by ScreenWidgets are the screen title and group object.
-This is because the style of these widgets differ greatly for each file type. For
-instance, with edl and adl files, groups are represented by a rectangle and title placed
-behind a collection of widgets. Conversely, bob files handle groups using its dedicated
-group object, which places widgets as children under the group object. Becuase of this,
-we need to define two functions: one for the additional screen widgets (such as the title),
-and one to represent the group widgets.
-
-We then need to define two functions that can be used to create multiple instances of
-these widgets. In this example, we provide two arguments: The 'bounds', to set the
-widgets size and position, and the 'title' to populate the label with.
+Additionally, formatters for the title and a group on a screen must be defined along
+with functions to create multiple components, for example a rectangle with a label on
+top. In this example, we provide two arguments: The `bounds`, to set the widgets size
+and position, and the `title` to populate the label with.
 
 ```{literalinclude} ../../src/pvi/_format/dls.py
-:end-before: SCREEN_INI DOCS REF
+:language: python
 :start-after: MAKE_WIDGETS DOCS REF
+:end-before: SCREEN_INI DOCS REF
 ```
 
-## Construct a Screen Object
+## Construct a ScreenFormatter
 
-Provided that you have defined the LayoutProperties, template, ScreenWidgets and the
-screen title and group object functions, we are now ready to define a screen object.
+These formatters can be used to define a `ScreenFormatterFactory`
 
 ```{literalinclude} ../../src/pvi/_format/dls.py
-:end-before: SCREEN_FORMAT DOCS REF
+:language: python
 :start-after: SCREEN_INI DOCS REF
+:end-before: SCREEN_FORMAT DOCS REF
 ```
 
-Note that screen_cls and group_cls are defined separately here as GroupFactories. This is
-because they take in the make_widgets function, which has the possibility of returning
-multiple widgets. (In edl files for example, we return a rectangle and label widget to
-represent a group.)
-
-The screen object itself contains two key functions: The 'screen' function takes a
-deserialised device.yaml file and converts each of its components into widgets. It then
-calculates the size and position of these widgets to generate a uniform screen layout.
-On the output of this, we can call a (screen.)format function that populates these widgets
-with the extracted properties from the device.yaml, and converts them into the chosen file
-format:
+which can be used to instantiate a `ScreenFormatter` by passing a set of `Components`
+and a title. This can then create `WidgetFormatters` for each `Component` for the
+specific UI type the factory was parameterised with.
 
 ```{literalinclude} ../../src/pvi/_format/dls.py
-:end-before: SCREEN_WRITE DOCS REF
+:language: python
 :start-after: SCREEN_FORMAT DOCS REF
+:end-before: SCREEN_WRITE DOCS REF
 ```
 
-## Generate the Screen file
-
-After calling format on the screen object, you will be left with a list of strings that
-represent each widget in your chosen file format. The final step is to create a
-screen file by unpacking the list and writing each widget to the file:
-
-```{literalinclude} ../../src/pvi/_format/dls.py
-:start-after: SCREEN_WRITE DOCS REF
-```
-
-And thats it. With this you can now create your own custom formatters. Below you can
-find a complete example formatter, supporting both edl and bob file formats for DLS:
-
-```{literalinclude} ../../src/pvi/_format/dls.py
-:pyobject: DLSFormatter
-```
+In this case the `write_bob` function calls into the `lxml` library to format the
+`_Element` instances to text. For `str` formatters this would call
+`pathlib.Path.write_file`.