Add some more docs

.github/CONTRIBUTING.md

@@ -1,4 +1,4 @@
-# Contribute to the project
+# How to contribute to the project
 
 Contributions and issues are most welcome! All issues and pull requests are
 handled through [GitHub](https://github.com/bluesky/ophyd-async/issues). Also, please check for any existing issues before

README.md

@@ -13,16 +13,19 @@ Asynchronous Bluesky hardware abstraction code, compatible with control systems
 | Documentation |      <https://bluesky.github.io/ophyd-async>      |
 |   Releases    | <https://github.com/bluesky/ophyd-async/releases> |
 
-Ophyd-async is a Python library for asynchronously interfacing with hardware, intended to
-be used as an abstraction layer that enables experiment orchestration and data acquisition code to operate above the specifics of particular devices and control
-systems.
+Ophyd-async is a Python library for asynchronously interfacing with hardware, intended to be used as an abstraction layer that enables experiment orchestration and data acquisition code to operate above the specifics of particular devices and control systems.
 
-Both ophyd and ophyd-async are typically used with the [Bluesky Run Engine][] for experiment orchestration and data acquisition.
+Both ophyd sync and ophyd-async are typically used with the [Bluesky Run Engine][] for experiment orchestration and data acquisition.
 
-While [EPICS][] is the most common control system layer that ophyd-async can interface with, support for other control systems like [Tango][] will be supported in the future. The focus of ophyd-async is:
+The main differences from ophyd sync are:
 
-* Asynchronous signal access, opening the possibility for hardware-triggered scanning (also known as fly-scanning)
-* Simpler instantiation of devices (groupings of signals) with less reliance upon complex class hierarchies
+- Asynchronous Signal access, simplifying the parallel control of multiple Signals
+- Support for [EPICS][] PVA and [Tango][] as well as the traditional EPICS CA
+- Better library support for splitting the logic from the hardware interface to avoid complex class heirarchies
+
+It was written with the aim of implementing flyscanning in a generic and extensible way with highly customizable devices like PandABox and the Delta Tau PMAC products. Using async code makes it possible to do the "put 3 PVs in parallel, then get from another PV" logic that is common in flyscanning without the performance and complexity overhead of multiple threads.
+
+Devices from both ophyd sync and ophyd-async can be used in the same RunEngine and even in the same scan. This allows a per-device migration where devices are reimplemented in ophyd-async one by one.
 
 [Bluesky Run Engine]: http://blueskyproject.io/bluesky
 [EPICS]: http://www.aps.anl.gov/epics/

docs/explanations/design-goals.md

@@ -1,57 +1,67 @@
-Design Goals
-============
+# Design goals and differences with ophyd sync
 
+Ophyd-async was designed to be a library for asynchronously interfacing with hardware. As such it fulfils the same role in the bluesky ecosystem as [ophyd sync](https://github.com/bluesky/ophyd): an abstraction layer that enables experiment orchestration and data acquisition code to operate above the specifics of particular devices and control systems. This document details the design goals and the differences with ophyd sync.
 
-Parity with Ophyd
------------------
+## Asynchronous Signal access
 
-It should be possible to migrate applications that use ophyd_ to ophyd-async. Meaning it must support:
+A fundamental part of ophyd-async is [](#asyncio). This allows lightweight and deterministic control of multiple signals, making it possible to do the "put 2 PVs in parallel, then get from another PV" logic that is common in flyscanning without the performance and complexity overhead of multiple threads.
 
-- Definition of devices
-- Conformity to the bluesky protocols
-- Epics (ChannelAccess) as a backend
+For instance, the threaded version of the above looks something like this:
+```python
+def set_signal_thread(signal, value):
+    t = Thread(signal.set, value)
+    t.start()
+    return
 
-Ophyd-async should provide built-in support logic for controlling `the same set of devices as ophyd <https://blueskyproject.io/ophyd/user/reference/builtin-devices.html>`_. 
+def run():
+    t1 = set_signal_thread(signal1, value1)
+    t2 = set_signal_thread(signal2, value2)
+    t1.join()
+    t2.join()
+    value = signal3.get_value()
+```
+This gives the overhead of co-ordinating multiple OS threads, which requires events and locking for any more complicated example.
 
+Compare to the asyncio version:
+```python
+async def run():
+    await asyncio.gather(
+        signal1.set(value1),
+        signal2.set(value2)
+    )
+    value = await signal3.get_value()
+```
+This runs in a single OS thread, but has predictable interrupt behavior, allowing for a much more readable linear flow.
 
-Clean Device Definition
------------------------
+```{seealso}
+[](../how-to/interact-with-signals.md) for examples of the helpers that are easier to write with asyncio.
+```
 
-It should be easy to define devices with signals that talk to multiple backends and to cleanly organize device logic via composition.
+## Support for CA, PVA, Tango
 
-We need to be able to:
+As well as the tradition EPICS Channel Access, ophyd-async was written to allow other Control system protocols, like EPICS PV Access and Tango. An ophyd-async [](#Signal) contains no control system specific logic, but takes a [](#SignalBackend) that it uses whenever it needs to talk to the control system. Likewise at the [](#Device) level, a [](#DeviceConnector) allows control systems to fulfil the type hints of [declarative devices](./declarative-vs-procedural.md).
 
-- Separate the Device interface from the multiple pieces of logic that might use that Device in a particular way
-- Define that Signals of a particular type exist without creating them so backends like Tango or EPICS + PVI can fill them in
+```{seealso}
+[](./devices-signals-backends.md) for more information on how these fit together, and [](../tutorials/implementing-devices.md) for examples of Devices in different control systems.
+```
 
+## Clean Device Definition
 
-Parity with Malcolm
--------------------
+For highly customizable devices like [PandABox](https://quantumdetectors.com/products/pandabox) there are often different pieces of logic that can talk to the same underlying hardware interface. The Devices in ophyd-async are structured so that the logic and interface can be split, and thus can be cleanly organized via composition rather than inheritance. 
 
-.. seealso:: `./flyscanning`
+## Ease the implementation of flyscanning
 
-Ophyd-async should provide the same building blocks for defining flyscans scans as malcolm_. It should support PandA and Zebra as timing masters by default, but also provide easy helpers for developers to write support for their own devices.
+One of the major drivers for ophyd-async was to ease the implementation of flyscanning. A library of flyscanning helpers is being developed to aid such strategies as:
+- Definition of scan paths via [ScanSpec](https://github.com/dls-controls/scanspec)
+- PVT Trajectory scanning in [Delta Tau motion controllers](https://github.com/dls-controls/pmac)
+- Position compare and capture using a [PandABox](https://quantumdetectors.com/products/pandabox)
 
-It should enable motor trajectory scanning and multiple triggering rates based around a base rate, and pausing/resuming scans. Scans should be modelled using scanspec_, which serves as a universal language for defining trajectory and time-resolved scans, and converted to the underlying format of the given motion controller. It should also be possible to define an outer scan .
+These strategies will be ported from DLS's previous flyscanning software [Malcolm](https://github.com/dls-controls/pymalcolm) and improved to take advantage of the flexibility of bluesky's plan definitions.
 
+```{seealso}
+The documents on flyscanning in the [bluesky cookbook](http://blueskyproject.io/bluesky-cookbook/glossary/flyscanning.html)
+```
 
-Improved Trajectory Calculation
--------------------------------
+## Parity and interoperativity with ophyd sync
 
-Ophyd-async will provide and improve upon the algorithms that malcolm_ uses to calculate trajectories for supported hardware.
-
-The EPICS pmac_ module supports trajectory scanning, specifying a growing array of positions, velocities and time for axes to move through to perform a scan. 
-Ophyd-async will provide mechanisms for specifying these scans via a scanspec_, calculating run-ups and turnarounds based on motor parameters, keeping the trajectory scan arrays filled based on the ScanSpec, and allowing this scan to be paused and resumed.
-
-
-Outstanding Design Decisions
-----------------------------
-
-To view and contribute to discussions on outstanding decisions, please see the design_ label in our Github issues.
-
-
-.. _ophyd: https://github.com/bluesky/ophyd
-.. _malcolm: https://github.com/dls-controls/pymalcolm
-.. _scanspec: https://github.com/dls-controls/scanspec
-.. _design: https://github.com/bluesky/ophyd-async/issues?q=is%3Aissue+is%3Aopen+label%3Adesign
-.. _pmac: https://github.com/dls-controls/pmac
+Devices from both ophyd sync and ophyd-async can be used in the same RunEngine and even in the same scan. This allows a per-device migration where devices are reimplemented in ophyd-async one by one. Eventually ophyd sync will gain feature parity with ophyd sync, supporting [the same set of devices as ophyd](https://blueskyproject.io/ophyd/user/reference/builtin-devices.html)

docs/explanations/device-connection-strategies.md

@@ -2,50 +2,47 @@
 
 There are various ways you can connect an ophyd-async Device, depending on whether you are running under a RunEngine or not. This article details each of those modes and why you might want to connect in that mode
 
-## Up front connection 
-
-In a sync context, the ophyd-async :python:`init_devices` requires the bluesky event-loop
-to connect to devices. In an async context, it does not.
-
-## Sync Context
-
-In a sync context the run-engine must be initialized prior to connecting to devices.
-We enfore usage of the bluesky event-loop in this context.
-
-The following will fail if :python:`RE = RunEngine()` has not been called already:
-
-.. code:: python
-
-  with init_devices():
-      device1 = Device1(prefix)
-      device2 = Device2(prefix)
-      device3 = Device3(prefix)
-
-The :python:`init_devices` connects to devices in the event-loop created in the run-engine.
-
-
-## Async Context
-
-In an async context device connection is decoupled from the run-engine.
-The following attempts connection to all the devices in the :python:`init_devices`
-before or after run-engine initialization.
-
-.. code:: python
-
-  async def connection_function() :
-      async with init_devices():
-          device1 = Device1(prefix)
-          device2 = Device2(prefix)
-          device3 = Device3(prefix)
-
-  asyncio.run(connection_function())
-
-The devices will be unable to be used in the run-engine unless they share the same event-loop.
-When the run-engine is initialised it will create a new background event-loop to use if one
-is not passed in with :python:`RunEngine(loop=loop)`.
-
-If the user wants to use devices in the async :python:`init_devices` within the run-engine
-they can either:
-
-* Run the :python:`init_devices` first and pass the event-loop into the run-engine.
-* Initialize the run-engine first and run the :python:`init_devices` using the bluesky event-loop.
+## Connections without a RunEngine
+
+It's unlikely that ophyd-async Devices will be used without a RunEngine in production, so connections without a RunEngine should be confined to tests. Inside an async test or fixture you will see code that calls [](#Device.connect):
+```python
+@pytest.fixture
+async def my_device():
+    device = MyDevice(name="device")
+    await device.connect(mock=True)
+    return device
+```
+or equivalently uses [](#init_devices) as an async context manager:
+```python
+@pytest.fixture
+async def my_device():
+    async with init_devices():
+        device = MyDevice()
+    return device
+```
+The second form will be used when there are multiple devices to create as the connect is done in parallel. Both of these will run tasks in the current running event loop.
+
+## Connections with a RunEngine
+
+In tests and in production, ophyd-async Devices will be connected in the RunEngine event-loop. This runs in a background thread, so the fixture or test will be synchronous.
+
+Assuming you have [created an RE fixture](#run-engine-fixture), then you can still use [](#init_devices), but as a sync context manager:
+```python
+@pytest.fixture
+def my_device(RE):
+    with init_devices():
+        device = MyDevice()
+    return device
+```
+This will use the RunEngine event loop in the background to connect the Devices, relying on the fact the RunEngine is a singleton to find it: if you don't create a RunEngine or use the RE fixture this will fail.
+
+Alternatively you can use the [](#ensure_connected) plan to connect the Device:
+```python
+@pytest.fixture
+def my_device(RE):
+    device = MyDevice(name="device")
+    RE(ensure_connected(device))
+    return device
+```
+
+Both are equivalent, but you are more likely to use the latter directly in a test case rather than in a fixture.

docs/explanations/devices-signals-backends.md

@@ -0,0 +1,5 @@
+# Devices, Signals and their Backends
+
+```{autoclasstree} ophyd_async.core.SignalRW ophyd_async.core.SignalR
+
+```

docs/how-to/choose-right-baseclass.md

@@ -1,15 +1,33 @@
-# Decision Flowchart for Creating a New ophyd_async Device
+# How to choose the right base class when implementing a new Device
 
-This document contains a decision flowchart designed to guide developers through the process of creating a new ophyd_async device in the Ophyd library. It outlines a series of decisions based on the device's capabilities, such as file writing, reading values from process variables (PVs), and mobility within scans. The flowchart helps in determining the appropriate class inheritance and methods to override for optimal device functionality and integration into the system.
+When writing a new Device there are several base classes to choose from that will give you certain functionality. This document details how to choose between them, and the next steps to take to implement your Device.
 
-```{mermaid}
-
-  flowchart TD
-    start([Start]) --> isFileWriting{Is it a File Writing Detector?}
-    isFileWriting -- Yes --> useStandardDetector[Use StandardDetector]
-    isFileWriting -- No --> producesPVValue{Does it produce a value from a PV you want to read in a scan?}
-    producesPVValue -- Yes --> isMovable{Is it something that you move in a scan?}
-    isMovable -- Yes --> useReadableMovable[Use StandardReadable + AsyncMovable + Override set method]
-    isMovable -- No --> useReadable[Use StandardReadable]
-    producesPVValue -- No --> useDevice[Use Device]
+```{seealso}
+[](../tutorials/implementing-devices.md) gives some examples of how these baseclass might be used
 ```
+
+## Utility baseclasses
+
+There are some utility baseclasses that allow you to create a Device pre-populated with the right verbs to work in bluesky plans:
+
+- [](#StandardReadable) allows you to compose the values of child Signals and Devices together so that you can `read()` the Device during a step scan.
+- [](#StandardDetector) allows file-writing detectors to be used within both step and flyscans, reporting periodic references to the data that has been written so far. An instance of a [](#DetectorController) and a [](#DetectorWriter) are required to provide this functionality.
+- [](#StandardFlyer) allows actuators (like a motor controller) to be used within a flyscan. Implementing a [](#FlyerController) is required to provide this functionality.
+
+## Adding verbs via protocols
+
+There are some [bluesky protocols](inv:bluesky#hardware) that show the verbs you can implement to add functionality in standard plans. For example:
+
+- [](#bluesky.protocols.Movable) to add behavior during `bps.mv` and `bps.abs_set`
+- [](#bluesky.protocols.Triggerable) to add behavior before `read()` in `bps.scan`
+
+It is not strictly required to add the protocol class as a baseclass (the presence of a method with the right signature is all that is required) but generally this is done so that the IDE gives you help when filling in the method, and the type checker knows to check that you have filled it in correctly.
+
+## Control system specific baseclass
+
+It is possible to create [procedural devices](../explanations/declarative-vs-procedural.md) using just [](#Device) as a baseclass, but if you wish to make a declarative Device then you need a control system specific baseclass:
+
+- [](#EpicsDevice) for EPICS CA and PVA devices
+- [](#TangoDevice) for Tango devices
+
+If you are creating a [](#StandardDetector) then multiple inheritance of that and the control system specific baseclass makes initialization order tricky, so you should use [](#TangoConnector), [](#EpicsConnector), [](#PviConnector) or [](#fastcs_connector) and pass it into the `super().__init__` rather than using the control system specific baseclass. An example of this is [](#HDFPanda).

docs/how-to/implement-ad-detector.md

@@ -1,3 +1,17 @@
+# How to implement a Device for an EPICS areaDetector
+
+This document will walk through the steps taken to implement an ophyd-async Device that talks to an EPICS areaDetector
+
+## Create a module to put the code in
+
+The first stage is to make a module in the `ophyd-async` repository to put the code in:
+
+- If you haven't already, `git clone [email protected]:bluesky/ophyd-async.git` and make a new branch to work in
+- Create a new directory under `src/ophyd_async/epics/` which is the lowercase version of the epics support module for the detector
+  - For example, for [`ADSimDetector`](https://github.com/areaDetector/ADSimDetector) make a directory `adsimdetector`
+- Make an empty `__init__.py` within that directory
+
+
 .. note::
 
    Ophyd async is included on a provisional basis until the v1.0 release and