Merge pull request #108 from ably/integration/protocol-2.0

Integration: Protocol Version `2`

.github/workflows/assemble.yml

@@ -5,6 +5,8 @@ on:
   push:
     branches:
       - main
+    tags:
+      - 'v*'
 
 jobs:
   build:

CONTRIBUTING.md

@@ -42,11 +42,35 @@ by adding a local development HTTP server.
 When making changes to [the spec](textile/features.textile), please follow these guidelines:
 
 - **Ordering**: Spec items should generally appear in ID order, but priority should be placed on ordering them in a way that makes coherent sense, even if that results in them being numbered out-of-order. For example, if `XXX1`, `XXX2` and `XXX3` exist but it would make more sense for `XXX3` to follow `XXX1`, then just move the spec items accordingly without changing their IDs
-- **Addition**: When adding a new spec item, choose an ID that is greater than all others that exist in the given section, even if there is a gap in the currently assigned IDs. This is desirable so that client library references to spec items are still semantically valid even after they are removed from the spec rather than now having different semantics due to the ID being re-used. For example, if `XXX1a` and `XXX1c` exist but `XXX1b` doesn’t because it was removed in the past, then introduce `XXX1d` for the new spec item rather than re-using `XXX1b`
-- **Removal**: When removing a spec item, it must remain but replace all text with “This clause has been deleted.”. See [#1057](https://github.com/ably/docs/pull/1057) for an example of this in practice.
+- **Addition**: When adding a new spec item, choose an ID that is greater than all others that exist in the given section, even if there is a gap in the currently assigned IDs.
+- **Modification**: Spec items should never be mutated, except to patch a mistake that doesn't change the semantics for SDK implementations. Follow the guidance outlined here in respect of _Replacement_ if the meaning or scope of a spec point needs to change.
+- **Removal**: When removing a spec item, it must remain but replace all text with `This clause has been deleted. It was valid up to and including specification version @X.Y@.` (uses textile markup).
+- **Replacement**: When replacing a spec item, it must remain but replace all text with `This clause has been replaced by "@Z@":#Z. It was valid up to and including specification version @X.Y@.` (uses textile markup).
 - **Deprecation**: Our approach to deprecating features is yet to be fully evolved and documented, however we have a current standard in place whereby the text "(deprecated)" is inserted at the beginning of a specification point to declare that it will be removed in a future release. The likely outcome is that in the next major release of the spec/protocol we'll remove that spec item, per guidance above.
 
+### Additional Notes on Features Spec Point _Removal_ and _Replacement_
+
+Specification version references included in _Removal_ and _Replacement_ notices are in the form `X.Y` because they only need to include the `major` (`X`) and `minor` (`Y`) components of the specification version. See [Specification Version](README.md#specification-version).
+
+If the clause being removed or replaced has subclauses then the expected implication is that they are also being removed or replaced. Therefore, aligned with this guidance, they must remain but with their text replaced. This might mean that in some cases, for example, a clause is marked as replaced but some or all of its subclauses are marked as removed.
+
+Variations on the replacement text for removed or replaced spec items is allowed, as long as the overarching structure remains the same. For example:
+
+- When a spec item has been removed because a new spec point has made it redudant, which is the case for `RTN15f` after specification version `1.2`, where the replacement text is: `This clause has been deleted (redundant to "RTN19a":#RTN19a). It was valid up to and including specification version @1.2@.`
+- When a spec item has been replaced by more than one new spec item, which is the case for `RTN16b` after specification version `1.2`, where the replacement text is: `This clause has been replaced by "@RTN16g@":#RTN16g and "@RTN16m@":#RTN16m. It was valid up to and including specification version @1.2@.`
+
+Historically, before the above guidance was established - in particular around _Removal_ and _Replacement_ - there have been some cases where spec points were completely deleted.
+This left us open to the problem that client library references to spec items could end up semantically invalid if that spec point was re-used later.
+For example, if `XXX1a` and `XXX1c` exist but `XXX1b` doesn’t because it was removed in the past (prior to this guidance being established), then we should introduce `XXX1d` for the new spec item rather than re-using `XXX1b`.
+
 ## Release Process
 
-This will be documented when we work on
-[#6](https://github.com/ably/specification/issues/6).
+Use our standard [Release Process](https://github.com/ably/engineering/blob/main/sdk/releases.md#release-process), where:
+
+- there is no 'Publish Workflow' to be triggered in this repository
+- the version to be bumped in the release branch is the [Specification Version](README.md#specification-version)
+- in addition to pushing a full Git version tag (to persist as static and immutable) in the form `v<major>.<minor>.<patch>`, tags should also be added or moved for higher levels of specification version granularity:
+  - `v<major>.<minor>` - e.g. `v1.2` for latest patch to version 1.2 of the specification
+  - `v<major>` - e.g. `v2` for the latest enhancements and patches to version 2 of the specification
+
+It's worth emphasising here, for clarity, that bumps to the [Protocol Version](README.md#protocol-version) or [Build Version](README.md#build-version) are not part of the release process and do not appear in release branches or the pull requests that represent those release branches. If these versions need to be bumped then that is done as part of a feature change or addition, where that feature is then subsequently implicitly incorporated into a release at the time of executing the release process.

README.md

@@ -8,6 +8,75 @@ _[Ably](https://ably.com) is the platform that powers synchronized digital exper
 
 This repository has been created as the new home for the 'features spec' that describes the interfaces, implementation details and behaviours of SDKs (sometimes referred to as client libraries) that provide application developers with support to integrate and leverage the Ably platform in their solutions.
 
+## Versions
+
+Defined within the contents of this repository are multiple version numbers:
+
+| Version | Format | Location of Definition | Scope |
+| ------- | ------ | ---------------------- | ----- |
+| [**Specification**](#specification-version) | [SemVer](https://semver.org/) | The `versions.specification` field in [`meta.yaml`](meta.yaml) | Format and usage are specified in `CSV1`. The specification documents - files in [the `textile/` folder](textile/), the 'content'. |
+| [**Protocol**](#protocol-version) | Integer from version `2` onwards (it was Decimal prior to that, for example `1.2`). | The `versions.protocol` field in [`meta.yaml`](meta.yaml) | Format and usage are specified in `CSV2`. The wire protocol used when communicating with the Ably service. Specified for REST requests under `RSC7a` and for Realtime connections under `RTN2f`. Sometimes referred to as 'API version' (when 'API' is referring to details of the protocols understood by the Ably service). |
+| [**Build**](#build-version) | [SemVer](https://semver.org/) | The `version` field in [`package.json`](package.json) | Other files hosted in this repository which render the specification for viewers or otherwise check it or build artifacts from it. |
+
+### Specification Version
+
+These documents contain the authorative guidance for developers writing and maintaining SDKs that interact with the Ably service.
+
+Examples of changes that would result in a `major` bump, being backwards incompatible changes, include:
+
+- The SDK API used by app developers has changed in a way that is breaking
+- Required SDK behaviour has changed in a way that necessitates that changes are made to SDK implementations
+- There is a change to the way that the service interacts with SDKs when they announce compatibility with a newer [protocol version](#protocol-version)
+
+Examples of changes that would result in a `minor` bump, being backwards compatible changes, include:
+
+- The SDK API used by app developers has changed in a way that is backwards compatible (typically a new feature or enhancement)
+- There is new required SDK behaviour that only needs SDKs to add new behaviours, without modifying existing ones
+- There is a new element of the wire protocol which SDKs may start using in order to offer a new feature/enhancement
+
+Examples of changes that would result in a `patch` bump include:
+
+- Spelling mistake or typo corrections, where the meaning was previously clear and hasn't changed as a result of the fix
+- Formatting improvements in the textile markup, where those changes don't alter the way that the rendered output is interpreted
+- Improvements intended to make meaning clearer, where the intended meaning has not changed  - clearly these are changes that are likely to be subjective in nature, so it is suggested that these changes are rigorously reviewed by as many individuals as possible
+
+Where the 'SDK API' refers to the IDL and interface names documented in the [features spec](textile/features.textile),
+representing the surface area of the SDK that is directly visible to and used by app developers who are using that SDK to use the Ably service.
+
+It is expected that there may be specification changes which involve features spec point _Removal_, _Replacement_ or _Deprecation_ where that change only necessitates a `minor` bump to the specification version number - that is, the impact of the change is deemed backwards compatible according to the constraints outlined above. See [Contributing Guidance: Features Spec Points](CONTRIBUTING.md#features-spec-points).
+
+It is not anticipated that specification versions will ever need to use a pre-release suffix.
+The specification is primarily used by SDK developers and bumps to the specification version will only be made after the changes being released have been thoroughly reviewed and likely to have already have been implemented/prototyped in at least one SDK.
+Therefore, the idea of a 'preview' release of the specification doesn't make sense.
+
+See also:
+
+- [Ably SDK Team: Guidance on Releases: Version Bump](https://github.com/ably/engineering/blob/main/sdk/releases.md#version-bump)
+
+### Protocol Version
+
+Incremented when changes are made to the Ably service where at least one of the following applies:
+
+- the wire protocol API has changed
+- the behaviour of the service has changed in a way that SDKs need to reflect or respond to
+- the behaviour of the SDK needs to change in a way that the service needs to reflect or respond to
+
+We are using a simple integer value here because there is no need for structured versioning.
+When an SDK adopts a new protocol version then it's 'all or nothing', in that all implications of supporting that protocol version must be catered for by the changes made to that SDK in order to adopt the new protocol version.
+
+Whether SDK changes are as a result of wire protocol changes that are breaking (incompatible) or enhancing in nature is not something that we need the protocol version to imply by its value.
+The Ably service promises to support SDKs connecting to it using older protocol versions.
+This 'service makes right' feature of the service means that all we need the protocol version for is to track when a behavioural change has been made to the service implementation, alongside its corresponding wire protocol API, that requires the service to know that the SDK it's communicating with also understands this new behaviour.
+
+### Build Version
+
+This code (including `.js`, Ruby, `.html.hbs` and `.css` files) has been written specifically to read and process the files contained in this repository that contain specification content.
+Variously they may be referred to as 'build tools' or 'toolchain'.
+
+For the time being there is no exposure of these tools outside of this repository - that is, they are used for [local development](CONTRIBUTING.md#local-development-workflow) and are also executed from our [GitHub workflows](.github/workflows/).
+For this reason, we have no need to have any formal release process for them.
+The `1.0.0` value for `version` specified in [`package.json`](package.json) carries nominal importance and will remain static until we get to a point of needing it to change, for example if we end up exporting an npm package from here.
+
 ## Future Direction for Specification Point Adherence Tracking
 
 We have

meta.yaml

@@ -0,0 +1,8 @@
+versions:
+  # Must conform to Semantic Versioning (https://semver.org/).
+  # Should never need to use a pre-release suffix.
+  specification: 2.0.0
+
+  # Must be an Integer value.
+  # Previously, prior to version 2 (i.e. versions 0.8, 1.0, 1.1 and 1.2) it was a Decimal value.
+  protocol: 2

package.json

@@ -1,6 +1,6 @@
 {
-  "name": "@ably/specification",
-  "version": "1.2.0-alpha.1",
+  "name": "@ably/specification-build",
+  "version": "1.0.0",
   "scripts": {
     "build": "npm-run-all build:generate build:tailwind",
     "build:generate": "./scripts/build",

scripts/build

@@ -16,6 +16,7 @@ ROOT_SOURCE_NAME = 'index'
 TEMPLATE_PATH = File.expand_path('../../templates/docs-textile.html.hbs', __FILE__)
 OUTPUT_PATH = File.expand_path('../../output', __FILE__)
 COPYRIGHT_PATH = File.expand_path('../../COPYRIGHT', __FILE__)
+META_PATH = File.expand_path('../../meta.yaml', __FILE__)
 
 BUILD_CONTEXT_SHA = ENV['ABLY_BUILD_CONTEXT_SHA']
 BUILD_CONTEXT_URL = ENV['ABLY_BUILD_CONTEXT_URL']
@@ -71,6 +72,8 @@ def remove_frontmatter(file_contents)
   body_contents
 end
 
+versions = YAML.load_file(META_PATH)['versions']
+
 handlebars = Handlebars::Handlebars.new
 template = handlebars.compile(File.read(TEMPLATE_PATH))
 
@@ -84,7 +87,14 @@ plain_file_names = source_file_names.collect {
 } - [ROOT_SOURCE_NAME]
 
 source_file_names.each do |file_name|
+  print "#{file_name} ... "
+
+  # We could possibly have used the Handlebars gem we're using elsewhere for this, however it was barfing on features.textile for no clear reason.
+  # Regular expression find and replace (gsub) works for our simple needs here, so we're using that for simplicity.
   textile = remove_frontmatter(File.read(File.join(SOURCE_PATH, file_name)))
+    .gsub(/\{\{\s*SPECIFICATION_VERSION\s*\}\}/, versions['specification'].to_s)
+    .gsub(/\{\{\s*PROTOCOL_VERSION\s*\}\}/, versions['protocol'].to_s)
+
   bodyHtml = RedCloth.new(textile, [:no_span_caps]).to_html(*REDCLOTH_RULES)
   plain_file_name = file_name.delete_suffix(SOURCE_EXTENSION)
   is_root = (plain_file_name == ROOT_SOURCE_NAME)
@@ -103,5 +113,5 @@ source_file_names.each do |file_name|
     Dir.mkdir folder_path
   end
   File.write(File.join(folder_path, "index.html"), html)
-  puts "✓ #{file_name}"
+  puts "✓"
 end