-
Notifications
You must be signed in to change notification settings - Fork 102
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
260 changed files
with
11,865 additions
and
159 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
133 changes: 133 additions & 0 deletions
133
dotnet-test-samples/schema-and-contract-testing/README.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,133 @@ | ||
| # .NET: Schema and Contract testing for Event-Driven Architectures | ||
|
|
||
| --- | ||
|
|
||
| ## Contents | ||
|
|
||
| - [Introduction](#introduction) | ||
| - [Schema testing](#schema-testing-schema-difference) | ||
| - [Goal](#goal) | ||
| - [Description](#description) | ||
| - [System under test](#system-under-test-sut) | ||
| - [Limitations](#limitations) | ||
| - [Contract testing](#contract-testing) | ||
|
|
||
| - [Goal](#goal-1) | ||
| - [Description](#description-1) | ||
| - [System under test](#system-under-test-sut-1) | ||
| - [Limitations](#limitations-1) | ||
|
|
||
| - [Key Files in the Project](#key-files-in-this-project) | ||
| - [Run the tests](#run-the-tests) | ||
|
|
||
| --- | ||
|
|
||
| ## Introduction | ||
|
|
||
| Event driven architectures decouple producers and consumers at the infrastructure layer, but these resources may still be coupled at the application layer by the event contract. Consumers may write business logic that expects events to conform to specific schemas. If schemas change over time the consumer code may fail. Creating automated tests that validate event schemas can help increase the quality of your systems and prevent breaking changes. | ||
|
|
||
| ## Schema Testing | ||
|
|
||
| ### Goal | ||
|
|
||
| To prevent event schema changes from breaking consumer applications. | ||
|
|
||
| ### Description | ||
|
|
||
| Schema tests are designed to check for changes that are not backward-compatible, such as removing elements, changing a field type or adding new mandatory elements. Adding optional elements is considered backward-compatible. If a new schema version is backward-compatible, the schema tests will pass and the consumer application business logic should not fail. | ||
|
|
||
| ### System Under Test (SUT) | ||
|
|
||
| This test is integration service agnostic, as it runs against the schema published to a schema registry. Amazon EventBridge Schema Registry is one example of such a registry. It provides schemas for all events that are generated by AWS services. When you turn on schema discovery it will automatically detect and provide schemas for your custom events (OpenAPI and JSON Schemas). You can also use your own Git repository as a registry for storing schemas. In our test we will use a local “schemas” folder as a registry for simplicity. | ||
|
|
||
| The diagram below shows an example of how schema tests may run as part of a producer’s CI/CD pipeline. You can [view a sample schema test](tests/schema-testing.test.ts) that could be run during the "Build" step of the process. | ||
|
|
||
|  | ||
|
|
||
| ### Schema Test Limitations | ||
|
|
||
| Schema tests have two limitations worth mentioning. | ||
|
|
||
| First, schemas are inherently ambiguous as they do not capture business requirements. They may not express that a particular combination of fields is invalid, or they may not explicitly state the dependencies between fields. For example, when you create an AWS Lambda function, the `runtime` property is mandatory for zip packaging formats, but this field is not required for container image packaging formats. The schema for the Lambda `create-function` API payload will not convey this requirement. If an event consumer makes invalid assumptions about a new version of the payload, its business logic may fail even if there are technically no breaking schema changes. | ||
|
|
||
| Second, unless an event consumer uses SDK/code bindings generated specifically for the version of schema in use, the code and the schema may drift apart over time and the consumer business logic may eventually fail. In these cases, the schema validation test results create a false sense of security. | ||
|
|
||
| Both of these limitations can be addressed with [contract testing](#contract-testing). | ||
|
|
||
| ## Contract Testing | ||
|
|
||
| ### System Under Test (SUT) | ||
|
|
||
| This test is integration service agnostic as it uses sample events generated from a schema published to a schema registry. The diagram below shows one way of implementing contract testing as part of producer’s CI/CD pipeline. | ||
|
|
||
| The diagram below shows an example of how contract testing can be used as part of a producer’s CI/CD pipeline. You can [view a sample contract test](tests/SchemaTesting.UnitTest.ContractTests.cs) that could be run during the "Build" step of the process. | ||
|
|
||
| In this example, the sample schema is [loaded from disk.](tests/SchemaTesting.UnitTest/LocalDiskSchemaReader.cs) This could be replaced to retrieve schemas from EventBridge Schema Registry using the EventBridge SDK. An example of this can be found in the [EventBridgeSchemaRegistryReader class](tests/SchemaTesting.UnitTest/EventBridgeSchemaRegistryReader.cs). | ||
|
|
||
|  | ||
|
|
||
| ### Goal | ||
|
|
||
| To identify the side effects of non-breaking schema changes on consumer applications. | ||
|
|
||
| ### Description | ||
|
|
||
| Contract testing uses sample events provided by producers to validate consumer side business logic and to provide a stronger guarantee of preventing breaking changes. Contract tests are owned by consumers. | ||
|
|
||
| Consider the following example. In a schema test, changing the format of an `address` field from a full address to a street address only and then adding new event fields like `country`, `city`, `state` and `postcode` would not produce a failure. These schema changes would be considered to be backward compatible, since the changes only add new elements. In contrast, a consumer test expecting a full address in the `address` field would fail and notify us that this change is actually not backward compatible. | ||
|
|
||
| There are other common strategies for implementing contract testing. One strategy is to establish test doubles. Test doubles are resources that replace production objects for testing purposes. The contract tests with test doubles run on the consumer side and expect that calls to the doubles return the same results as calls to the production application. Review the [Pact.io documentation](https://docs.pact.io/) for examples of this approach. | ||
|
|
||
| ### Limitations | ||
|
|
||
| Writing tests and performing on-going maintenance requires a degree of additional effort. However once written, schema changes can be done more efficiently and with less effort to prevent and resolve breaking changes. | ||
|
|
||
| You may choose to be selective in deciding where to implement contract testing. Start by identifying critical applications where the cost of breaking changes is highest. Applications with greater tolerances for failure may choose to rely on schema testing only. | ||
|
|
||
| --- | ||
|
|
||
| ### Key Files in this Project | ||
|
|
||
| - [SchemaTesting.cs](tests/SchemaTesting.UnitTest/SchemaTests.cs) - unit test for schema testing | ||
| - [ContractTesting.cs](tests/SchemaTesting.UnitTest/ContractTests.cs) - unit test for contract testing | ||
| - [schemas](schemas) - versioned schemas used for schema testing | ||
| - [events](events) - sample events used for contract testing | ||
|
|
||
| ## Deploy the sample application | ||
|
|
||
| This example includes both local tests and tests that can execute against the Amazon Event Bridge Schema Registry. To use schema registry for testing, you first need to deploy the sample application and then run some load through the system. | ||
|
|
||
| ```bash | ||
| sam build | ||
| sam deploy --guided | ||
| ``` | ||
|
|
||
| You can make a POST request to the generated API endpoint on the `/customer` route, using the below request body. | ||
|
|
||
| ```json | ||
| { | ||
| "firstName": "James", | ||
| "lastName": "Eastham", | ||
| "address": "Holborn Viaduct, London, Greater London, England, The World" | ||
| } | ||
| ``` | ||
|
|
||
| Check your Amazon Event Bridge Schema Registry. You will see events under the `com.dev.customer@CustomerCreatedEventV1` heading. It can take a few minutes for new schemas to be registered. | ||
|
|
||
| ## Run the Tests | ||
|
|
||
| ### Bash | ||
| ``` | ||
| cd ./tests/SchemaTesting.UnitTests | ||
| dotnet test | ||
| ``` | ||
|
|
||
| ### Windows | ||
| ``` | ||
| cd .\tests\SchemaTesting.UnitTests | ||
| dotnet test | ||
| ``` | ||
|
|
||
| [Top](#contents) | ||
|
|
||
| --- |
Oops, something went wrong.