forked from airbytehq/airbyte
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Docs: update contributing guides (airbytehq#27812)
* changes * update how to pr and readme * update contributeing guides * update code of conduct text * update submit new connector * submit new connector doc * make one line for submit new conn find request * improve words
- Loading branch information
1 parent
98a0dad
commit 62ee702
Showing
17 changed files
with
254 additions
and
331 deletions.
There are no files selected for viewing
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,66 @@ | ||
# Changes to CDK or Low-Code Connector | ||
|
||
#### Open an issue, or find a similar one. | ||
Before jumping into the code please first: | ||
1. Verify if an existing [connector](https://github.com/airbytehq/airbyte/issues?q=is%3Aopen+is%3Aissue+label%3Aarea%2Fconnectors+-label%3Aneeds-triage+label%3Acommunity) | ||
2. If you don't find an existing issue... | ||
1. [Report a Bug Connector](https://github.com/airbytehq/airbyte/issues/new?assignees=&labels=type%2Fbug%2Carea%2Fconnectors%2Cneeds-triage&projects=&template=1-issue-connector.yaml) | ||
2. [Request a New Connector Feature](https://github.com/airbytehq/airbyte/issues/new?assignees=&labels=type%2Fenhancement%2Cneeds-triage&projects=&template=6-feature-request.yaml) | ||
|
||
This will enable our team to make sure your contribution does not overlap with existing works and will comply with the design orientation we are currently heading the product toward. If you do not receive an update on the issue from our team, please ping us on [Slack](https://slack.airbyte.io)! | ||
|
||
:::info | ||
Make sure you're working on an issue had been already triaged to not have your contribution declined. | ||
::: | ||
|
||
#### Code your contribution | ||
1. To contribute to a connector, fork the [Connector repository](https://github.com/airbytehq/airbyte). | ||
2. Open a branch for your work | ||
3. Code the change | ||
4. To each custom function you added or changed its a must to write unit **tests**. | ||
5. Ensure all tests pass. For connectors, this includes acceptance tests as well. | ||
6. Update the `metadata.yaml`, `Dockerfile` version following the [guidelines](./resources/pull-requests-handbook.md#semantic-versioning-for-connectors) | ||
7. Update the changelog entry in documentation in `docs/integrations/<connector-name>.md` | ||
|
||
:::info | ||
There is a README file inside each connector folder with instructions to run test locally | ||
::: | ||
|
||
:::warning | ||
Pay attention to breaking changes to connectors. You can read more [here](#breaking-changes-to-connectors). | ||
::: | ||
|
||
|
||
#### Open a pull request | ||
1. Rebase master with your branch before submitting a pull request. | ||
2. Open the pull request. | ||
3. Follow the [title convention](./resources/pull-requests-handbook.md#pull-request-title-convention) for Pull Requests | ||
4. Link to an existing Issue | ||
5. Update the [description](./resources/pull-requests-handbook.md#descriptions) | ||
6. Wait for a review from a community maintainer or our team. | ||
|
||
#### 4. Review process | ||
When we review, we look at: | ||
* Does the PR solve the issue? | ||
* Is the proposed solution reasonable? | ||
* Is it tested? \(unit tests or integation tests\) | ||
* Is it introducing security risks? | ||
* Is it introducing a breaking change? | ||
Once your PR passes, we will merge it 🎉. | ||
|
||
|
||
## Breaking Changes to Connectors | ||
|
||
Often times, changes to connectors can be made without impacting the user experience. However, there are some changes that will require users to take action before they can continue to sync data. These changes are considered **Breaking Changes** and require a | ||
|
||
1. A **Major Version** increase | ||
2. An Airbyte Engineer to follow the [Connector Breaking Change Release Playbook](https://docs.google.com/document/u/0/d/1VYQggHbL_PN0dDDu7rCyzBLGRtX-R3cpwXaY8QxEgzw/edit) before merging. | ||
|
||
### Types of Breaking Change(s): | ||
A breaking change is any change that will require users to take action before they can continue to sync data. The following are examples of breaking changes: | ||
|
||
- **Spec Change** - The configuration required by users of this connector have been changed and syncs will fail until users reconfigure or re-authenticate. This change is not possible via a Config Migration | ||
- **Schema Change** - The type of a property previously present within a record has changed | ||
- **Stream or Property Removal** - Data that was previously being synced is no longer going to be synced. | ||
- **Destination Format / Normalization Change** - The way the destination writes the final data or how normalization cleans that data is changing in a way that requires a full-refresh.** | ||
- **State Changes** - The format of the source’s state has changed, and the full dataset will need to be re-synced |
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,23 @@ | ||
# Issues and Requests | ||
|
||
### Report a Bug | ||
Bug reports help us make Airbyte better for everyone. We provide a preconfigured template for bugs to make it very clear what information we need. | ||
|
||
Please search within our [already reported bugs](https://github.com/airbytehq/airbyte/issues?q=is%3Aissue+is%3Aopen+label%3Atype%2Fbug) before raising a new one to make sure you're not raising a duplicate. | ||
|
||
|
||
### Request new Features or Connector | ||
|
||
Requesting new features or connectors is an essential way to contribute. Your input helps us understand your needs and priorities, enabling us to enhance the functionality and versatility of Airbyte. | ||
|
||
### Reporting Security Issues | ||
|
||
Please do not create a public GitHub issue. If you've found a security issue, please email us directly at [[email protected]](mailto:[email protected]) instead of raising an issue. | ||
|
||
### Upvoting Issues, Features or Connector Requests | ||
|
||
You are welcome to add your own reactions to the existing issues. We will take them in consideration in our prioritization efforts, especially for connectors. | ||
|
||
❤️ means that this task is CRITICAL to you. | ||
|
||
👍 means it is important to you. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
File renamed without changes.
File renamed without changes.
File renamed without changes.
69 changes: 69 additions & 0 deletions
69
docs/contributing-to-airbyte/resources/pull-requests-handbook.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,69 @@ | ||
# Pull Request Handbook | ||
|
||
### Pull Request Title Convention | ||
|
||
When creating a pull request follow the naming conventions depending on the change being made. | ||
In general the pull request title starts with an emoji with the connector you're doing the changes, eg (✨ Source E-Commerce: add new stream `Users`). | ||
Airbyte uses this pattern to automatically assign team reviews and build the product release notes. | ||
|
||
| Pull Request Type | Emoji | Examples | | ||
| ----------------- | ----- | ---------| | ||
| New Connector (Source or Destination) | 🎉 | 🎉 New Destination: Database | | ||
| Add a feature to an existing connector | ✨ | ✨ Source E-Commerce: add new stream `Users` | | ||
| Fix a bug | 🐛 | 🐛 Source E-Commerce: fix start date parameter in spec | | ||
| Documentation (updates or new entries) | 📝 | 📝 Fix Database connector changelog | | ||
| It's a breaking change | 🚨 | 🚨🚨🐛 Source Kafka: fix a complex bug | | ||
|
||
For more information about [breaking changes](#breaking-changes-to-connectors). A maintainer will help and instruct about possible breaking changes. | ||
|
||
Any refactors, cleanups, etc.. that are not visible improvements to the user should not have emojis. | ||
|
||
If you're code change is doing more than one change type at once we strongly recommend to break into multiple pull requests. It helps us to review and merge your contribution. | ||
|
||
## Descriptions | ||
|
||
**Context**: Provide enough information \(or a link to enough information\) in the description so team members with no context can understand what the issue or PR is trying to accomplish. This usually means you should include two things: | ||
|
||
1. Some background information motivating the problem | ||
2. A description of the problem itself | ||
3. Good places to start reading and file changes that can be skipped | ||
|
||
Some examples: | ||
|
||
_insufficient context_: `Create an OpenAPI to JSON schema generator`. Unclear what the value or problem being solved here is. | ||
|
||
_good context_: | ||
|
||
```text | ||
When creating or updating connectors, we spend a lot of time manually transcribing JSON Schema files based on OpenAPI docs. This is ncessary because OpenAPI and JSON schema are very similar but not perfectly compatible. This process is automatable. Therefore we should create a program which converts from OpenAPI to JSONSchema format. | ||
``` | ||
|
||
## Semantic Versioning for Connectors | ||
|
||
Changes to connector behavior should always be accompanied by a version bump and a changelog entry. We use [semantic versioning](https://semver.org/) to version changes to connectors. Since connectors are a bit different from APIs, we have our own take on semantic versioning, focusing on maintaining the best user experience of using a connector. | ||
|
||
- Major: a version in which a change is made which requires manual intervention (update to config or configured catalog) for an existing connection to continue to succeed, or one in which data that was previously being synced will no longer be synced | ||
- Minor: a version that introduces user-facing functionality in a backwards compatible manner | ||
- Patch: a version that introduces backwards compatible bug fixes or performance improvements | ||
|
||
### Examples | ||
|
||
Here are some examples of code changes and their respective version changes: | ||
|
||
| Change | Impact | Version Change | | ||
|-----------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------|----------------| | ||
| Adding a required parameter to a connector's `spec` | Users will have to add the new parameter to their `config` | Major | | ||
| Changing a format of a parameter in a connector's `spec` from a single parameter to a `oneOf` | Users will have to edit their `config` to define their old parameter value in the `oneOf` format | Major | | ||
| Removing a stream from a connector's `catalog` | Data that was being synced will no longer be synced | Major | | ||
| Renaming a stream in a connector's `catalog` | Users will have to update the name of the stream in their `catalog` | Major | | ||
| Removing a column from a stream in a connector's `catalog` | Users will have to remove that column from their `catalog`, data that was being synced will no longer be synced | Major | | ||
| Renaming a column from a stream in a connector's `catalog` | Users will have to update the name of the column in their `catalog` | Major | | ||
| Changing the datatype for a column of a stream in a connector's `catalog` | Users will have to update that data type in their `catalog`, data that was being synced will have changed format | Major | | ||
| Adding a non-required parameter to a connector's `spec` | Users will have the option to use the required parameter in the future | Minor | | ||
| Adding a stream in a connector's `catalog` | Additional data will be synced | Minor | | ||
| Adding a column to a stream's schema in a connector's `catalog` | Additional data will be synced | Minor | | ||
| Updating the format of the connector's `STATE` | Incremental streams will automatically run a full refresh only for the next sync | Patch | | ||
| Optimizing a connector's performance | Syncs will be faster | Patch | | ||
| Fixing a bug in a connector | Some syncs that would have failed will now succeed | Patch | | ||
|
||
Trying to contribute, and don't see the change you want to make in this list? Call it out in your PR and your reviewer will help you pick the correct type of version change. Feel free to contribute the results back to this list! |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file was deleted.
Oops, something went wrong.
Oops, something went wrong.