Skip to content

Commit

Permalink
Docs: update contributing guides (airbytehq#27812)
Browse files Browse the repository at this point in the history
* changes

* update how to pr and readme

* update contributeing guides

* update code of conduct text

* update submit new connector

* submit new connector doc

* make one line for submit new conn find request

* improve words
  • Loading branch information
marcosmarxm authored Jun 30, 2023
1 parent 98a0dad commit 62ee702
Show file tree
Hide file tree
Showing 17 changed files with 254 additions and 331 deletions.
207 changes: 44 additions & 163 deletions docs/contributing-to-airbyte/README.md

Large diffs are not rendered by default.

66 changes: 66 additions & 0 deletions docs/contributing-to-airbyte/change-cdk-connector.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
# Changes to CDK or Low-Code Connector

#### Open an issue, or find a similar one.
Before jumping into the code please first:
1. Verify if an existing [connector](https://github.com/airbytehq/airbyte/issues?q=is%3Aopen+is%3Aissue+label%3Aarea%2Fconnectors+-label%3Aneeds-triage+label%3Acommunity)
2. If you don't find an existing issue...
1. [Report a Bug Connector](https://github.com/airbytehq/airbyte/issues/new?assignees=&labels=type%2Fbug%2Carea%2Fconnectors%2Cneeds-triage&projects=&template=1-issue-connector.yaml)
2. [Request a New Connector Feature](https://github.com/airbytehq/airbyte/issues/new?assignees=&labels=type%2Fenhancement%2Cneeds-triage&projects=&template=6-feature-request.yaml)

This will enable our team to make sure your contribution does not overlap with existing works and will comply with the design orientation we are currently heading the product toward. If you do not receive an update on the issue from our team, please ping us on [Slack](https://slack.airbyte.io)!

:::info
Make sure you're working on an issue had been already triaged to not have your contribution declined.
:::

#### Code your contribution
1. To contribute to a connector, fork the [Connector repository](https://github.com/airbytehq/airbyte).
2. Open a branch for your work
3. Code the change
4. To each custom function you added or changed its a must to write unit **tests**.
5. Ensure all tests pass. For connectors, this includes acceptance tests as well.
6. Update the `metadata.yaml`, `Dockerfile` version following the [guidelines](./resources/pull-requests-handbook.md#semantic-versioning-for-connectors)
7. Update the changelog entry in documentation in `docs/integrations/<connector-name>.md`

:::info
There is a README file inside each connector folder with instructions to run test locally
:::

:::warning
Pay attention to breaking changes to connectors. You can read more [here](#breaking-changes-to-connectors).
:::


#### Open a pull request
1. Rebase master with your branch before submitting a pull request.
2. Open the pull request.
3. Follow the [title convention](./resources/pull-requests-handbook.md#pull-request-title-convention) for Pull Requests
4. Link to an existing Issue
5. Update the [description](./resources/pull-requests-handbook.md#descriptions)
6. Wait for a review from a community maintainer or our team.

#### 4. Review process
When we review, we look at:
* ‌Does the PR solve the issue?
* Is the proposed solution reasonable?
* Is it tested? \(unit tests or integation tests\)
* Is it introducing security risks?
* Is it introducing a breaking change?
‌Once your PR passes, we will merge it 🎉.


## Breaking Changes to Connectors

Often times, changes to connectors can be made without impacting the user experience.  However, there are some changes that will require users to take action before they can continue to sync data.  These changes are considered **Breaking Changes** and require a

1. A **Major Version** increase 
2. An Airbyte Engineer to follow the  [Connector Breaking Change Release Playbook](https://docs.google.com/document/u/0/d/1VYQggHbL_PN0dDDu7rCyzBLGRtX-R3cpwXaY8QxEgzw/edit) before merging.

### Types of Breaking Change(s):
A breaking change is any change that will require users to take action before they can continue to sync data. The following are examples of breaking changes:

- **Spec Change** - The configuration required by users of this connector have been changed and syncs will fail until users reconfigure or re-authenticate.  This change is not possible via a Config Migration 
- **Schema Change** - The type of a property previously present within a record has changed
- **Stream or Property Removal** - Data that was previously being synced is no longer going to be synced.
- **Destination Format / Normalization Change** - The way the destination writes the final data or how normalization cleans that data is changing in a way that requires a full-refresh.**
- **State Changes** - The format of the source’s state has changed, and the full dataset will need to be re-synced
51 changes: 0 additions & 51 deletions docs/contributing-to-airbyte/issues-and-pull-requests.md

This file was deleted.

23 changes: 23 additions & 0 deletions docs/contributing-to-airbyte/issues-and-requests.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# Issues and Requests

### Report a Bug
Bug reports help us make Airbyte better for everyone. We provide a preconfigured template for bugs to make it very clear what information we need.

‌Please search within our [already reported bugs](https://github.com/airbytehq/airbyte/issues?q=is%3Aissue+is%3Aopen+label%3Atype%2Fbug) before raising a new one to make sure you're not raising a duplicate.


### Request new Features or Connector

Requesting new features or connectors is an essential way to contribute. Your input helps us understand your needs and priorities, enabling us to enhance the functionality and versatility of Airbyte.

### Reporting Security Issues

Please do not create a public GitHub issue. If you've found a security issue, please email us directly at [[email protected]](mailto:[email protected]) instead of raising an issue.

### Upvoting Issues, Features or Connector Requests

You are welcome to add your own reactions to the existing issues. We will take them in consideration in our prioritization efforts, especially for connectors.

❤️ means that this task is CRITICAL to you.

👍 means it is important to you.
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ Install it in IntelliJ:
2. Turn on the auto add final. Go into IntelliJ Preferences
1. Plugins - install Save Actions if not already installed
1. If you're running Intellij 2023.1 or higher, the official version may not work. Try manually installing [this fork](https://github.com/fishermans/intellij-plugin-save-actions/releases/tag/v2.6.0) (see [Github issue](https://github.com/dubreuia/intellij-plugin-save-actions/issues/427)).
2. Go to Save Actions in the preferences [left navigation column](../assets/docs/save_actions_settings.png) (NOT Tools > Actions on Save -- that is a different tool)
2. Go to Save Actions in the preferences [left navigation column](../../assets/docs/save_actions_settings.png) (NOT Tools > Actions on Save -- that is a different tool)
1. `Activate save actions on save` > check the box
2. `Active save actions on shortcut` > check the box
3. `Activate save actions on batch` > check the box
Expand Down
File renamed without changes.
69 changes: 69 additions & 0 deletions docs/contributing-to-airbyte/resources/pull-requests-handbook.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
# Pull Request Handbook

### Pull Request Title Convention

When creating a pull request follow the naming conventions depending on the change being made.
In general the pull request title starts with an emoji with the connector you're doing the changes, eg (✨ Source E-Commerce: add new stream `Users`).
Airbyte uses this pattern to automatically assign team reviews and build the product release notes.

| Pull Request Type | Emoji | Examples |
| ----------------- | ----- | ---------|
| New Connector (Source or Destination) | 🎉 | 🎉 New Destination: Database |
| Add a feature to an existing connector || ✨ Source E-Commerce: add new stream `Users` |
| Fix a bug | 🐛 | 🐛 Source E-Commerce: fix start date parameter in spec |
| Documentation (updates or new entries) | 📝 | 📝 Fix Database connector changelog |
| It's a breaking change | 🚨 | 🚨🚨🐛 Source Kafka: fix a complex bug |

For more information about [breaking changes](#breaking-changes-to-connectors). A maintainer will help and instruct about possible breaking changes.

Any refactors, cleanups, etc.. that are not visible improvements to the user should not have emojis.

If you're code change is doing more than one change type at once we strongly recommend to break into multiple pull requests. It helps us to review and merge your contribution.

## Descriptions

**Context**: Provide enough information \(or a link to enough information\) in the description so team members with no context can understand what the issue or PR is trying to accomplish. This usually means you should include two things:

1. Some background information motivating the problem
2. A description of the problem itself
3. Good places to start reading and file changes that can be skipped

Some examples:

_insufficient context_: `Create an OpenAPI to JSON schema generator`. Unclear what the value or problem being solved here is.

_good context_:

```text
When creating or updating connectors, we spend a lot of time manually transcribing JSON Schema files based on OpenAPI docs. This is ncessary because OpenAPI and JSON schema are very similar but not perfectly compatible. This process is automatable. Therefore we should create a program which converts from OpenAPI to JSONSchema format.
```

## Semantic Versioning for Connectors

Changes to connector behavior should always be accompanied by a version bump and a changelog entry. We use [semantic versioning](https://semver.org/) to version changes to connectors. Since connectors are a bit different from APIs, we have our own take on semantic versioning, focusing on maintaining the best user experience of using a connector.

- Major: a version in which a change is made which requires manual intervention (update to config or configured catalog) for an existing connection to continue to succeed, or one in which data that was previously being synced will no longer be synced
- Minor: a version that introduces user-facing functionality in a backwards compatible manner
- Patch: a version that introduces backwards compatible bug fixes or performance improvements

### Examples

Here are some examples of code changes and their respective version changes:

| Change | Impact | Version Change |
|-----------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------|----------------|
| Adding a required parameter to a connector's `spec` | Users will have to add the new parameter to their `config` | Major |
| Changing a format of a parameter in a connector's `spec` from a single parameter to a `oneOf` | Users will have to edit their `config` to define their old parameter value in the `oneOf` format | Major |
| Removing a stream from a connector's `catalog` | Data that was being synced will no longer be synced | Major |
| Renaming a stream in a connector's `catalog` | Users will have to update the name of the stream in their `catalog` | Major |
| Removing a column from a stream in a connector's `catalog` | Users will have to remove that column from their `catalog`, data that was being synced will no longer be synced | Major |
| Renaming a column from a stream in a connector's `catalog` | Users will have to update the name of the column in their `catalog` | Major |
| Changing the datatype for a column of a stream in a connector's `catalog` | Users will have to update that data type in their `catalog`, data that was being synced will have changed format | Major |
| Adding a non-required parameter to a connector's `spec` | Users will have the option to use the required parameter in the future | Minor |
| Adding a stream in a connector's `catalog` | Additional data will be synced | Minor |
| Adding a column to a stream's schema in a connector's `catalog` | Additional data will be synced | Minor |
| Updating the format of the connector's `STATE` | Incremental streams will automatically run a full refresh only for the next sync | Patch |
| Optimizing a connector's performance | Syncs will be faster | Patch |
| Fixing a bug in a connector | Some syncs that would have failed will now succeed | Patch |

Trying to contribute, and don't see the change you want to make in this list? Call it out in your PR and your reviewer will help you pick the correct type of version change. Feel free to contribute the results back to this list!
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,6 @@ By default, the find function in IntelliJ is not scoped and will include all fil
4. Add the following filter to the `Exclude files` option: `connectors/**/.venv`
5. Press OK to confirm your options.

![](../.gitbook/assets/monorepo-exclude-files.png)

#### Manual Workaround

Expand Down
34 changes: 0 additions & 34 deletions docs/contributing-to-airbyte/sonar-qube-workflow.md

This file was deleted.

Loading

0 comments on commit 62ee702

Please sign in to comment.