Skip to content

Commit

Permalink
Update _data-prepper/pipelines/configuration/sinks/s3.md
Browse files Browse the repository at this point in the history
Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Melissa Vagi <[email protected]>
  • Loading branch information
vagimeli and natebower authored Jun 25, 2024
1 parent a0b8d94 commit 11a2612
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion _data-prepper/pipelines/configuration/sinks/s3.md
Original file line number Diff line number Diff line change
Expand Up @@ -167,7 +167,7 @@ The `codec` determines how the `s3` source formats data written to each S3 objec

### `avro` codec

The `avro` codec writes an event as an [Apache Avro](https://avro.apache.org/) document. Because Avro requires a schema, you may either define the schema or have Data Prepper automatically generate it. Defining your own schema is recommended, as this will allow it to be tailored to your particular use case.
The `avro` codec writes an event as an [Apache Avro](https://avro.apache.org/) document. Because Avro requires a schema, you may either define the schema or have Data Prepper automatically generate it. Defining your own schema is recommended because this will allow it to be tailored to your particular use case.

Check failure on line 170 in _data-prepper/pipelines/configuration/sinks/s3.md

View workflow job for this annotation

GitHub Actions / style-job

[vale] reported by reviewdog 🐶 [OpenSearch.Spelling] Error: Avro. If you are referencing a setting, variable, format, function, or repository, surround it with tic marks. Raw Output: {"message": "[OpenSearch.Spelling] Error: Avro. If you are referencing a setting, variable, format, function, or repository, surround it with tic marks.", "location": {"path": "_data-prepper/pipelines/configuration/sinks/s3.md", "range": {"start": {"line": 170, "column": 48}}}, "severity": "ERROR"}

Check failure on line 170 in _data-prepper/pipelines/configuration/sinks/s3.md

View workflow job for this annotation

GitHub Actions / style-job

[vale] reported by reviewdog 🐶 [OpenSearch.Spelling] Error: Avro. If you are referencing a setting, variable, format, function, or repository, surround it with tic marks. Raw Output: {"message": "[OpenSearch.Spelling] Error: Avro. If you are referencing a setting, variable, format, function, or repository, surround it with tic marks.", "location": {"path": "_data-prepper/pipelines/configuration/sinks/s3.md", "range": {"start": {"line": 170, "column": 98}}}, "severity": "ERROR"}

When you provide your own Avro schema, that schema defines the final structure of your data. Any extra values inside any incoming events that are not mapped in the Arvo schema will not be included in the final destination. Data Prepper does not allow the use of the `include_keys` or `exclude_keys` with a custom schema. This is to avoid confusion between a custom Arvo schema and the `include_keys` or `exclude_keys` sink configurations.

Check failure on line 172 in _data-prepper/pipelines/configuration/sinks/s3.md

View workflow job for this annotation

GitHub Actions / style-job

[vale] reported by reviewdog 🐶 [OpenSearch.Spelling] Error: Avro. If you are referencing a setting, variable, format, function, or repository, surround it with tic marks. Raw Output: {"message": "[OpenSearch.Spelling] Error: Avro. If you are referencing a setting, variable, format, function, or repository, surround it with tic marks.", "location": {"path": "_data-prepper/pipelines/configuration/sinks/s3.md", "range": {"start": {"line": 172, "column": 27}}}, "severity": "ERROR"}

Expand Down

0 comments on commit 11a2612

Please sign in to comment.