Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Gh workflow #2

Merged
merged 37 commits into from
Jul 3, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
38cf9c6
Update key value processor documentation with new config options (#7413)
kkondaka Jun 24, 2024
33bb46c
Add version to the warning headers (#7459)
kolchfa-aws Jun 25, 2024
a8ad33f
Adds the max_request_length documentation for appropriate Data Preppe…
dlvenable Jun 25, 2024
37e451b
adding explanation to jwks validation #7367 #20240522 (#7398)
AntonEliatra Jun 25, 2024
f242d11
20240322 Adding absolute path for config.yml (#7380)
leanneeliatra Jun 25, 2024
af780e1
Rename metricsanalytics to prometheusmetrics (#7482)
kolchfa-aws Jun 25, 2024
412b73d
Add a video requirement to a PR template (#7488)
kolchfa-aws Jun 25, 2024
e446f02
Add 2.15 version (#7483)
kolchfa-aws Jun 25, 2024
464a727
Add 2.15 to version history (#7484)
kolchfa-aws Jun 25, 2024
4fbb3e3
Add release notes 2.15 (#7486)
kolchfa-aws Jun 25, 2024
a40fb93
Add new s3 sink documentation for Data Prepper 2.8 (#7163)
graytaylor0 Jun 25, 2024
4fc1c43
Adding information running script in OpenSearch (#7397)
leanneeliatra Jun 26, 2024
4ddc3e7
Fix monitoring predict requests in ML settings (#7512)
kolchfa-aws Jun 26, 2024
962d0ee
Add oidc docker example with keycloak #1566 (#7372)
AntonEliatra Jun 26, 2024
1c3119b
Add General Guidelines for concurrent segment search (#7402)
jed326 Jun 26, 2024
9696bc1
Fixes Update compare-search-results.md (#7534)
hdhalter Jun 26, 2024
74831ed
Drop repeated key/word in inverted index (#7538)
xofbd Jun 27, 2024
bfbcde5
Remove Vale from date processor (#7556)
Naarcha-AWS Jun 27, 2024
a673b24
Add selective download feature to Data Prepper sources section (#6247)
vagimeli Jun 27, 2024
e880d6f
Update prometheusmetrics.md (#7561)
hdhalter Jun 27, 2024
8059035
Update Benchmark index to reflect TOC (#7560)
Naarcha-AWS Jun 28, 2024
2df01a7
Fix broken link to wildcard field type in index.md (#7565)
tomuk5 Jun 28, 2024
d30d86e
Add Render Template API (#7219)
Naarcha-AWS Jul 1, 2024
61fa844
Update docs for new clause count setting (#7391)
harshavamsi Jul 1, 2024
0597163
Document new ingest and search pipeline allowlist settings (#7414)
andrross Jul 1, 2024
4623b64
Add documentation for using rerank and normalization processors toget…
kolchfa-aws Jul 1, 2024
40236a1
Update syntax.md -- where-filter (#7593)
smacrakis Jul 1, 2024
7645ef2
Update syntax.md (#7595)
smacrakis Jul 1, 2024
9b6357f
Update-jwt-docs (#7236)
donggyu04 Jul 2, 2024
c521450
Add more information to Python docs (#7506)
hdhalter Jul 2, 2024
0f889fe
Correct step 2 of custom local models (#7600)
kolchfa-aws Jul 2, 2024
509d8f0
Update options and add more examples for add_entries processor (#7412)
oeyh Jul 2, 2024
9442ac0
Update multiple data sources and add TSVB and Vega viz types (#7229)
vagimeli Jul 2, 2024
09fb9d2
Add geodistance query documentation (#7607)
kolchfa-aws Jul 3, 2024
786c74d
Fix: expand_wildcards in _refresh API. (#7620)
dblock Jul 3, 2024
94a9626
Added documentation for cat?sort and format. (#7619)
dblock Jul 3, 2024
22afcb7
Add PR checklist
kolchfa-aws Jul 3, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,9 @@ _List any issues this PR will resolve, e.g. Closes [...]._
### Version
_List the OpenSearch version to which this PR applies, e.g. 2.14, 2.12--2.14, or all._

### Frontend features
_If you're submitting documentation for an OpenSearch Dashboards feature, add a video that shows how a user will interact with the UI step by step. A voiceover is optional._

### Checklist
- [ ] By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and subject to the [Developers Certificate of Origin](https://github.com/opensearch-project/OpenSearch/blob/main/CONTRIBUTING.md#developer-certificate-of-origin).
For more information on following Developer Certificate of Origin and signing off your commits, please check [here](https://github.com/opensearch-project/OpenSearch/blob/main/CONTRIBUTING.md#developer-certificate-of-origin).
30 changes: 30 additions & 0 deletions .github/workflows/pr-checklist.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
name: PR Checklist

on:
pull_request:
types: [opened]

jobs:
add-checklist-and-assignees:
runs-on: ubuntu-latest

steps:
- name: Comment PR with checklist
uses: peter-evans/create-or-update-comment@v3
with:
token: ${{ secrets.GITHUB_TOKEN }}
issue-number: ${{ github.event.pull_request.number }}
body: |
Thank you for submitting your PR. The PR states are Tech review -> Doc review -> Editorial review. If you're a developer submitting documentation for a feature you implemented, have the documentation reviewed by your team. If you need a tech review, let us know. Here's a checklist of the PR progression:

### PR Checklist
- [x] Tech Review
- [ ] Doc Review
- [ ] Editorial Review

- name: Add assignees to the PR
uses: peter-evans/create-or-update-comment@v3
with:
token: ${{ secrets.GITHUB_TOKEN }}
issue-number: ${{ github.event.pull_request.number }}
assignees: ${{ github.actor }}, kolchfa-aws
1 change: 1 addition & 0 deletions _about/version-history.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ permalink: /version-history/

OpenSearch version | Release highlights | Release date
:--- | :--- | :---
[2.15.0](https://github.com/opensearch-project/opensearch-build/blob/main/release-notes/opensearch-release-notes-2.15.0.md) | Includes parallel ingestion processing, SIMD support for exact search, and the ability to disable doc values for the k-NN field. Adds wildcard and derived field types. Improves performance for single-cardinality aggregations, rolling upgrades to remote-backed clusters, and more metrics for top N queries. For a full list of release highlights, see the Release Notes. | 25 June 2024
[2.14.0](https://github.com/opensearch-project/opensearch-build/blob/main/release-notes/opensearch-release-notes-2.14.0.md) | Includes performance improvements to hybrid search and date histogram queries with multi-range traversal, ML model integration within the Ingest API, semantic cache for LangChain applications, low-level vector query interface for neural sparse queries, and improved k-NN search filtering. Provides an experimental tiered cache feature. For a full list of release highlights, see the Release Notes. | 14 May 2024
[2.13.0](https://github.com/opensearch-project/opensearch-build/blob/main/release-notes/opensearch-release-notes-2.13.0.md) | Makes agents and tools and the OpenSearch Assistant Toolkit generally available. Introduces vector quantization within OpenSearch. Adds LLM guardrails and hybrid search with aggregations. Adds the Bloom filter skipping index for Apache Spark data sources, I/O-based admission control, and the ability to add an alerting cluster that manages all alerting tasks. For a full list of release highlights, see the Release Notes. | 2 April 2024
[2.12.0](https://github.com/opensearch-project/opensearch-build/blob/main/release-notes/opensearch-release-notes-2.12.0.md) | Makes concurrent segment search and conversational search generally available. Provides an experimental OpenSearch Assistant Toolkit, including agents and tools, workflow automation, and OpenSearch Assistant for OpenSearch Dashboards UI. Adds a new match-only text field, query insights to monitor top N queries, and k-NN search on nested fields. For a full list of release highlights, see the Release Notes. | 20 February 2024
Expand Down
126 changes: 122 additions & 4 deletions _api-reference/cat/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,17 +24,54 @@ GET _cat
```
{% include copy-curl.html %}

The response is an ASCII cat (`=^.^=`) and a list of operations:

```
=^.^=
/_cat/allocation
/_cat/segment_replication
/_cat/segment_replication/{index}
/_cat/shards
/_cat/shards/{index}
/_cat/cluster_manager
/_cat/nodes
/_cat/tasks
/_cat/indices
/_cat/indices/{index}
/_cat/segments
/_cat/segments/{index}
/_cat/count
/_cat/count/{index}
/_cat/recovery
/_cat/recovery/{index}
/_cat/health
/_cat/pending_tasks
/_cat/aliases
/_cat/aliases/{alias}
/_cat/thread_pool
/_cat/thread_pool/{thread_pools}
/_cat/plugins
/_cat/fielddata
/_cat/fielddata/{fields}
/_cat/nodeattrs
/_cat/repositories
/_cat/snapshots/{repository}
/_cat/templates
/_cat/pit_segments
/_cat/pit_segments/{pit_id}
```

## Optional query parameters

You can use the following query parameters with any CAT API to filter your results.
The root `_cat` API does not take any parameters, but individual APIs, such as `/_cat/nodes` accept the following query parameters.

Parameter | Description
:--- | :--- |
`v` | Provides verbose output by adding headers to the columns. It also adds some formatting to help align each of the columns together. All examples in this section include the `v` parameter.
`help` | Lists the default and other available headers for a given operation.
`h` | Limits the output to specific headers.
`format` | Returns the result in JSON, YAML, or CBOR formats.
`sort` | Sorts the output by the specified columns.
`format` | The format in which to return the result. Valid values are `json`, `yaml`, `cbor`, and `smile`.
`s` | Sorts the output by the specified columns.

### Query parameter usage examples

Expand All @@ -59,7 +96,6 @@ sample-alias1 sample-index-1 - - - -
Without the verbose parameter, `v`, the response simply returns the alias names:

```

.kibana .kibana_1 - - - -
sample-alias1 sample-index-1 - - - -
```
Expand All @@ -72,6 +108,24 @@ To see all the available headers, use the `help` parameter:
GET _cat/<operation_name>?help
```

For example, to see the available headers for the CAT aliases operation, send the following request:

```json
GET _cat/aliases?help
```
{% include copy-curl.html %}

The response contains the available headers:

```
alias | a | alias name
index | i,idx | index alias points to
filter | f,fi | filter
routing.index | ri,routingIndex | index routing
routing.search | rs,routingSearch | search routing
is_write_index | w,isWriteIndex | write index
```

### Get a subset of headers

To limit the output to a subset of headers, use the `h` parameter:
Expand All @@ -80,7 +134,71 @@ To limit the output to a subset of headers, use the `h` parameter:
GET _cat/<operation_name>?h=<header_name_1>,<header_name_2>&v
```

For example, to limit aliases to only the alias name and index, send the following request:

```json
GET _cat/aliases?h=alias,index
```
{% include copy-curl.html %}

The response contains the requested information:

```
.kibana .kibana_1
sample-alias1 sample-index-1
```

Typically, for any operation you can find out what headers are available using the `help` parameter, and then use the `h` parameter to limit the output to only the headers that you care about.

### Sort by a header

To sort the output by a header, use the `s` parameter:

```json
GET _cat/<operation_name>?s=<header_name_1>,<header_name_2>
```

For example, to sort aliases by alias and then index, send the following request:

```json
GET _cat/aliases?s=i,a
```
{% include copy-curl.html %}

The response contains the requested information:

```
sample-alias2 sample-index-1
sample-alias1 sample-index-2
```

### Retrieve data in JSON format

By default, CAT APIs return data in `text/plain` format.

To retrieve data in JSON format, use the `format=json` parameter:

```json
GET _cat/<operation_name>?format=json
```

For example, to retrieve aliases in JSON format, send the following request:

```json
GET _cat/aliases?format=json
```
{% include copy-curl.html %}

The response contains data in JSON format:

```json
[
{"alias":".kibana","index":".kibana_1","filter":"-","routing.index":"-","routing.search":"-","is_write_index":"-"},
{"alias":"sample-alias-1","index":"sample-index-1","filter":"-","routing.index":"-","routing.search":"-","is_write_index":"-"}
]
```

Other supported formats are [YAML](https://yaml.org/), [CBOR](https://cbor.io/), and [Smile](https://github.com/FasterXML/smile-format-specification).

If you use the Security plugin, make sure you have the appropriate permissions.
{: .note }
34 changes: 34 additions & 0 deletions _api-reference/common-parameters.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,3 +90,37 @@ The following request specifies filters to limit the fields returned in the resp

GET _search?filter_path=<field_name>.*,-<field_name>
```

## Units

OpenSearch APIs support the following units.

### Time units

The following table lists all supported time units.

Units | Specify as
:--- | :---
Days | `d`
Hours | `h`
Minutes | `m`
Seconds | `s`
Milliseconds | `ms`
Microseconds | `micros`
Nanoseconds | `nanos`

### Distance units

The following table lists all supported distance units.

Units | Specify as
:--- | :---
Miles | `mi` or `miles`
Yards | `yd` or `yards`
Feet | `ft` or `feet`
Inches | `in` or `inch`
Kilometers | `km` or `kilometers`
Meters | `m` or `meters`
Centimeters | `cm` or `centimeters`
Millimeters | `mm` or `millimeters`
Nautical miles | `NM`, `nmi`, or `nauticalmiles`
4 changes: 3 additions & 1 deletion _api-reference/document-apis/bulk.md
Original file line number Diff line number Diff line change
Expand Up @@ -124,7 +124,9 @@ All actions support the same metadata: `_index`, `_id`, and `_require_alias`. If
{ "doc" : { "title": "World War Z" }, "doc_as_upsert": true }
```

You can specify a script for more complex document updates:
You can specify a script for more complex document updates by defining the script with the `source` or `id` from a document:



- Script
```json
Expand Down
2 changes: 1 addition & 1 deletion _api-reference/index-apis/refresh.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ The following table lists the available query parameters. All query parameters a
| :--- | :--- | :--- |
| `ignore_unavailable` | Boolean | When `false`, the request returns an error when it targets a missing or closed index. Default is `false`.
| `allow_no_indices` | Boolean | When `false`, the Refresh Index API returns an error when a wildcard expression, index alias, or `_all` targets only closed or missing indexes, even when the request is made against open indexes. Default is `true`. |
| `expand_wildcard` | String | The type of index that the wildcard patterns can match. If the request targets data streams, this argument determines whether the wildcard expressions match any hidden data streams. Supports comma-separated values, such as `open,hidden`. Valid values are `all`, `open`, `closed`, `hidden`, and `none`.
| `expand_wildcards` | String | The type of index that the wildcard patterns can match. If the request targets data streams, this argument determines whether the wildcard expressions match any hidden data streams. Supports comma-separated values, such as `open,hidden`. Valid values are `all`, `open`, `closed`, `hidden`, and `none`.



Expand Down
114 changes: 114 additions & 0 deletions _api-reference/render-template.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
---
layout: default
title: Render Template
nav_order: 82
---

# Render Template

Check failure on line 7 in _api-reference/render-template.md

View workflow job for this annotation

GitHub Actions / vale

[vale] _api-reference/render-template.md#L7

[OpenSearch.HeadingCapitalization] 'Render Template' is a heading and should be in sentence case.
Raw output
{"message": "[OpenSearch.HeadingCapitalization] 'Render Template' is a heading and should be in sentence case.", "location": {"path": "_api-reference/render-template.md", "range": {"start": {"line": 7, "column": 3}}}, "severity": "ERROR"}

The Render Template API renders a [search template]({{site.url}}{{site.baseurl}}/search-plugins/search-template/) as a search query.

## Paths and HTTP methods

```
GET /_render/template
POST /_render/template
GET /_render/template/<id>
POST /_render/template/<id>
```

## Path parameters

The Render Template API supports the following optional path parameter.

| Parameter | Type | Description |
| :--- | :--- | :--- |
| `id` | String | The ID of the search template to render. |

## Request options

The following options are supported in the request body of the Render Template API.

| Parameter | Required | Type | Description |
| :--- | :--- | :--- | :--- |
| `id` | Conditional | String | The ID of the search template to render. Is not required if the ID is provided in the path or if an inline template is specified by the `source`. |
| `params` | No | Object | A list of key-value pairs that replace Mustache variables found in the search template. The key-value pairs must exist in the documents being searched. |
| `source` | Conditional | Object | An inline search template to render if a search template is not specified. Supports the same parameters as a [Search]({{site.url}}{{site.baseurl}}/api-reference/search/) API request and [Mustache](https://mustache.github.io/mustache.5.html) variables. |

## Example request

Both of the following request examples use the search template with the template ID `play_search_template`:

```json
{
"source": {
"query": {
"match": {
"play_name": "{{play_name}}"
}
}
},
"params": {
"play_name": "Henry IV"
}
}
```

### Render template using template ID

The following example request validates a search template with the ID `play_search_template`:

```json
POST _render/template
{
"id": "play_search_template",
"params": {
"play_name": "Henry IV"
}
}
```
{% include copy.html %}

### Render template using `_source`

If you don't want to use a saved template, or want to test a template before saving, you can test a template with the `_source` parameter using [Mustache](https://mustache.github.io/mustache.5.html) variables, as shown in the following example:

```
{
"source": {
"from": "{{from}}{{^from}}10{{/from}}",
"size": "{{size}}{{^size}}10{{/size}}",
"query": {
"match": {
"play_name": "{{play_name}}"
}
}
},
"params": {
"play_name": "Henry IV"
}
}
```
{% include copy.html %}

## Example response

OpenSearch responds with information about the template's output:

```json
{
"template_output": {
"from": "0",
"size": "10",
"query": {
"match": {
"play_name": "Henry IV"
}
}
}
}
```




Loading
Loading