Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add overlapping fields (index-alias for index state management) #7611

Closed
wants to merge 1 commit into from
Closed

Add overlapping fields (index-alias for index state management) #7611

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 56 additions & 1 deletion _im-plugin/index-alias.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ POST <index>/_alias/<alias name>
```
{% include copy-curl.html %}

The `<index>` in the above requests can be an index name, a comma-separated list of index names, or a wildcard expression. Use `_all` to refer to all indexes.
The `<index>` in the preceding requests can be an index name, a comma-separated list of index names, or a wildcard expression. Use `_all` to refer to all indexes.

To check if `alias1` refers to `index-1`, run one of the following commands:

Expand Down Expand Up @@ -272,3 +272,58 @@ DELETE index-1/_alias/alias1
{% include copy-curl.html %}

After running the request, `alias1` no longer refers to `index-1` but still refers to `index-2`.

## Overlapping field names
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Technical reviewer: Please confirm this example is relevant to an OpenSearch user and addresses the following user feedback: Undocumented behavior for when two indexes with overlapping field names are associated with the same alias. What happens when these are different types? There appears to be some kind of implicit casting that occurs. I tested the example using Dev Tools. If another example is more appropriate, please replace the draft example with your example. Thank you.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any problem case they provided? I'm not sure what is this implicit casting they are refering to.

Copy link
Contributor Author

@vagimeli vagimeli Aug 30, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quote reply

@bowenlan-amzn This is feedback from a user via the documentation comments. No other info provided. I found this info, but don't know if it's related https://forum.opensearch.org/t/help-with-text-field-mapping-and-implicit-conversion-of-integer-and-boolean-to-text/16226. Should we close this and open a new PR when we have more details?


When querying an alias that points to multiple indexes with overlapping field names of different data types, OpenSearch attempts to merge the data from these indexes. However, due to the conflicting data types, OpenSearch performs implicit casting or type conversion on the field values to reconcile the differences.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
When querying an alias that points to multiple indexes with overlapping field names of different data types, OpenSearch attempts to merge the data from these indexes. However, due to the conflicting data types, OpenSearch performs implicit casting or type conversion on the field values to reconcile the differences.
When querying an alias that points to multiple indexes with overlapping field names of different data types, OpenSearch attempts to merge the data from these indexes.


Here's an example:

```json
PUT /index1
{
"mappings": {
"properties": {
"field1": {
"type": "text"
}
}
}
}

PUT /index2
{
"mappings": {
"properties": {
"field1": {
"type": "long"
}
}
}
}

POST /_aliases
{
"actions": [
{
"add": {
"index": "index1",
"alias": "my_alias"
}
},
{
"add": {
"index": "index2",
"alias": "my_alias"
}
}
]
}
```
{% include copy-curl.html %}

In this example, the two indexes (`index1` and `index2`) have a field named `field1`, but with different data types (`text` and `long`, respectively). The index alias `my_alias` is then created to point to both indexes.

When querying this alias, OpenSearch attempts to merge the data from both indexes. If a document in `index1` has a string value for `field1`, and a document in `index2` has a numeric value for the same field, OpenSearch performs implicit casting to reconcile the data types.

Note that this implicit casting can lead to unexpected results or data loss. For example, if a string value cannot be cast to a numeric type, it may be treated as a missing value or cause an error.
Loading