Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Sub-fields naming collision #506

Closed
A-Gray-Cat opened this issue Aug 1, 2024 · 2 comments
Closed

[BUG] Sub-fields naming collision #506

A-Gray-Cat opened this issue Aug 1, 2024 · 2 comments
Assignees
Labels
bug Something isn't working Lang:PPL Pipe Processing Language support

Comments

@A-Gray-Cat
Copy link

A-Gray-Cat commented Aug 1, 2024

What is the bug?
For example, if a log line contains two sub-fields that have the same name, direct query would ignore the top level fields and think both fields are duplicated. For example, the sql query below will return one field ip, with the top level fields stripped. The PPL query won't return any results and won't raise errors either.

SELECT
     src_endpoint.ip,
     dst_endpoint.ip 
FROM
securitylake.amazon_security_lake_glue_db_us_east_1.amazon_security_lake_table_us_east_1_vpc_flow_2_0
WHERE time_dt BETWEEN CURRENT_TIMESTAMP - INTERVAL '1' DAY AND CURRENT_TIMESTAMP
LIMIT 10



source = securitylake.amazon_security_lake_glue_db_us_east_1.amazon_security_lake_table_us_east_1_vpc_flow_2_0
| where time_dt > '2024-07-31 00:00:00 ' and region = 'us-east-1'
| fields src_endpoint.ip
| head 10 

How can one reproduce the bug?
Steps to reproduce the behavior:

  1. Go to Data sources -> securitylake -> query data -> open log explorer
  2. Run the follow queries:
SELECT
     src_endpoint.ip,
     dst_endpoint.ip 
FROM
securitylake.amazon_security_lake_glue_db_us_east_1.amazon_security_lake_table_us_east_1_vpc_flow_2_0
WHERE time_dt BETWEEN CURRENT_TIMESTAMP - INTERVAL '1' DAY AND CURRENT_TIMESTAMP
LIMIT 10



source = securitylake.amazon_security_lake_glue_db_us_east_1.amazon_security_lake_table_us_east_1_vpc_flow_2_0
| where time_dt > '2024-07-31 00:00:00 ' and region = 'us-east-1'
| fields src_endpoint.ip
| head 10
  1. Check the returned results

What is the expected behavior?
Both fields are returned properly without the top fields stripped.

What is your host/environment?

  • OS: [e.g. iOS]
  • Version 2.13
  • Plugins

Do you have any screenshots?
If applicable, add screenshots to help explain your problem.

Do you have any additional context?
Add any other context about the problem.

@A-Gray-Cat A-Gray-Cat added bug Something isn't working untriaged labels Aug 1, 2024
@engechas
Copy link
Contributor

engechas commented Aug 2, 2024

Related issue: opensearch-project/observability#1854

@YANG-DB YANG-DB added Lang:PPL Pipe Processing Language support and removed untriaged labels Aug 12, 2024
@YANG-DB YANG-DB moved this to Todo in PPL Commands Aug 12, 2024
@YANG-DB
Copy link
Member

YANG-DB commented Aug 12, 2024

it may be a possible issue in resolveField method

@YANG-DB YANG-DB self-assigned this Aug 12, 2024
@YANG-DB YANG-DB moved this from Todo to In Progress in PPL Commands Aug 12, 2024
@YANG-DB YANG-DB moved this from In Progress to InReview in PPL Commands Aug 13, 2024
@YANG-DB YANG-DB moved this from InReview to Done in PPL Commands Aug 20, 2024
@YANG-DB YANG-DB closed this as completed by moving to Done in PPL Commands Aug 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working Lang:PPL Pipe Processing Language support
Projects
Status: Done
Development

No branches or pull requests

3 participants