Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Iceberg sync #771

Merged
merged 7 commits into from
Jan 8, 2025
Merged

Iceberg sync #771

merged 7 commits into from
Jan 8, 2025

Conversation

SergeiPatiakin
Copy link
Collaborator

Support Iceberg tables as sync targets

@SergeiPatiakin SergeiPatiakin requested review from gruuya and a team December 31, 2024 13:25
Copy link
Contributor

@gruuya gruuya left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice, especially seeing how it is a smaller change than I anticipated.

src/frontend/flight/handler.rs Outdated Show resolved Hide resolved
@@ -609,3 +609,70 @@ async fn test_sync_custom_store(

Ok(())
}

#[tokio::test]
async fn test_sync_iceberg_custom_store(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like test_sync_custom_store and this test share the setup/results, maybe we can just parametrize the former to do exercise this routine for both formats (we can ditch the local parametrization in it since it won't work for Iceberg)?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The setup, verification and parametrization are all subtly different so I'm not sure it's worth it.

src/sync/planner.rs Outdated Show resolved Hide resolved
@SergeiPatiakin SergeiPatiakin requested a review from gruuya January 6, 2025 15:45
@SergeiPatiakin SergeiPatiakin merged commit c126a02 into main Jan 8, 2025
2 checks passed
@SergeiPatiakin SergeiPatiakin deleted the feat/iceberg-sync branch January 8, 2025 10:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants