You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
TableConfiguration should be the single point in the kernel where we validate a table's protocol and metadata. This makes it easy to ensure these two actions are mutually compatible. This also allows kernel to draw a clear distinction between raw data (Protocol & Metadata), and validated data in TableConfiguration.
Tasks
Protocol's try_new method performs some validation on the set of reader features and checks that they are supported. Move this validation to TableConfiguration::try_new.
Ensure that the set of reader features is a superset of writer features.
Eagerly parse the reader and writer features so that we do not parse them repeatedly such as in ensure_read_supported.
Add validation for Metadata in the TableConfiguration constructor. Ex: it should check partition columns are present in the schema.
Move any tests that check protocol and metadata validity to instead check the TableConfiguration. Ex: the tests for protocol such as test_ensure_read_supported
Create a framework for testing valid/invalid protocol/metadata configurations.
Migrate integration tests in table_changes/mod.rs that check failed configurations to instead be unit tests for TableConfiguration. This ensures that we cover all edge cases.
The text was updated successfully, but these errors were encountered:
Motivation
TableConfiguration
should be the single point in the kernel where we validate a table's protocol and metadata. This makes it easy to ensure these two actions are mutually compatible. This also allows kernel to draw a clear distinction between raw data (Protocol & Metadata), and validated data inTableConfiguration
.Tasks
try_new
method performs some validation on the set of reader features and checks that they are supported. Move this validation toTableConfiguration::try_new
.ensure_read_supported
.Metadata
in the TableConfiguration constructor. Ex: it should check partition columns are present in the schema.test_ensure_read_supported
table_changes/mod.rs
that check failed configurations to instead be unit tests for TableConfiguration. This ensures that we cover all edge cases.The text was updated successfully, but these errors were encountered: