Skip to content

Commit

Permalink
Merge branch 'main' into main
Browse files Browse the repository at this point in the history
  • Loading branch information
mattmartin14 authored Feb 12, 2025
2 parents 245b4a9 + 8fcdc95 commit e3e9611
Show file tree
Hide file tree
Showing 6 changed files with 644 additions and 499 deletions.
5 changes: 5 additions & 0 deletions mkdocs/docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -417,6 +417,11 @@ catalog:
s3.secret-access-key: password
```

| Key | Example | Description |
|------------------------------| ------- | --------------------------------- |
| hive.hive2-compatible | true | Using Hive 2.x compatibility mode |
| hive.kerberos-authentication | true | Using authentication via Kerberos |

When using Hive 2.x, make sure to set the compatibility flag:

```yaml
Expand Down
35 changes: 18 additions & 17 deletions mkdocs/docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,23 +40,24 @@ pip install "pyiceberg[s3fs,hive]"

You can mix and match optional dependencies depending on your needs:

| Key | Description: |
| ------------ | ------------------------------------------------------------------------- |
| hive | Support for the Hive metastore |
| glue | Support for AWS Glue |
| dynamodb | Support for AWS DynamoDB |
| sql-postgres | Support for SQL Catalog backed by Postgresql |
| sql-sqlite | Support for SQL Catalog backed by SQLite |
| pyarrow | PyArrow as a FileIO implementation to interact with the object store |
| pandas | Installs both PyArrow and Pandas |
| duckdb | Installs both PyArrow and DuckDB |
| ray | Installs PyArrow, Pandas, and Ray |
| daft | Installs Daft |
| s3fs | S3FS as a FileIO implementation to interact with the object store |
| adlfs | ADLFS as a FileIO implementation to interact with the object store |
| snappy | Support for snappy Avro compression |
| gcsfs | GCSFS as a FileIO implementation to interact with the object store |
| rest-sigv4 | Support for generating AWS SIGv4 authentication headers for REST Catalogs |
| Key | Description: |
|---------------|---------------------------------------------------------------------------|
| hive | Support for the Hive metastore |
| hive-kerberos | Support for Hive metastore in Kerberos environment |
| glue | Support for AWS Glue |
| dynamodb | Support for AWS DynamoDB |
| sql-postgres | Support for SQL Catalog backed by Postgresql |
| sql-sqlite | Support for SQL Catalog backed by SQLite |
| pyarrow | PyArrow as a FileIO implementation to interact with the object store |
| pandas | Installs both PyArrow and Pandas |
| duckdb | Installs both PyArrow and DuckDB |
| ray | Installs PyArrow, Pandas, and Ray |
| daft | Installs Daft |
| s3fs | S3FS as a FileIO implementation to interact with the object store |
| adlfs | ADLFS as a FileIO implementation to interact with the object store |
| snappy | Support for snappy Avro compression |
| gcsfs | GCSFS as a FileIO implementation to interact with the object store |
| rest-sigv4 | Support for generating AWS SIGv4 authentication headers for REST Catalogs |

You either need to install `s3fs`, `adlfs`, `gcsfs`, or `pyarrow` to be able to fetch files from an object store.

Expand Down
Loading

0 comments on commit e3e9611

Please sign in to comment.