Skip to content

Commit

Permalink
docs: update README examples (#194)
Browse files Browse the repository at this point in the history
  • Loading branch information
xushiyan committed Nov 22, 2024
1 parent 10d492a commit 90f7498
Showing 1 changed file with 45 additions and 13 deletions.
58 changes: 45 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,29 +62,30 @@ users and projects.
Read a Hudi table into a PyArrow table.

```python
from hudi import HudiTable

hudi_table = HudiTable("/tmp/trips_table")
records = hudi_table.read_snapshot()

from hudi import HudiTableBuilder
import pyarrow as pa
import pyarrow.compute as pc

hudi_table = (
HudiTableBuilder
.from_base_uri("/tmp/trips_table")
.with_option("hoodie.read.as.of.timestamp", "0")
.build()
)
records = hudi_table.read_snapshot(filters=[("city", "=", "san_francisco")])

arrow_table = pa.Table.from_batches(records)
result = arrow_table.select(
["rider", "ts", "fare"]).filter(
pc.field("fare") > 20.0)
result = arrow_table.select(["rider", "city", "ts", "fare"])
print(result)
```

### Rust
### Rust (DataFusion)

<details>
<summary>Add crate hudi with datafusion feature to your application to query a Hudi table.</summary>

```shell
cargo new my_project --bin && cd my_project
cargo add tokio@1 datafusion@39
cargo add tokio@1 datafusion@42
cargo add hudi --features datafusion
```

Expand All @@ -102,9 +103,11 @@ use hudi::HudiDataSource;
#[tokio::main]
async fn main() -> Result<()> {
let ctx = SessionContext::new();
let hudi = HudiDataSource::new("/tmp/trips_table").await?;
let hudi = HudiDataSource::new_with_options(
"/tmp/trips_table",
[("hoodie.read.as.of.timestamp", "20241122010827898")]).await?;
ctx.register_table("trips_table", Arc::new(hudi))?;
let df: DataFrame = ctx.sql("SELECT * from trips_table where fare > 20.0").await?;
let df: DataFrame = ctx.sql("SELECT * from trips_table where city = 'san_francisco'").await?;
df.show().await?;
Ok(())
}
Expand All @@ -116,6 +119,35 @@ Ensure cloud storage credentials are set properly as environment variables, e.g.
Relevant storage environment variables will then be picked up. The target table's base uri with schemes such
as `s3://`, `az://`, or `gs://` will be processed accordingly.

Alternatively, you can pass the storage configuration as options to the `HudiTableBuilder` or `HudiDataSource`.

### Python

```python
from hudi import HudiTableBuilder

hudi_table = (
HudiTableBuilder
.from_base_uri("s3://bucket/trips_table")
.with_option("aws_region", "us-west-2")
.build()
)
```

### Rust (DataFusion)

```rust
use hudi::HudiDataSource;

async fn main() -> Result<()> {
let hudi = HudiDataSource::new_with_options(
"s3://bucket/trips_table",
[("aws_region", "us-west-2")]
).await?;
}

```

## Contributing

Check out the [contributing guide](./CONTRIBUTING.md) for all the details about making contributions to the project.

0 comments on commit 90f7498

Please sign in to comment.