incremental data push to OPA

[vimalagrawal]

Not able to find any documentation or example to understand how I can push incremental data ( only the changes) to OPA. It is not possible to load whole data for smaller changes for us. Can someone please guide me.

[anderseknert]

Hi @vimalagrawal! Did you see the REST API documentation for the /v1/data endpoint? It's over here.

[vimalagrawal]

Hi @anderseknert,
I continue to learn how to deal with data in OPA. Currently I am looking at wasmer as solution to integrate opa as a library in C program. I am using opa_eval() api to provide data pointer to OPA but I would like to use different data files at different nodes like data/1.json and data/2.json containing different data ( I could do that with opa in server mode). How do I provide multiple data files and multiple paths to OPA using opa_eval or any other api? I see one another api in OPA documentation i.e. opa_value_add_path() but I am not sure if I understood it correctly and I don't see example of usage of some apis in any documentation. I will appreciate any help.

[anderseknert]

@srenatus might be able to provide some pointers. I've not worked much with Wasm myself, I'm afraid.

[srenatus]

Yeah you can manage a wasm instance's data. OPA itself does it, too, when you load a wasm bundle into the OPA server, so have a look at these methods: https://github.com/open-policy-agent/opa/blob/8be5b3cb0ebd3f639db69d4378dcd0e0c689fb29/internal/wasm/sdk/internal/wasm/vm.go#L520-L567. I'm afraid this isn't well-documented, and it's a feature few people use. (Also, those methods aren't exported through npm-opa-wasm, but since you're wrapping the wasm module yourself with wasmer, that's probably not your concern.)

[vimalagrawal]

I still can't make out how to use it outside OPA source from C application.
exported function is :
opa_errc opa_value_add_path(opa_value *data, opa_value *path, opa_value *v)

how to get this *data that I can pass to this api. is there any api to get *data from OPA?
basically I am understanding that this api might help me to change some specific record in data on the fly (without reloading whole data).

[srenatus]

I haven't used it in a while but IIRC *data is your current heap pointer.