Best way to use Rego Go lib to process policies with changing data and input

[VasiliyS]

I'm using the Go lib to build a policy checker (an API GW extension) which fetches attributes for an identity (SPIFFE SAN) from an external DB and adds it along with the usual input parameters ( path, method, etc) to the PreparedPartialQuery.

My current implementation does it via submitting the call attributes via EvalParsedInput and identity attributes via a Write transaction to the underlying Store, which gets Aborted at the the end of the eval.

I'm wondering if it's the best way to do it from the memory (heap allocs) and performance standpoint. I guess Store is really meant as a cache of sorts, which is updated but not always gets re-written.
Would it be better to add the attributes part via new function, which will be called from within the module? Or shall I just put in int the Input (it feels somehow wrong)? Any other way to do it better?

Any ideas and shared experience are highly appreciated!

[srenatus]

Would it be better to add the attributes part via new function, which will be called from within the module? Or shall I just put in int the Input (it feels somehow wrong)? Any other way to do it better?

The docs on external data mostly apply to the golang lib use case, too.

It usually boils down to how often your data changes whether input or data is the right place. How large is your data? Unless it's very very large, updating the inmem store for data that doesn't change too frequently is a good approach. It'll do heap allocations, it's in memory after all, but it won't do that for every policy evaluation.

You can think of OPA's store as a workbench: what should be at hand for policy evaluation, but isn't different for each and every input you feed into it, could very well live in the store.

[VasiliyS]

Thank you, @srenatus! You've confirmed my understanding.

In my case I need to change the "data" ( i.e. context for the Policy eval) for every evaluation. So the easiest is to add it to the Input to avoid forking of the inmem store via a write transaction and then discarding it for every eval call (at least I think that's what happens behind the scenes, after looking at the lib's code). Even though it seems to work reasonably well, but I'm concerned that at potentially 1000s calls a minute it will eventually create GC pauses, etc.

I should probably figure out a caching strategy for that context data (it actually shouldn't change that often, but still has to stay consistent with the DB) then Store sounds like a great way of doing it. Or adding a custom function, need to think about it :)