v0.50.0

This release contains a mix of new features, bugfixes, security fixes, optimizations and build updates related to
OPA's published images.

New Built-in Functions: JSON Schema Verification and Validation

These new built-in functions add functionality to verify and validate JSON Schema (#5486) (co-authored by @jkulvich and @johanfylling).

• json.verify_schema: Checks that the input is a valid JSON schema object
• json.match_schema: Checks that the document matches the JSON schema

See the documentation for all details.

Annotations scoped to package carries across modules

package scoped schema annotations are now applied across modules instead of only local to the module where
it's declared (#5251) (authored by @johanfylling). This change may cause compile-time errors and behavioural changes to
type checking when the schemas annotation is used, and to rules calling the rego.metadata.chain() built-in function:

• Existing projects with the same package declared in multiple files will trigger a rego_type_error: package annotation redeclared
  error if two or more of these are annotated with the package scope.
• If using the package scope, the schemas annotation will be applied to type checking also for rules declared in
  another file than the annotation declaration, as long as the package is the same.
• The chain of metadata returned by the rego.metadata.chain() built-in function will now contain an entry for the
  package even if the annotations are declared in another file, if the scope is package.

Remote bundle URL shorthand for run command

To load a remote bundle using opa run, the set directive can be provided multiple times as shown below:

 $ opa run -s --set "services.default.url=https://example.com" \
              --set "bundles.example.service=default" \
              --set "bundles.example.resource=/bundles/bundle.tar.gz" \
              --set "bundles.example.persist=true"

The following command can be used as a shorthand to easily start OPA with a remote bundle (#5674) (authored by @anderseknert):

$ opa run -s https://example.com/bundles/bundle.tar.gz

Performance Improvements for json.patch Built-in Function

Performance improvements in json.patch were achieved with the introduction of a new EditTree data structure,
which is built for applying in-place modifications to an ast.Term, and can render the final result of all edits efficiently
by applying all patches in a JSON-Patch sequence rapidly, and then collapsing all edits at the end with minimal wasted ast.Term copying (authored by @philipaconrad).
For more details and benchmarks refer #5494 and #5390.

Surface decision log errors via status API

Errors encountered during decision log uploads will now be surfaced via the Status API in addition to being logged. This
functionality should give users greater visibility into any issues OPA may face while processing, uploading logs etc (#5637) (authored by @ashutosh-narkar).

See the documentation for more details.

OPA Published Images Update

All published OPA images now run with a non-root uid/gid. The uid:gid is set to 1000:1000 for all images. As a result
there is no longer a need for the -rootless image variant and hence it will be not be published as part of future releases.
This change is in line with container security best practices. OPA can still be run with root privileges by explicitly setting the user,
either with the --user argument for docker run, or by specifying the securityContext in the Kubernetes Pod specification.

Runtime, Tooling, SDK

• server: Support compression of response payloads if HTTP client supports it (#5310) authored by @AdrianArnautu
• bundle: Ensure the bundle resulting from merging a set of bundles does not contain nil data (#5703) authored by @anderseknert
• repl: Use lowercase for repl commands only and keep any provided arguments as-is (#5229) authored by @Trolloldem
• metrics: New endpoint /metrics/alloc_bytes to show OPA's memory utilization (#5715) authored by @anderseknert
• server: When using OPA TLS authorization, authz policy authors will now have access to the client certificates
  presented as part of the TLS connection. This new data will be available under the key client_certificates (#5538) authored by @charlieegan3
• server: Use streaming implementation of json.Decode rather than using an intermediate buffer for the incoming request (#5661) authored by @anderseknert

Topdown and Rego

• ast: Extend compiler strict mode check to include unused arguments (#5602) authored by @boranx. This change may cause
  compile-time errors for policies that have unused arguments in the scope when the strict mode is enabled. These
  variables could be replaced with _ (wildcard) or get cleaned up if they are not intended to be used in the body of the functions.
• ast: Respect inlined schemas annotations even if --schema flag isn't used (#5506) authored by @johanfylling
• ast: Force type-checker to respect allow_net capability when fetching remote schemas (#5670) authored by @johanfylling
• ast/parse: Provide custom parsing options that allow location information of AST nodes to be included in their JSON
  representation. This location information can be used by tools that work with the OPA AST (#3143) authored by @charlieegan3

Docs

• docs/policy-reference: Fix typo in policy reference doc (#5654) authored by @alvarogomez93
• docs/extensions: Fix sample code provided in the custom built-in implementation example (#5666) authored by @Ronnie-personal
• docs/bundles: Clarify delta bundle behavior when it contains an empty list of patch operations (#5629) authored by @charlieegan3
• docs/http-api-authz: Update the HTTP API authz tutorial with steps related to proper bundle creation (#5682) authored by @lamoboos223
• Fix broken 'future keywords' url link (#5686) authored by @neelanjan00

Website + Ecosystem

• Ecosystem:

  • Styra Load (#5659) authored by @charlieegan3

• Website:

  • Update OPA documentation search to use Algolia v3 (#5706) authored by @Parsifal-M
  • Drop Google Universal Analytics (UA) code as part of Google Analytics 4 migration (authored by @chalin)

Miscellaneous

• Dependency bumps, notably:
  • golang from 1.20.1 to 1.20.2
  • github.com/containerd/containerd from 1.6.16 to 1.6.19
  • github.com/golang/protobuf from 1.5.2 to 1.5.3
  • golang.org/x/net from 0.5.0 to 0.8.0
  • google.golang.org/grpc from 1.52.3 to 1.53.0
  • OpenTelemetry-related dependencies (#5701)

v0.49.2

This release migrates the ORAS Go library from v1.2.2 to v2.
The earlier version of the library had a dependency on the docker
package. That version of the docker package had some reported vulnerabilities such as
CVE-2022-41716, CVE-2022-41720. The ORAS Go library v2 removes the dependency on the docker package.

v0.49.1

This is a bug fix release addressing the following Golang security issues:

Golang security fix CVE-2022-41723

A maliciously crafted HTTP/2 stream could cause excessive CPU consumption in the HPACK decoder, sufficient to cause a
denial of service from a small number of small requests.

Golang security fix CVE-2022-41724

Large handshake records may cause panics in crypto/tls. Both clients and servers may send large TLS handshake records
which cause servers and clients, respectively, to panic when attempting to construct responses.

Golang security fix CVE-2022-41722

A path traversal vulnerability exists in filepath.Clean on Windows. On Windows, the filepath.Clean function could
transform an invalid path such as "a/../c:/b" into the valid path "c:�". This transformation of a relative
(if invalid) path into an absolute path could enable a directory traversal attack.
After fix, the filepath.Clean function transforms this path into the relative (but still invalid) path ".

v0.49.0

This release focuses on bugfixes and documentation improvements, as well as a few small performance improvements.

Runtime, Tooling, SDK

• runtime: Update rule index's trie node scalar handling so that numerics compare correctly (#5585) authored by @ashutosh-narkar reported by @alvarogomez93
• ast: Improve error information when metadata yaml fails to compile (#4475) authored and reported by @johanfylling
• bundle: Retain metadata annotations for Wasm entrypoints during inspection (#5588) authored and reported by @johanfylling
• compile: Allow object generating rules to be annotated as entrypoints (#5577) authored and reported by @johanfylling
• plugins/discovery: Support for persisting and loading discovery bundle from disk (#2886) authored by @ashutosh-narkar reported by @anderseknert
• perf: Use json.Encode to avoid extra allocation (authored by @anderseknert)
• opa inspect: Fix prefix error when inspecting bundle from root (#5503) authored by @harikannan512 reported by @HarshPathakhp
• topdown: http.send to cache responses based on status code (#5617) authored by @ashutosh-narkar
• types: Add GoDoc about named types (authored by @wata727)
• deps: Remove github.com/pkg/errors dependency (authored by @Iceber)

Docs

• Update entrypoint documentation (#5565) authored by @johanfylling reported by @robertgartman
• Add missing folder argument in bundle build example (authored by @charlieegan3)
• Clarify crypto.x509.parse_certificates docs (authored by @charlieegan3)
• Added AWS S3 Web Identity Credentials info to tutorial (authored by @vishrana)
• docs/graphql: non-nullable id argument and typo fix (authored by @philipaconrad)

Website + Ecosystem

• Ecosystem:

  • ccbr (authored by @niuzhi)

• Website:

  • Show prominent warning when viewing old docs (authored by @charlieegan3)
  • Prevent navbar clipping on narrow screens + sticky nav (authored by @charlieegan3)

Miscellaneous

Dependency bumps:

• build: bump golang 1.19.4 -> 1.19.5 (authored by @yanggangtony)
• ci: aquasecurity/trivy-action from 0.8.0 to 0.9.0
• github.com/containerd/containerd from 1.6.15 to 1.6.16
• google.golang.org/grpc from 1.51.0 to 1.52.3

v0.48.0

This release rolls in security fixes from recent patch releases, along with a number of bugfixes, and a new builtin function.

Improved error reporting available in opa eval

A common frustration when writing policies in OPA is when an error happens, causing a rule to unexpectedly return undefined. Using --strict-builtin-errors would allow finding the first error encountered during evaluation, but terminates execution immediately.

To improve the debugging experience, it is now possible to display all of the errors encountered during normal evaluation of a policy, via the new --show-builtin-errors option.

Consider the following error-filled policy, multi-error.rego:

package play

this_errors(number) := result {
        result := number / 0
}

this_errors_too(number) := result {
        result := number / 0
}

res1 := this_errors(1)

res2 := this_errors_too(1)

Using --strict-builtin-errors, we would only see the first divide by zero error:

opa eval --strict-builtin-errors -d multi-error.rego data.play

1 error occurred: multi-error.rego:4: eval_builtin_error: div: divide by zero

Using --show-builtin-errors shows both divide by zero issues though:

opa eval --show-builtin-errors -d multi-error.rego data.play -f pretty

2 errors occurred:
multi-error.rego:4: eval_builtin_error: div: divide by zero
multi-error.rego:8: eval_builtin_error: div: divide by zero

By showing more errors up front, we hope this will improve the overall policy writing experience.

New Built-in Function: time.format

It is now possible to format a time value from nanoseconds to a formatted timestamp string via a built-in function. The builtin accepts 3 argument formats, each allowing for different options:

1. A number representing the nanoseconds since the epoch (UTC).
2. A two-element array of the nanoseconds, and a timezone string.
3. A three-element array of nanoseconds, timezone string, and a layout string (same format as for time.parse_ns).

See the documentation for all details.

Implemented by @burnerlee.

Optimization in rule indexing

Previously, every time the evaluator looked up a rule in the index, OPA performed checks for grounded refs over the entire index before looking up the rule.

Now, OPA performs all groundedness checks once at index construction time, which keeps index lookup times much more consistent as the number of indexed rules scales up.

Policies with large numbers of index-ready rules can expect a small performance lift, proportional to the number of indexed rules.

Bundle fetching with AWS Signing Version 4A

AWS has recently developed an extension to SigV4 called Signature Version 4A (SigV4A) which enables signatures that are valid in more than one AWS Region. This new signature method is required for signing multi-region API requests, such as Amazon S3 Multi-Region Access Points (MRAP).

OPA now supports this new request signing method for bundle fetching, which means that you can use an S3 MRAP as a bundle source. This is configured via the new services[<your_service_name>].credentials.s3_signing.signature_version field.

See the the documentation for more details.

Implemented by @jwineinger

Runtime

• rego: Check store modules before skipping parsing (authored by @charlieegan3)
• topdown/rego: Add BuiltinErrorList support to rego package, add to eval command (authored by @charlieegan3)
• topdown: Fix evaluator's re-wrapping of NDBCache errors (authored by @srenatus)
• Fix potential memory leak from http.send in interquery cache (authored by @asleire)
• ast/parser: Detect function rule head + contains keyword (#5525) authored and reported by @philipaconrad
• ast/visit: Add SomeDecl to visitor walks (#5480) authored by @srenatus
• ast/visit: Include LazyObject in visitor walks (#5479) authored by @srenatus reported by @benweint

Tooling, SDK

• topdown: cache undefined rule evaluations (#593) authored by @edpaget reported by @tsdandall
• topdown: Specify host verification policy for http redirects (#5388) authored and reported by @ashutosh-narkar
• providers/aws: Refactor + Fix 2x Authorization header append issue (#5472) authored by @philipaconrad reported by @Hiieu
• Add support to enable ND builtin cache via discovery (#5457) authored by @ashutosh-narkar reported by @asadali
• format: Only use ref heads for all rule heads if necessary (#5449) authored and reported by @srenatus
• opa inspect: Fix path of data namespaces on windows (authored by @shm12)
• ast+cmd: Only enforcing schemas annotations if --schema flag is used (authored by @johanfylling)
• sdk: Allow use of a query tracer (authored by @charlieegan3)
• sdk: Allow use of metrics, profilers, and instrumentation (authored by @charlieegan3)
• sdk: Return provenance information in Result types (authored by @charlieegan3)
• sdk: Allow use of StrictBuiltinErrors (authored by @charlieegan3)
• Allow print calls in IR (authored by @anderseknert)
• tester/runner: Fix panic'ing case in utility function (#5496) authored and reported by @philipaconrad

Docs

• Community page updates (authored by @anderseknert)
• Update Hugo version, update deprecated Page fields (authored by @charlieegan3)
• docs: Update TLS-based Authentication Example (#5521) authored by @charlieegan3 reported by @jjthom87
• docs: Update opa eval flags to link to bundle docs (authored by @charlieegan3)
• docs: Make SDK first option for Go integraton (authored by @anderseknert)
• docs: Fix typo on Policy Language page. (authored by @mcdonagj)
• docs/integrations: Update kubescape repo links (authored by @dwertent)
• docs/oci: Corrected config section (authored by @ogazitt)
• website/frontpage: Update Learn More links (authored by @pauly4it)
• integrations.yaml: Ensure inventors listed in organizations (authored by @anderseknert)
• integrations: Fix malformed inventors item (authored by @anderseknert)
• Add Digraph to ADOPTERS.md (authored by @jamesphlewis)

Miscellaneous

• Remove changelog maintainer mention filter (authored by @anderseknert)
• Chore: Fix len check in the ast/visit_test error message (authored by @boranx)
• opa inspect: Fix wrong windows bundle tar files path separator (authored by @shm12)
• Add CHANGELOG.md to website build triggers (authored by @srenatus)

Dependency bumps:

• Golang 1.19.3 -> 1.19.4
• github.com/containerd/containerd from 1.6.10 -> 1.6.15
• github.com/dgraph-io/badger/v3
• golang.org/x/net to 0.5.0
• json5 and postcss-modules
• oras.land/oras-go from 1.2.1 -> 1.2.2

CI/Distribution fixes:

• Update base images for non debug builds (authored by @charlieegan3)
• Remove deprecated linters in golangci config (authored by @yanggangtony)

v0.47.4

This is a bug fix release addressing a panic in opa test.

• tester/runner: Fix panic'ing case in utility function. (#5496)

v0.47.3

This is a bug fix release addressing an issue that prevented OPA from fetching bundles stored in S3 buckets.

• providers/aws: Refactor + fix 2x Authorization header append issue. (#5472)

v0.47.2

This is a second security fix to address CVE-2022-41717/GO-2022-1144.

We previously believed that upgrading the Golang version and its stdlib would be sufficient
to address the problem. It turns out we also need to bump the x/net dependency to v0.4.0.,
a version that hadn't existed when v0.46.2 was released.

This release bumps the golang.org/x/net dependency to v0.4.0, and contains no other
changes over v0.46.2.

Note that the affected code is OPA's HTTP server. So if you're using OPA as a Golang library,
or if your confident that your OPA's HTTP interface is protected by other means (as it should
be -- not exposed to the public internet), you're OK.

v0.46.3

This is a second security fix to address CVE-2022-41717/GO-2022-1144.

We previously believed that upgrading the Golang version and its stdlib would be sufficient
to address the problem. It turns out we also need to bump the x/net dependency to v0.4.0.,
a version that hadn't existed when v0.46.2 was released.

This release bumps the golang.org/x/net dependency to v0.4.0, and contains no other
changes over v0.46.2.

Note that the affected code is OPA's HTTP server. So if you're using OPA as a Golang library,
or if your confident that your OPA's HTTP interface is protected by other means (as it should
be -- not exposed to the public internet), you're OK.

v0.47.1

This is a bug fix release addressing two issues: one security issue, and one bug
related to formatting backwards-compatibility.

Golang security fix CVE-2022-41717

An attacker can cause excessive memory growth in a Go server accepting HTTP/2 requests.

Since we advise against running an OPA service exposed to the general public of the
internet, potential attackers would be limited to people that are already capable of
sending direct requests to the OPA service.

opa fmt and backwards compatibility (#5449)

In v0.46.1, it was possible that opa fmt would format a rule in such a way that:

1. Before formatting, it was working fine with older OPA versions, and
2. after formatting, it would only work with OPA version >= 0.46.1.

This backwards incompatibility wasn't intended, and has now been fixed.