Skip to content

Commit

Permalink
Merge branch 'main' into rebwill/add-run-instructions-to-enos-scenarios
Browse files Browse the repository at this point in the history
  • Loading branch information
rebwill authored Jan 10, 2025
2 parents 44dce13 + bcd89f8 commit b33ed62
Show file tree
Hide file tree
Showing 74 changed files with 1,953 additions and 373 deletions.
2 changes: 1 addition & 1 deletion builtin/logical/pki/metadata.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 3 additions & 0 deletions changelog/29097.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
```release-note:bug
sdk/database: Fix a bug where slow database connections can cause goroutines to be blocked.
```
3 changes: 3 additions & 0 deletions changelog/29237.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
```release-note:improvement
core: Add activation flags. A mechanism for users to opt in to new functionality at a convenient time. Previously used only in Enterprise for SecretSync, activation flags are now available in CE for future features to use.
```
3 changes: 3 additions & 0 deletions changelog/29303.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
```release-note:change
core (enterprise): Add tracking of performance standbys by their HA node ID so that RPC connections can be more easily cleaned up when nodes are removed.
```
3 changes: 3 additions & 0 deletions changelog/29306.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
```release-note:improvement
raft/autopilot: We've updated the autopilot reconciliation logic (by updating the raft-autopilot dependency to v0.3.0) to avoid artificially increasing the quorum in presence of an unhealthy node. Now autopilot will start the reconciliation process by attempting to demote a failed voter node before any promotions, fixing the issue where Vault would initially increase quorum when faced with a failure of a voter node. In certain configurations, especially when using Vault Enterprise Redundancy Zones and losing a voter then a non-voter in quick succession, this would lead to a loss of quorum and cluster failure.
```
4 changes: 4 additions & 0 deletions changelog/29325.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
```release-note:improvement
identity: Added reporting in Vault logs during unseal to help identify any
duplicate identify resources in storage.
```
3 changes: 3 additions & 0 deletions changelog/29334.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
```release-note:bug
plugins: Fix a bug that causes zombie dbus-daemon processes on certain systems.
```
4 changes: 1 addition & 3 deletions command/commands.go
Original file line number Diff line number Diff line change
Expand Up @@ -551,9 +551,7 @@ func initCommands(ui, serverCmdUi cli.Ui, runOpts *RunOptions) map[string]cli.Co
}, nil
},
"plugin register": func() (cli.Command, error) {
return &PluginRegisterCommand{
BaseCommand: getBaseCommand(),
}, nil
return NewPluginRegisterCommand(getBaseCommand()), nil
},
"plugin reload": func() (cli.Command, error) {
return &PluginReloadCommand{
Expand Down
14 changes: 14 additions & 0 deletions command/plugin_register_stubs_oss.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
// Copyright (c) HashiCorp, Inc.
// SPDX-License-Identifier: BUSL-1.1

//go:build !enterprise

package command

import "github.com/hashicorp/cli"

func NewPluginRegisterCommand(baseCommand *BaseCommand) cli.Command {
return &PluginRegisterCommand{
BaseCommand: baseCommand,
}
}
9 changes: 7 additions & 2 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,13 @@ replace github.com/hashicorp/vault/api/auth/userpass => ./api/auth/userpass

replace github.com/hashicorp/vault/sdk => ./sdk

// The keyring library has an outstanding bug that causes zombie dbus-daemon
// processes on each execution. Vault has an indirect dependency on keyring via
// gosnowflake. This replace is a stopgap measure until either gosnowflake or
// keyring addresses the issue.
// See https://github.com/99designs/keyring/issues/103 and https://github.com/snowflakedb/gosnowflake/issues/1183
replace github.com/99designs/keyring => github.com/Jeffail/keyring v1.2.3

require (
cloud.google.com/go/cloudsqlconn v1.4.3
cloud.google.com/go/monitoring v1.21.2
Expand Down Expand Up @@ -387,7 +394,6 @@ require (
github.com/go-openapi/validate v0.24.0 // indirect
github.com/go-ozzo/ozzo-validation v3.6.0+incompatible // indirect
github.com/goccy/go-json v0.10.2 // indirect
github.com/godbus/dbus v0.0.0-20190726142602-4481cbc300e2 // indirect
github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang-jwt/jwt/v5 v5.2.1 // indirect
github.com/golang-sql/civil v0.0.0-20220223132316-b832511892a9 // indirect
Expand All @@ -406,7 +412,6 @@ require (
github.com/gophercloud/gophercloud v0.1.0 // indirect
github.com/gorilla/websocket v1.5.1 // indirect
github.com/grpc-ecosystem/go-grpc-middleware v1.4.0 // indirect
github.com/gsterjov/go-libsecret v0.0.0-20161001094733-a6f4afe4910c // indirect
github.com/hailocab/go-hostpool v0.0.0-20160125115350-e80d13ce29ed // indirect
github.com/hashicorp/cronexpr v1.1.2 // indirect
github.com/hashicorp/go-immutable-radix v1.3.1 // indirect
Expand Down
8 changes: 2 additions & 6 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -626,8 +626,6 @@ gioui.org v0.0.0-20210308172011-57750fc8a0a6/go.mod h1:RSH6KIUZ0p2xy5zHDxgAM4zum
git.sr.ht/~sbinet/gg v0.3.1/go.mod h1:KGYtlADtqsqANL9ueOFkWymvzUvLMQllU5Ixo+8v3pc=
github.com/99designs/go-keychain v0.0.0-20191008050251-8e49817e8af4 h1:/vQbFIOMbk2FiG/kXiLl8BRyzTWDw7gX/Hz7Dd5eDMs=
github.com/99designs/go-keychain v0.0.0-20191008050251-8e49817e8af4/go.mod h1:hN7oaIRCjzsZ2dE+yG5k+rsdt3qcwykqK6HVGcKwsw4=
github.com/99designs/keyring v1.2.2 h1:pZd3neh/EmUzWONb35LxQfvuY7kiSXAq3HQd97+XBn0=
github.com/99designs/keyring v1.2.2/go.mod h1:wes/FrByc8j7lFOAGLGSNEg8f/PaI3cgTBqhFkHUrPk=
github.com/AdaLogics/go-fuzz-headers v0.0.0-20230811130428-ced1acdcaa24 h1:bvDV9vkmnHYOMsOr4WLk+Vo07yKIzd94sVoIqshQ4bU=
github.com/AdaLogics/go-fuzz-headers v0.0.0-20230811130428-ced1acdcaa24/go.mod h1:8o94RPi1/7XTJvwPpRSzSUedZrtlirdB3r9Z20bi2f8=
github.com/Azure/azure-pipeline-go v0.2.3 h1:7U9HBg1JFK3jHl5qmo4CTZKFTVgMwdFHMVtCdfBE21U=
Expand Down Expand Up @@ -730,6 +728,8 @@ github.com/GoogleCloudPlatform/opentelemetry-operations-go/detectors/gcp v1.24.1
github.com/HdrHistogram/hdrhistogram-go v1.1.2/go.mod h1:yDgFjdqOqDEKOvasDdhWNXYg9BVp4O+o5f6V/ehm6Oo=
github.com/Jeffail/gabs/v2 v2.1.0 h1:6dV9GGOjoQgzWTQEltZPXlJdFloxvIq7DwqgxMCbq30=
github.com/Jeffail/gabs/v2 v2.1.0/go.mod h1:xCn81vdHKxFUuWWAaD5jCTQDNPBMh5pPs9IJ+NcziBI=
github.com/Jeffail/keyring v1.2.3 h1:WRmYdGPmHoJqX66KjGXQBALp6mUN00tD0ds5C4pqEsQ=
github.com/Jeffail/keyring v1.2.3/go.mod h1:xIg4RDmDwDuUFoU4IzDIT3b+HV24JUYlzo6ILZUH3Sc=
github.com/JohnCGriffin/overflow v0.0.0-20211019200055-46fa312c352c h1:RGWPOewvKIROun94nF7v2cua9qP+thov/7M50KEoeSU=
github.com/JohnCGriffin/overflow v0.0.0-20211019200055-46fa312c352c/go.mod h1:X0CRv0ky0k6m906ixxpzmDRLvX58TFUKS2eePweuyxk=
github.com/Masterminds/goutils v1.1.1 h1:5nUrii3FMTL5diU80unEVvNevw1nH4+ZV4DSLVJLSYI=
Expand Down Expand Up @@ -1174,8 +1174,6 @@ github.com/goccy/go-json v0.10.2 h1:CrxCmQqYDkv1z7lO7Wbh2HN93uovUHgrECaO5ZrCXAU=
github.com/goccy/go-json v0.10.2/go.mod h1:6MelG93GURQebXPDq3khkgXZkazVtN9CRI+MGFi0w8I=
github.com/gocql/gocql v1.0.0 h1:UnbTERpP72VZ/viKE1Q1gPtmLvyTZTvuAstvSRydw/c=
github.com/gocql/gocql v1.0.0/go.mod h1:3gM2c4D3AnkISwBxGnMMsS8Oy4y2lhbPRsH4xnJrHG8=
github.com/godbus/dbus v0.0.0-20190726142602-4481cbc300e2 h1:ZpnhV/YsD2/4cESfV5+Hoeu/iUR3ruzNvZ+yQfO03a0=
github.com/godbus/dbus v0.0.0-20190726142602-4481cbc300e2/go.mod h1:bBOAhwG1umN6/6ZUMtDFBMQR8jRg9O75tm9K00oMsK4=
github.com/godbus/dbus/v5 v5.0.4/go.mod h1:xhWf0FNVPg57R7Z0UbKHbJfkEywrmjJnf7w5xrFpKfA=
github.com/gofrs/flock v0.8.1 h1:+gYjHKf32LDeiEEFhQaotPbLuUXjY5ZqxKgXy7n59aw=
github.com/gofrs/flock v0.8.1/go.mod h1:F1TvTiK9OcQqauNUHlbJvyl9Qa1QvF/gOUDKA14jxHU=
Expand Down Expand Up @@ -1370,8 +1368,6 @@ github.com/grpc-ecosystem/grpc-gateway/v2 v2.7.0/go.mod h1:hgWBS7lorOAVIJEQMi4Zs
github.com/grpc-ecosystem/grpc-gateway/v2 v2.11.3/go.mod h1:o//XUCC/F+yRGJoPO/VU0GSB0f8Nhgmxx0VIRUvaC0w=
github.com/grpc-ecosystem/grpc-gateway/v2 v2.19.1 h1:/c3QmbOGMGTOumP2iT/rCwB7b0QDGLKzqOmktBjT+Is=
github.com/grpc-ecosystem/grpc-gateway/v2 v2.19.1/go.mod h1:5SN9VR2LTsRFsrEC6FHgRbTWrTHu6tqPeKxEQv15giM=
github.com/gsterjov/go-libsecret v0.0.0-20161001094733-a6f4afe4910c h1:6rhixN/i8ZofjG1Y75iExal34USq5p+wiN1tpie8IrU=
github.com/gsterjov/go-libsecret v0.0.0-20161001094733-a6f4afe4910c/go.mod h1:NMPJylDgVpX0MLRlPy15sqSwOFv/U1GZ2m21JhFfek0=
github.com/hailocab/go-hostpool v0.0.0-20160125115350-e80d13ce29ed h1:5upAirOpQc1Q53c0bnx2ufif5kANL7bfZWcc6VJWJd8=
github.com/hailocab/go-hostpool v0.0.0-20160125115350-e80d13ce29ed/go.mod h1:tMWxXQ9wFIaZeTI9F+hmhFiGpFmhOHzyShyFUhRm0H4=
github.com/hashicorp/cap v0.7.0 h1:atLIEU5lJslYXo1qsv7RtUL1HrJVVxnfkErIT3uxLp0=
Expand Down
187 changes: 187 additions & 0 deletions helper/activationflags/activation_flags.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,187 @@
// Copyright (c) HashiCorp, Inc.
// SPDX-License-Identifier: BUSL-1.1

package activationflags

import (
"context"
"fmt"
"maps"
"sync"

"github.com/hashicorp/vault/sdk/logical"
)

const (
storagePathActivationFlags = "activation-flags"
)

type FeatureActivationFlags struct {
activationFlagsLock sync.RWMutex
storage logical.Storage
activationFlags map[string]bool
}

func NewFeatureActivationFlags() *FeatureActivationFlags {
return &FeatureActivationFlags{
activationFlags: map[string]bool{},
}
}

func (f *FeatureActivationFlags) Initialize(ctx context.Context, storage logical.Storage) error {
f.activationFlagsLock.Lock()
defer f.activationFlagsLock.Unlock()

if storage == nil {
return fmt.Errorf("unable to access storage")
}

f.storage = storage

entry, err := f.storage.Get(ctx, storagePathActivationFlags)
if err != nil {
return fmt.Errorf("failed to get activation flags from storage: %w", err)
}
if entry == nil {
f.activationFlags = map[string]bool{}
return nil
}

var activationFlags map[string]bool
if err := entry.DecodeJSON(&activationFlags); err != nil {
return fmt.Errorf("failed to decode activation flags from storage: %w", err)
}

f.activationFlags = activationFlags

return nil
}

// Get is the helper function called by the activation-flags API read endpoint. This reads the
// actual values from storage, then updates the in-memory cache of the activation-flags. It
// returns a slice of the feature names which have already been activated.
func (f *FeatureActivationFlags) Get(ctx context.Context) ([]string, error) {
// Don't use nil slice declaration, we want the JSON to show "[]" instead of null
activated := []string{}

_, err := f.ReloadFlagsFromStorage(ctx)
if err != nil {
return activated, err
}

f.activationFlagsLock.Lock()
defer f.activationFlagsLock.Unlock()

for flag, set := range f.activationFlags {
if set {
activated = append(activated, flag)
}
}

return activated, nil
}

func (f *FeatureActivationFlags) ReloadFlagsFromStorage(ctx context.Context) (map[string]bool, error) {
f.activationFlagsLock.Lock()
defer f.activationFlagsLock.Unlock()

if f.storage == nil {
return map[string]bool{}, nil
}

entry, err := f.storage.Get(ctx, storagePathActivationFlags)
if err != nil {
return nil, fmt.Errorf("failed to get activation flags from storage: %w", err)
}
if entry == nil {
return map[string]bool{}, nil
}

var storageActivationFlags map[string]bool
if err := entry.DecodeJSON(&storageActivationFlags); err != nil {
return nil, fmt.Errorf("failed to decode activation flags from storage: %w", err)
}

// State Change Logic for Flags
//
// This logic determines changes to flags, but it does NOT account for flags that have been deleted.
// As of this writing, flag removal is not supported for activation flags.
//
// Valid State Transitions:
// 1. Unset (new flag) -> Active
// 2. Active -> Inactive
// 3. Inactive -> Active
//
// Behavior notes:
// - If a flag does not exist in-memory (`!ok`), it is treated as a new flag.
// Nodes should only react to the new flag if its state is being set to "Active".
// - If a flag exists in-memory, any change in its value (e.g., Active -> Inactive) is considered valid
// and is marked as a state change.
//
// The resulting `changedFlags` map will store the flags with their new values if they meet the above criteria.
changedFlags := map[string]bool{}
for flg, v := range storageActivationFlags {
oldValue, ok := f.activationFlags[flg]

switch {
// New flag: handle only if transitioning to "Active (true)"
case !ok && v:
changedFlags[flg] = v
default:
// Existing flag: detect state change
if oldValue != v {
changedFlags[flg] = v
}
}
}

// Update the in-memory flags after loading the latest values from storage
f.activationFlags = storageActivationFlags

return changedFlags, nil
}

// Write is the helper function called by the activation-flags API write endpoint. This stores
// the boolean value for the activation-flag feature name into Vault storage across the cluster
// and updates the in-memory cache upon success.
func (f *FeatureActivationFlags) Write(ctx context.Context, featureName string, activate bool) (err error) {
f.activationFlagsLock.Lock()
defer f.activationFlagsLock.Unlock()

if f.storage == nil {
return fmt.Errorf("unable to access storage")
}

activationFlags := f.activationFlags

clonedFlags := maps.Clone(f.activationFlags)
clonedFlags[featureName] = activate
// The cloned flags are updated but the in-memory state is only updated on success of the storage update.
defer func() {
if err == nil {
activationFlags[featureName] = activate
}
}()

entry, err := logical.StorageEntryJSON(storagePathActivationFlags, clonedFlags)
if err != nil {
return fmt.Errorf("failed to marshal object to JSON: %w", err)
}

err = f.storage.Put(ctx, entry)
if err != nil {
return fmt.Errorf("failed to save object in storage: %w", err)
}

return nil
}

// IsActivationFlagEnabled is true if the specified flag is enabled in the core.
func (f *FeatureActivationFlags) IsActivationFlagEnabled(featureName string) bool {
f.activationFlagsLock.RLock()
defer f.activationFlagsLock.RUnlock()

activated, ok := f.activationFlags[featureName]

return ok && activated
}
2 changes: 1 addition & 1 deletion helper/forwarding/types.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion helper/identity/mfa/types.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion helper/identity/types.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion helper/storagepacker/types.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading

0 comments on commit b33ed62

Please sign in to comment.