Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: strict and dynamic mode #477

Merged
merged 11 commits into from
Jul 5, 2024
Merged

docs: strict and dynamic mode #477

Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
463cf49
docs: strict and dynamic mode
chriswk Jul 5, 2024
119eea0
docs: it's behavior not mode
chriswk Jul 5, 2024
16d40e7
fix: replace generated ULID in CLI with placeholder
chriswk Jul 5, 2024
9868b84
docs: replace extra of in strict/dynamic mode doc
chriswk Jul 5, 2024
a632c6f
Update README.md
chriswk Jul 5, 2024
f2d9392
Update README.md
chriswk Jul 5, 2024
875cbcc
Update README.md
chriswk Jul 5, 2024
71649fa
Make it clear that Edge will always warn when using dynamic behavior
chriswk Jul 5, 2024
18e1446
docs: make it clear that edge will always print a warning with dynami…
chriswk Jul 5, 2024
564e3ce
docs: reformatted paragraph on strict mode
chriswk Jul 5, 2024
80cec0e
suggestion from PR
chriswk Jul 5, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
93 changes: 61 additions & 32 deletions CLI.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,8 @@ This document contains the help content for the `unleash-edge` command-line prog
* `-b`, `--base-path <BASE_PATH>` — Which base path should this server listen for HTTP traffic on

Default value: ``
* `-w`, `--workers <WORKERS>` — How many workers should be started to handle requests. Defaults to number of physical cpus
* `-w`, `--workers <WORKERS>` — How many workers should be started to handle requests. Defaults to number of physical
cpus
Copy link
Member

@sighphyre sighphyre Jul 5, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't really affect the markdown but what's going on here? IntelliJ enforcing a 120 character line?

Edit: Okay I changed my mind, it's happening everywhere and it makes the commit quite noisy

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like it


Default value: `16`
* `--tls-enable` — Should we bind TLS
Expand All @@ -43,25 +44,33 @@ This document contains the help content for the `unleash-edge` command-line prog

* `--tls-server-key <TLS_SERVER_KEY>` — Server key to use for TLS - Needs to be a path to a file
* `--tls-server-cert <TLS_SERVER_CERT>` — Server Cert to use for TLS - Needs to be a path to a file
* `--tls-server-port <TLS_SERVER_PORT>` — Port to listen for https connection on (will use the interfaces already defined)
* `--tls-server-port <TLS_SERVER_PORT>` — Port to listen for https connection on (will use the interfaces already
defined)

Default value: `3043`
* `--instance-id <INSTANCE_ID>` — Instance id. Used for metrics reporting

Default value: `01HX9QXY3MNKKAMP7XRS6RG2Q4`
Default value: `<GENERATED_ULID>`
* `-a`, `--app-name <APP_NAME>` — App name. Used for metrics reporting

Default value: `unleash-edge`
* `--markdown-help`

Possible values: `true`, `false`

* `--trust-proxy` — By enabling the trust proxy option. Unleash Edge will have knowledge that it's sitting behind a proxy and that the X-Forward-\* header fields may be trusted, which otherwise may be easily spoofed. Edge will use this to populate its context's remoteAddress field If you need to only trust specific ips or CIDR, enable this flag and then set `--proxy-trusted-servers`
* `--trust-proxy` — By enabling the trust proxy option. Unleash Edge will have knowledge that it's sitting behind a
proxy and that the X-Forward-\* header fields may be trusted, which otherwise may be easily spoofed. Edge will use
this to populate its context's remoteAddress field If you need to only trust specific ips or CIDR, enable this flag
and then set `--proxy-trusted-servers`

Possible values: `true`, `false`

* `--proxy-trusted-servers <PROXY_TRUSTED_SERVERS>` — Tells Unleash Edge which servers to trust the X-Forwarded-For. Accepts explicit Ip addresses or Cidrs (127.0.0.1/16). Accepts a comma separated list or multiple instances of the flag. E.g `--proxy-trusted-servers "127.0.0.1,192.168.0.1"` and `--proxy-trusted-servers 127.0.0.1 --proxy-trusted-servers 192.168.0.1` are equivalent
* `--disable-all-endpoint` — Set this flag to true if you want to disable /api/proxy/all and /api/frontend/all Because returning all toggles regardless of their state is a potential security vulnerability, these endpoints can be disabled
* `--proxy-trusted-servers <PROXY_TRUSTED_SERVERS>` — Tells Unleash Edge which servers to trust the X-Forwarded-For.
Accepts explicit Ip addresses or Cidrs (127.0.0.1/16). Accepts a comma separated list or multiple instances of the
flag. E.g `--proxy-trusted-servers "127.0.0.1,192.168.0.1"`
and `--proxy-trusted-servers 127.0.0.1 --proxy-trusted-servers 192.168.0.1` are equivalent
* `--disable-all-endpoint` — Set this flag to true if you want to disable /api/proxy/all and /api/frontend/all Because
returning all toggles regardless of their state is a potential security vulnerability, these endpoints can be disabled

Default value: `false`

Expand All @@ -80,8 +89,6 @@ This document contains the help content for the `unleash-edge` command-line prog

Default value: `Authorization`



## `unleash-edge edge`

Run in edge mode
Expand All @@ -90,30 +97,42 @@ Run in edge mode

###### **Options:**

* `-u`, `--upstream-url <UPSTREAM_URL>` — Where is your upstream URL. Remember, this is the URL to your instance, without any trailing /api suffix
* `-b`, `--backup-folder <BACKUP_FOLDER>` — A path to a local folder. Edge will write feature and token data to disk in this folder and read this back after restart. Mutually exclusive with the --redis-url option
* `-u`, `--upstream-url <UPSTREAM_URL>` — Where is your upstream URL. Remember, this is the URL to your instance,
without any trailing /api suffix
* `-b`, `--backup-folder <BACKUP_FOLDER>` — A path to a local folder. Edge will write feature and token data to disk in
this folder and read this back after restart. Mutually exclusive with the --redis-url option
* `-m`, `--metrics-interval-seconds <METRICS_INTERVAL_SECONDS>` — How often should we post metrics upstream?

Default value: `60`
* `-f`, `--features-refresh-interval-seconds <FEATURES_REFRESH_INTERVAL_SECONDS>` — How long between each refresh for a token
* `-f`, `--features-refresh-interval-seconds <FEATURES_REFRESH_INTERVAL_SECONDS>` — How long between each refresh for a
token

Default value: `10`
* `--token-revalidation-interval-seconds <TOKEN_REVALIDATION_INTERVAL_SECONDS>` — How long between each revalidation of a token
* `--token-revalidation-interval-seconds <TOKEN_REVALIDATION_INTERVAL_SECONDS>` — How long between each revalidation of
a token

Default value: `3600`
* `-t`, `--tokens <TOKENS>` — Get data for these client tokens at startup. Accepts comma-separated list of tokens. Hot starts your feature cache
* `-H`, `--custom-client-headers <CUSTOM_CLIENT_HEADERS>` — Expects curl header format (-H <HEADERNAME>: <HEADERVALUE>) for instance `-H X-Api-Key: mysecretapikey`
* `-s`, `--skip-ssl-verification` — If set to true, we will skip SSL verification when connecting to the upstream Unleash server
* `-t`, `--tokens <TOKENS>` — Get data for these client tokens at startup. Accepts comma-separated list of tokens. Hot
starts your feature cache
* `-H`, `--custom-client-headers <CUSTOM_CLIENT_HEADERS>` — Expects curl header format (-H <HEADERNAME>: <HEADERVALUE>)
for instance `-H X-Api-Key: mysecretapikey`
* `-s`, `--skip-ssl-verification` — If set to true, we will skip SSL verification when connecting to the upstream
Unleash server

Default value: `false`

Possible values: `true`, `false`

* `--pkcs8-client-certificate-file <PKCS8_CLIENT_CERTIFICATE_FILE>` — Client certificate chain in PEM encoded X509 format with the leaf certificate first. The certificate chain should contain any intermediate certificates that should be sent to clients to allow them to build a chain to a trusted root
* `--pkcs8-client-key-file <PKCS8_CLIENT_KEY_FILE>` — Client key is a PEM encoded PKCS#8 formatted private key for the leaf certificate
* `--pkcs12-identity-file <PKCS12_IDENTITY_FILE>` — Identity file in pkcs12 format. Typically this file has a pfx extension
* `--pkcs8-client-certificate-file <PKCS8_CLIENT_CERTIFICATE_FILE>` — Client certificate chain in PEM encoded X509
format with the leaf certificate first. The certificate chain should contain any intermediate certificates that should
be sent to clients to allow them to build a chain to a trusted root
* `--pkcs8-client-key-file <PKCS8_CLIENT_KEY_FILE>` — Client key is a PEM encoded PKCS#8 formatted private key for the
leaf certificate
* `--pkcs12-identity-file <PKCS12_IDENTITY_FILE>` — Identity file in pkcs12 format. Typically this file has a pfx
extension
* `--pkcs12-passphrase <PKCS12_PASSPHRASE>` — Passphrase used to unlock the pkcs12 file
* `--upstream-certificate-file <UPSTREAM_CERTIFICATE_FILE>` — Extra certificate passed to the client for building its trust chain. Needs to be in PEM format (crt or pem extensions usually are)
* `--upstream-certificate-file <UPSTREAM_CERTIFICATE_FILE>` — Extra certificate passed to the client for building its
trust chain. Needs to be in PEM format (crt or pem extensions usually are)
* `--upstream-request-timeout <UPSTREAM_REQUEST_TIMEOUT>` — Timeout for requests to the upstream server

Default value: `5`
Expand Down Expand Up @@ -143,11 +162,23 @@ Run in edge mode

Possible values: `tcp`, `tls`, `redis`, `rediss`, `redis-unix`, `unix`

* `--token-header <TOKEN_HEADER>` — Token header to use for both edge authorization and communication with the upstream server
* `--token-header <TOKEN_HEADER>` — Token header to use for both edge authorization and communication with the upstream
server

Default value: `Authorization`
* `--strict` — If set to true, Edge starts with strict behavior. Strict behavior means that Edge will refuse tokens
outside of the scope of the startup tokens

Default value: `false`

Possible values: `true`, `false`

* `--dynamic` — If set to true, Edge starts with dynamic behavior. Dynamic behavior means that Edge will accept tokens
outside of the scope of the startup tokens

Default value: `false`

Possible values: `true`, `false`

## `unleash-edge offline`

Expand All @@ -158,13 +189,13 @@ Run in offline mode
###### **Options:**

* `-b`, `--bootstrap-file <BOOTSTRAP_FILE>` — The file to load our features from. This data will be loaded at startup
* `-t`, `--tokens <TOKENS>` — Tokens that should be allowed to connect to Edge. Supports a comma separated list or multiple instances of the `--tokens` argument
* `-r`, `--reload-interval <RELOAD_INTERVAL>` — The interval in seconds between reloading the bootstrap file. Disabled if unset or 0
* `-t`, `--tokens <TOKENS>` — Tokens that should be allowed to connect to Edge. Supports a comma separated list or
multiple instances of the `--tokens` argument
* `-r`, `--reload-interval <RELOAD_INTERVAL>` — The interval in seconds between reloading the bootstrap file. Disabled
if unset or 0

Default value: `0`



## `unleash-edge health`

Perform a health check against a running edge instance
Expand All @@ -176,9 +207,8 @@ Perform a health check against a running edge instance
* `-e`, `--edge-url <EDGE_URL>` — Where the instance you want to health check is running

Default value: `http://localhost:3063`
* `-c`, `--ca-certificate-file <CA_CERTIFICATE_FILE>` — If you're hosting Edge using a self-signed TLS certificate use this to tell healthcheck about your CA


* `-c`, `--ca-certificate-file <CA_CERTIFICATE_FILE>` — If you're hosting Edge using a self-signed TLS certificate use
this to tell healthcheck about your CA

## `unleash-edge ready`

Expand All @@ -191,14 +221,13 @@ Perform a ready check against a running edge instance
* `-e`, `--edge-url <EDGE_URL>` — Where the instance you want to health check is running

Default value: `http://localhost:3063`
* `-c`, `--ca-certificate-file <CA_CERTIFICATE_FILE>` — If you're hosting Edge using a self-signed TLS certificate use this to tell the readychecker about your CA


* `-c`, `--ca-certificate-file <CA_CERTIFICATE_FILE>` — If you're hosting Edge using a self-signed TLS certificate use
this to tell the readychecker about your CA

<hr/>

<small><i>
This document was generated automatically by
<a href="https://crates.io/crates/clap-markdown"><code>clap-markdown</code></a>.
This document was generated automatically by
<a href="https://crates.io/crates/clap-markdown"><code>clap-markdown</code></a>.
</i></small>

46 changes: 45 additions & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,49 @@ Running Edge in Docker with our recommended setup:
docker run -it -p 3063:3063 -e STRICT=true -e UPSTREAM_URL=<yourunleashinstance> unleashorg/unleash-edge:<mostrecentversion> edge
```

## Edge behaviors

As of version 19.2.0, Unleash Edge now supports two behaviors when running in edge mode: **strict** and **dynamic**. We recommend adopting the new **strict** behavior, while **dynamic** remains as a legacy option that will be deprecated and removed in a future release.

For legacy reasons, **dynamic** behavior is still the default. However, a warning will be logged at startup to indicate its deprecation.

Please note that these behaviors are mutually exclusive.

### Strict behavior

If started with the `--strict` flag or the `STRICT` environment variable, Edge now starts with strict behavior and must
be
given
tokens at startup.
Edge will lock down access for new tokens that have a wider or different access scope than the tokens it was started
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Edge will lock down access for new tokens that have a wider or different access scope than the tokens it was started
Edge will refuse access for new tokens that have a wider or different access scope than the tokens it was started

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is cool but I'd like to get a bit more specific in terms of what access means here. We do a nice job providing examples but I think we can do even better just by explaining what this is.

Let's lift this:

Strict behavior still allows using tokens (both frontend and client) with narrower access than what the tokens it was started with has.

And combine it with this section and then build on that. Something like...?

Edge will refuses requests from SDKs that have a wider or different access scope than the initial tokens. Specifically, this means incoming requests must have a token that exactly matches the environment and is bound to a project or projects specified in that token.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can also tell a story on why this is valuable. Struggling to not go "full marketer" here but something like "this gives you more predictable control over which SDKs have access to Edge and prevents you from expanding the scope at runtime accidentally"

with.

E.g. if you start with one wildcard token with access to development `*:development.<somelongrandomstring` and later a
client comes with a
token accessing a different environment, say production `*:production.<somedifferentrandomstring>` Edge will return 403,
because that would change the scope of access.

Strict behavior still allows using tokens (both frontend and client) with narrower access than what the tokens it was
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we could take this and the paragraph below out

started with has.

E.g. If you start with one wildcard token with access to the development
environment `*:development.<somelongrandomstring>` and your clients use various tokens with access to specific projects
in the development environment, Edge will filter features to only grant access to the narrower scope.

### Dynamic behavior

With dynamic behavior, Edge behaves as it has since v1.0.0. Any new client tokens are validated against upstream and if
found
to be valid, a refresh job will be configured with the minimum set of tokens that are able to fetch all projects and
environments Edge has seen. Set with `--dynamic` or the `DYNAMIC` environment variable. (19.2.0 / July 4th 2024): We're
looking to deprecate this
behavior. If your company needs this behavior, reach out to us on Slack, the final decision has not been made yet. In
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just feels a bit more friendly

Suggested change
behavior. If your company needs this behavior, reach out to us on Slack, the final decision has not been made yet. In
behavior. If you need this behavior, reach out to us on Slack, the final decision has not been made yet. In

19.2.0
this behavior is still the default, but Edge will log that you should make a choice between dynamic or strict behavior.
If
you
explicitly choose dynamic behavior, Edge will warn about the planned deprecation.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even if you don't explicitly choose it and we default to it, we still show the warning. It behaves as shown in the PR description of #476

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's true.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. Changed text to When dynamic behavior is selected (by default or by choice), Edge will print a warning about the planned deprecation


## Deploying

See our page on [Deploying Edge](./docs/deploying.md)
Expand Down Expand Up @@ -121,7 +164,8 @@ See more about available logging and log levels at https://docs.rs/env_logger/la

- Old Edge version. In order to guarantee metrics on newer Unleash versions, you will need to be using Edge v17.0.0 or
newer
- Old SDK clients. We've noticed that some clients, particularly early Python (1.x branch) as well as earlier .NET SDKs (we
- Old SDK clients. We've noticed that some clients, particularly early Python (1.x branch) as well as earlier .NET
SDKs (we
recommend you use 4.1.5 or newer) struggle to post metrics with the strict headers Edge requires.

## Development
Expand Down
37 changes: 23 additions & 14 deletions server/src/builder.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,9 @@ pub(crate) fn build_offline_mode(

fn build_offline(offline_args: OfflineArgs) -> EdgeResult<CacheContainer> {
if offline_args.tokens.is_empty() {
return Err(EdgeError::NoTokens("No tokens provided. Tokens must be specified when running in offline mode".into()));
return Err(EdgeError::NoTokens(
"No tokens provided. Tokens must be specified when running in offline mode".into(),
));
}

if let Some(bootstrap) = offline_args.bootstrap_file {
Expand Down Expand Up @@ -163,7 +165,9 @@ async fn build_edge(args: &EdgeArgs) -> EdgeResult<EdgeInfo> {
}

if args.strict && args.tokens.is_empty() {
return Err(EdgeError::NoTokens("No tokens provided. Tokens must be specified when running with strict behavior".into()));
return Err(EdgeError::NoTokens(
"No tokens provided. Tokens must be specified when running with strict behavior".into(),
));
}

let (token_cache, feature_cache, engine_cache) = build_caches();
Expand Down Expand Up @@ -242,22 +246,25 @@ pub async fn build_caches_and_refreshers(args: CliArgs) -> EdgeResult<EdgeInfo>

#[cfg(test)]
mod tests {
use crate::{builder::{build_edge, build_offline}, cli::{EdgeArgs, OfflineArgs, TokenHeader}};
use crate::{
builder::{build_edge, build_offline},
cli::{EdgeArgs, OfflineArgs, TokenHeader},
};

#[test]
fn should_fail_with_empty_tokens_when_offline_mode() {
let args = OfflineArgs {
bootstrap_file: None,
tokens: vec![],
reload_interval: Default::default()
reload_interval: Default::default(),
};

let result = build_offline(args);
assert!(result.is_err());
assert_eq!(result
.err()
.unwrap()
.to_string(), "No tokens provided. Tokens must be specified when running in offline mode");
assert_eq!(
result.err().unwrap().to_string(),
"No tokens provided. Tokens must be specified when running in offline mode"
);
}

#[tokio::test]
Expand All @@ -276,16 +283,18 @@ mod tests {
upstream_request_timeout: Default::default(),
upstream_socket_timeout: Default::default(),
custom_client_headers: Default::default(),
token_header: TokenHeader { token_header: "Authorization".into() },
token_header: TokenHeader {
token_header: "Authorization".into(),
},
upstream_certificate_file: Default::default(),
token_revalidation_interval_seconds: Default::default(),
};

let result = build_edge(&args).await;
assert!(result.is_err());
assert_eq!(result
.err()
.unwrap()
.to_string(), "No tokens provided. Tokens must be specified when running with strict behavior");
assert_eq!(
result.err().unwrap().to_string(),
"No tokens provided. Tokens must be specified when running with strict behavior"
);
}
}
}
4 changes: 2 additions & 2 deletions server/src/cli.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -175,11 +175,11 @@ pub struct EdgeArgs {
#[clap(long, env, global = true, default_value = "Authorization")]
pub token_header: TokenHeader,

/// If set to true, Edge starts with strict behavior. Strict behavior means that Edge will refuse tokens outside of the scope of the startup tokens
/// If set to true, Edge starts with strict behavior. Strict behavior means that Edge will refuse tokens outside the scope of the startup tokens
#[clap(long, env, default_value_t = false)]
pub strict: bool,

/// If set to true, Edge starts with dynamic behavior. Dynamic behavior means that Edge will accept tokens outside of the scope of the startup tokens
/// If set to true, Edge starts with dynamic behavior. Dynamic behavior means that Edge will accept tokens outside the scope of the startup tokens
#[clap(long, env, default_value_t = false, conflicts_with = "strict")]
pub dynamic: bool,
}
Expand Down
3 changes: 2 additions & 1 deletion server/src/client_api.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1101,7 +1101,8 @@ mod tests {
assert_eq!(result.status(), StatusCode::NOT_FOUND);
}
#[tokio::test]
pub async fn still_subsumes_tokens_after_moving_registration_to_initial_hydration_when_dynamic() {
pub async fn still_subsumes_tokens_after_moving_registration_to_initial_hydration_when_dynamic()
{
let upstream_features_cache: Arc<DashMap<String, ClientFeatures>> =
Arc::new(DashMap::default());
let upstream_token_cache: Arc<DashMap<String, EdgeToken>> = Arc::new(DashMap::default());
Expand Down
Loading