Skip to content

Commit

Permalink
Make using new package manager valkey installed more freindly, fix th…
Browse files Browse the repository at this point in the history
…e places where its assume you have redis-cli, and add the difference distros valkey instalation option

Signed-off-by: avifenesh <[email protected]>
  • Loading branch information
avifenesh committed Jan 26, 2025
1 parent 8ba7ccd commit a998656
Show file tree
Hide file tree
Showing 9 changed files with 162 additions and 70 deletions.
39 changes: 23 additions & 16 deletions csharp/DEVELOPER.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Developer Guide

This document describes how to set up your development environment to build and test the GLIDE for Redis C# wrapper.
This document describes how to set up your development environment to build and test the GLIDE for valkey C# wrapper.

### Development Overview

Expand All @@ -11,7 +11,7 @@ The C# client contains the following parts:
1. Rust part of the C# client located in `lib/src`; it communicates with [GLIDE core rust library](../glide-core/README.md).
2. C# part of the client located in `lib`; it translates Rust async API into .Net async API.
3. Integration tests for the C# client located in `tests` directory.
4. A dedicated benchmarking tool designed to evaluate and compare the performance of GLIDE for Redis and other .Net clients. It is located in `<repo root>/benchmarks/csharp`.
4. A dedicated benchmarking tool designed to evaluate and compare the performance of GLIDE for valkey and other .Net clients. It is located in `<repo root>/benchmarks/csharp`.

TODO: examples, UT, design docs

Expand All @@ -22,7 +22,7 @@ Software Dependencies:
- .Net SDK 6 or later
- git
- rustup
- redis
- valkey

Please also install the following packages to build [GLIDE core rust library](../glide-core/README.md):

Expand Down Expand Up @@ -53,9 +53,25 @@ For example, on Linux you can copy it to `/usr/bin`:
sudo cp protoc /usr/bin/
```

**Redis installation**
**Valkey installation**

```bash
# With yum (CentOS, Fedora, RHEL, AL2023)
sudo yum install valkey

# With apt (Debian, Ubuntu, Mint)
sudo apt-get install valkey

# With zypper (OpenSUSE)
sudo zypper install valkey

# Using package manager (Arch Linux)
sudo pacman -S valkey

# With brew (MacOS)
brew install valkey
```

To install `redis-server` and `redis-cli` on your host, follow the [Redis Installation Guide](https://redis.io/docs/install/install-redis/).

**Dependencies installation for Ubuntu**

Expand All @@ -77,7 +93,7 @@ source "$HOME/.cargo/env"

#### Building and installation steps

Before starting this step, make sure you've installed all software requirments.
Before starting this step, make sure you've installed all software requirements.

1. Clone the repository
```bash
Expand All @@ -101,8 +117,7 @@ dotnet test

4. Run benchmark

1. Ensure that you have installed `redis-server` and `redis-cli` on your host. You can find the Redis installation guide at the following link: [Redis Installation Guide](https://redis.io/docs/install/install-redis/install-redis-on-linux/).

1. Ensure that you have installed `valkey-server` and `valkey-cli` on your host. You can find the valkey installation guide above.
2. Execute the following command from the root project folder:

```bash
Expand Down Expand Up @@ -135,11 +150,3 @@ Rust linter:
cargo clippy --all-features --all-targets -- -D warnings
cargo fmt --all -- --check
```

### Submodules

After pulling new changes, ensure that you update the submodules by running the following command:

```bash
git submodule update
```
73 changes: 43 additions & 30 deletions node/DEVELOPER.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,14 +16,14 @@ Software Dependencies

If your Nodejs version is below the supported version specified in the client's [documentation](https://github.com/valkey-io/valkey-glide/blob/main/node/README.md#nodejs-supported-version), you can upgrade it using [NVM](https://github.com/nvm-sh/nvm?tab=readme-ov-file#install--update-script).

- npm
- git
- GCC
- pkg-config
- protoc (protobuf compiler)
- openssl
- openssl-dev
- rustup
- npm
- git
- GCC
- pkg-config
- protoc (protobuf compiler)
- openssl
- openssl-dev
- rustup

**Dependencies installation for Ubuntu**

Expand Down Expand Up @@ -56,6 +56,27 @@ curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
source "$HOME/.cargo/env"
```

**Valkey Server and CLI**
For testing purposes, you need to install the Valkey server and CLI. You can follow the instructions [here](https://valkey.io/topics/installation/).
Some distributions has a package manager for Valkey. You can install it using the following command, base on your distribution:

```bash
# With yum (CentOS, Fedora, RHEL, AL2023)
sudo yum install valkey

# With apt (Debian, Ubuntu, Mint)
sudo apt-get install valkey

# With zypper (OpenSUSE)
sudo zypper install valkey

# Using package manager (Arch Linux)
sudo pacman -S valkey

# With brew (MacOS)
brew install valkey
```

#### Building and installation steps

Before starting this step, make sure you've installed all software requirments.
Expand Down Expand Up @@ -107,16 +128,16 @@ Before starting this step, make sure you've installed all software requirments.
5. Integrating the built GLIDE package into your project:
Add the package to your project using the folder path with the command `npm install <path to GLIDE>/node`.
- For a fast build, execute `npm run build`. This will perform a full, unoptimized build, which is suitable for developing tests. Keep in mind that performance is significantly affected in an unoptimized build, so it's required to build with the `build:release` or `build:benchmark` option when measuring performance.
- If your modifications are limited to the TypeScript code, run `npm run build-external` to build the external package without rebuilding the internal package.
- If your modifications are limited to the Rust code, execute `npm run build-internal` to build the internal package and generate TypeScript code.
- To generate Node's protobuf files, execute `npm run build-protobuf`. Keep in mind that protobuf files are generated as part of full builds (e.g., `build`, `build:release`, etc.).
- For a fast build, execute `npm run build`. This will perform a full, unoptimized build, which is suitable for developing tests. Keep in mind that performance is significantly affected in an unoptimized build, so it's required to build with the `build:release` or `build:benchmark` option when measuring performance.
- If your modifications are limited to the TypeScript code, run `npm run build-external` to build the external package without rebuilding the internal package.
- If your modifications are limited to the Rust code, execute `npm run build-internal` to build the internal package and generate TypeScript code.
- To generate Node's protobuf files, execute `npm run build-protobuf`. Keep in mind that protobuf files are generated as part of full builds (e.g., `build`, `build:release`, etc.).
> Note: Once building completed, you'll find the compiled JavaScript code in the `node/build-ts` folder.

### Troubleshooting

- If the build fails after running `npx tsc` because `glide-rs` isn't found, check if your npm version is in the range 9.0.0-9.4.1, and if so, upgrade it. 9.4.2 contains a fix to a change introduced in 9.0.0 that is required in order to build the library.
- If the build fails after running `npx tsc` because `glide-rs` isn't found, check if your npm version is in the range 9.0.0-9.4.1, and if so, upgrade it. 9.4.2 contains a fix to a change introduced in 9.0.0 that is required in order to build the library.
### Test
Expand Down Expand Up @@ -186,14 +207,6 @@ It has command history and bash-like search (`Ctrl+R`).
Shell hangs on exit (`Ctrl+D`) if you don't close the clients. Use `Ctrl+C` to kill it and/or close clients before exit.
### Submodules
After pulling new changes, ensure that you update the submodules by running the following command:
```bash
git submodule update
```
### Linters
Development on the Node wrapper may involve changes in either the TypeScript or Rust code. Each language has distinct linter tests that must be passed before committing changes.
Expand All @@ -202,13 +215,13 @@ Development on the Node wrapper may involve changes in either the TypeScript or
**TypeScript:**
- ESLint
- Prettier
- ESLint
- Prettier
**Rust:**
- clippy
- fmt
- clippy
- fmt
#### Running the linters
Expand All @@ -231,8 +244,8 @@ Development on the Node wrapper may involve changes in either the TypeScript or
### Recommended extensions for VS Code
- [Prettier - Code formatter](https://marketplace.visualstudio.com/items?itemName=esbenp.prettier-vscode) - JavaScript / TypeScript formatter.
- [ESLint](https://marketplace.visualstudio.com/items?itemName=dbaeumer.vscode-eslint) - linter.
- [Jest Runner](https://marketplace.visualstudio.com/items?itemName=firsttris.vscode-jest-runner) - in-editor test runner.
- [Jest Test Explorer](https://marketplace.visualstudio.com/items?itemName=kavod-io.vscode-jest-test-adapter) - adapter to the VSCode testing UI.
- [rust-analyzer](https://marketplace.visualstudio.com/items?itemName=rust-lang.rust-analyzer) - Rust language support for VSCode.
- [Prettier - Code formatter](https://marketplace.visualstudio.com/items?itemName=esbenp.prettier-vscode) - JavaScript / TypeScript formatter.
- [ESLint](https://marketplace.visualstudio.com/items?itemName=dbaeumer.vscode-eslint) - linter.
- [Jest Runner](https://marketplace.visualstudio.com/items?itemName=firsttris.vscode-jest-runner) - in-editor test runner.
- [Jest Test Explorer](https://marketplace.visualstudio.com/items?itemName=kavod-io.vscode-jest-test-adapter) - adapter to the VSCode testing UI.
- [rust-analyzer](https://marketplace.visualstudio.com/items?itemName=rust-lang.rust-analyzer) - Rust language support for VSCode.
8 changes: 4 additions & 4 deletions node/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,16 +14,16 @@ The release of Valkey GLIDE was tested on the following platforms:

Linux:

- Ubuntu 22.04.1 (x86_64 and aarch64)
- Amazon Linux 2023 (AL2023) (x86_64)
- Ubuntu 22.04.1 (x86_64 and aarch64)
- Amazon Linux 2023 (AL2023) (x86_64)

macOS:

- macOS 14.7 (Apple silicon/aarch_64)
- macOS 14.7 (Apple silicon/aarch_64)

Alpine:

- node:alpine (default on aarch64 and x86_64)
- node:alpine (default on aarch64 and x86_64)

## NodeJS supported version

Expand Down
20 changes: 18 additions & 2 deletions node/hybrid-node-tests/commonjs-test/commonjs-test.cjs
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,25 @@ const PORT_NUMBER = 4001;
let server;
let port;

function flushallOnPort(port) {
function checkCliAvailability(cli) {
return new Promise((resolve) => {
execFile(cli, ["--version"], (error) => {
resolve(!error);
});
});
}

async function flushallOnPort(port) {
const redisCliAvailable = await checkCliAvailability("redis-cli");
const valkeyCliAvailable = await checkCliAvailability("valkey-cli");

if (!redisCliAvailable && !valkeyCliAvailable) {
throw new Error("Neither redis-cli nor valkey-cli is available");
}

const cli = redisCliAvailable ? "valkey-cli" : "redis-cli";
return new Promise((resolve, reject) => {
execFile("redis-cli", ["-p", port, "FLUSHALL"], (error, _, stderr) => {
execFile(cli, ["-p", port, "FLUSHALL"], (error, _, stderr) => {
if (error) {
console.error(stderr);
reject(error);
Expand Down
20 changes: 18 additions & 2 deletions node/hybrid-node-tests/ecmascript-test/ecmascript-test.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,25 @@ const PORT_NUMBER = 4001;
let server;
let port;

function flushallOnPort(port) {
function checkCliAvailability(cli) {
return new Promise((resolve) => {
execFile(cli, ["--version"], (error) => {
resolve(!error);
});
});
}

async function flushallOnPort(port) {
const redisCliAvailable = await checkCliAvailability("redis-cli");
const valkeyCliAvailable = await checkCliAvailability("valkey-cli");

if (!redisCliAvailable && !valkeyCliAvailable) {
throw new Error("Neither redis-cli nor valkey-cli is available");
}

const cli = redisCliAvailable ? "valkey-cli" : "redis-cli";
return new Promise((resolve, reject) => {
execFile("redis-cli", ["-p", port, "FLUSHALL"], (error, _, stderr) => {
execFile(cli, ["-p", port, "FLUSHALL"], (error, _, stderr) => {
if (error) {
console.error(stderr);
reject(error);
Expand Down
42 changes: 32 additions & 10 deletions node/tests/TestUtilities.ts
Original file line number Diff line number Diff line change
Expand Up @@ -193,17 +193,39 @@ export async function GetAndSetRandomValue(client: Client) {
expect(intoString(result)).toEqual(value);
}

function checkCliAvailability(cli: string): Promise<boolean> {
return new Promise((resolve) => {
exec(`${cli} --version`, (error) => {
resolve(!error);
});
});
}

export function flushallOnPort(port: number): Promise<void> {
return new Promise<void>((resolve, reject) =>
exec(`redis-cli -p ${port} FLUSHALL`, (error, _, stderr) => {
if (error) {
console.error(stderr);
reject(error);
} else {
resolve();
}
}),
);
return new Promise<void>((resolve, reject) => {
checkCliAvailability("valkey-cli")
.then(async (redisCliAvailable) => {
const valkeyCliAvailable =
await checkCliAvailability("redis-cli");

if (!redisCliAvailable && !valkeyCliAvailable) {
throw new Error(
"Neither redis-cli nor valkey-cli is available",
);
}

const cli_1 = redisCliAvailable ? "valkey-cli" : "redis-cli";
exec(`${cli_1} -p ${port} FLUSHALL`, (error, _, stderr) => {
if (error) {
console.error(stderr);
reject(error);
} else {
resolve();
}
});
})
.catch(reject);
});
}

/**
Expand Down
19 changes: 19 additions & 0 deletions python/DEVELOPER.md
Original file line number Diff line number Diff line change
Expand Up @@ -126,6 +126,25 @@ maturin develop --release --strip

Ensure that you have installed `redis-server` or `valkey-server` along with `redis-cli` or `valkey-cli` on your host. You can find the Redis installation guide at the following link: [Redis Installation Guide](https://redis.io/docs/install/install-redis/install-redis-on-linux/). You can get Valkey from the following link: [Valkey Download](https://valkey.io/download/).

Or by using the following commands for the fitting package manager for your system (depends on the distro version):

```bash
# With yum (CentOS, Fedora, RHEL, AL2023)
sudo yum install valkey

# With apt (Debian, Ubuntu, Mint)
sudo apt-get install valkey

# With zypper (OpenSUSE)
sudo zypper install valkey

# Using package manager (Arch Linux)
sudo pacman -S valkey

# With brew (MacOS)
brew install valkey
```

From a terminal, change directory to the GLIDE source folder and type:

```bash
Expand Down
6 changes: 3 additions & 3 deletions utils/TestUtils.ts
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,9 @@ function parseOutput(input: string): {
.split(",")
.map((address) => address.split(":"))
.map((address) => [address[0], Number(address[1])]) as [
string,
number,
][];
string,
number,
][];

if (clusterFolder === undefined || ports === undefined) {
throw new Error(`Insufficient data in input: ${input}`);
Expand Down
5 changes: 2 additions & 3 deletions utils/cluster_manager.py
Original file line number Diff line number Diff line change
Expand Up @@ -355,8 +355,7 @@ def get_server_command() -> str:
return server
except Exception as e:
logging.error(f"Error checking {server}: {e}")
raise Exception(
"Neither valkey-server nor redis-server found in the system.")
raise Exception("Neither valkey-server nor redis-server found in the system.")

def get_server_version(server_name):
result = subprocess.run(
Expand Down Expand Up @@ -693,7 +692,7 @@ def wait_for_all_topology_views(
if output is not None and output.count(f"{server.host}") == len(servers):
# Server is ready, get the node's role
cmd_args = [
"redis-cli",
CLI_COMMAND,
"-h",
server.host,
"-p",
Expand Down

0 comments on commit a998656

Please sign in to comment.