Skip to content

Latest commit

 

History

History
103 lines (74 loc) · 4.29 KB

DEVELOPER.md

File metadata and controls

103 lines (74 loc) · 4.29 KB

Developer Guide

This document describes how to set up your development environment to build and test the GLIDE for Redis Python wrapper.

Development Overview

The GLIDE for Redis Python wrapper consists of both Python and Rust code. Rust bindings for Python are implemented using PyO3, and the Python package is built using maturin. The Python and Rust components communicate using the protobuf protocol.

Build

  • Follow the building instructions for the Python wrapper in the Build from source section to clone the code and build the wrapper.

  • Install Python development requirements with:

    pip install -r python/dev_requirements.txt

  • For a fast build, execute maturin develop without the release flag. This will perform an unoptimized build, which is suitable for developing tests. Keep in mind that performance is significantly affected in an unoptimized build, so it's required to include the "--release" flag when measuring performance.

Test

To run tests, use the following command:

pytest --asyncio-mode=auto

To execute a specific test, include the -k <test_name> option. For example:

pytest --asyncio-mode=auto -k test_socket_set_and_get

Submodules

After pulling new changes, ensure that you update the submodules by running the following command:

git submodule update

Generate protobuf files

During the initial build, Python protobuf files were created in python/python/glide/protobuf. If modifications are made to the protobuf definition files (.proto files located in glide-core/src/protofuf), it becomes necessary to regenerate the Python protobuf files. To do so, run:

GLIDE_ROOT_FOLDER_PATH=. # e.g. /home/ubuntu/glide-for-redis
protoc -Iprotobuf=${GLIDE_ROOT_FOLDER_PATH}/glide-core/src/protobuf/ --python_out=${GLIDE_ROOT_FOLDER_PATH}/python/python/glide ${GLIDE_ROOT_FOLDER_PATH}/glide-core/src/protobuf/*.proto

Protobuf interface files

To generate the protobuf files with Python Interface files (pyi) for type-checking purposes, ensure you have installed mypy-protobuf with pip, and then execute the following command:

GLIDE_ROOT_FOLDER_PATH=. # e.g. /home/ubuntu/glide-for-redis
MYPY_PROTOC_PATH=`which protoc-gen-mypy`
protoc --plugin=protoc-gen-mypy=${MYPY_PROTOC_PATH} -Iprotobuf=${GLIDE_ROOT_FOLDER_PATH}/glide-core/src/protobuf/ --python_out=${GLIDE_ROOT_FOLDER_PATH}/python/python/glide --mypy_out=${GLIDE_ROOT_FOLDER_PATH}/python/python/glide ${GLIDE_ROOT_FOLDER_PATH}/glide-core/src/protobuf/*.proto

Linters

Development on the Python wrapper may involve changes in either the Python or Rust code. Each language has distinct linter tests that must be passed before committing changes.

Language-specific Linters

Python:

  • flake8
  • isort
  • black
  • mypy

Rust:

  • clippy
  • fmt

Running the linters

Run from the main /python folder

  1. Python

    Note: make sure to generate protobuf with interface files before running mypy linter

    pip install -r dev_requirements.txt
isort . --profile black --skip-glob python/glide/protobuf
black . --exclude python/glide/protobuf
flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics --exclude=python/glide/protobuf,.env/* --extend-ignore=E230
flake8 . --count --exit-zero --max-complexity=12 --max-line-length=127 --statistics --exclude=python/glide/protobuf,.env/* --extend-ignore=E230
# run type check
mypy .
  2. Rust 
    rustup component add clippy rustfmt
cargo clippy --all-features --all-targets -- -D warnings
cargo fmt --manifest-path ./Cargo.toml --all

Recommended extensions for VS Code