Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add benchmark scripts #26

Merged
merged 10 commits into from
May 17, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@
*.jpeg filter=lfs diff=lfs merge=lfs -text
*.svg filter=lfs diff=lfs merge=lfs -text
94 changes: 94 additions & 0 deletions .github/workflows/benchmark.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
---
# This starter workflow is for a CMake project running on multiple platforms. There is a different starter workflow if you just want a single platform.
# See: https://github.com/actions/starter-workflows/blob/main/ci/cmake-single-platform.yml
name: Benchmark on multiple platforms
on:
push:
branches: ["develop"]

pull_request:
branches: ["main"]

workflow_dispatch:

jobs:
graph:
runs-on: ${{ matrix.os }}
strategy:
# Set fail-fast to false to ensure that feedback is delivered for all matrix combinations. Consider changing this to true when your workflow is stable.
fail-fast: false
# Set up a matrix to run the following 3 configurations:
# 1. <Windows, Release, latest MSVC compiler toolchain on the default runner image, default generator>
# 2. <Linux, Release, latest GCC compiler toolchain on the default runner image, default generator>
# 3. <Linux, Release, latest Clang compiler toolchain on the default runner image, default generator>
#
# To add more build types (Release, Debug, RelWithDebInfo, etc.) customize the build_type list.
matrix:
os: [ubuntu-latest, macos-14]
build_type: [Release]
c_compiler: [clang]
include:
- os: ubuntu-latest
c_compiler: clang
cpp_compiler: clang++
cmake_extra_options: '-GNinja'
- os: macos-14
c_compiler: clang
cpp_compiler: clang++
cmake_extra_options: '-GNinja'

steps:
- uses: actions/checkout@v4

- name: Set reusable strings
# Turn repeated input strings (such as the build output directory) into step outputs. These step outputs can be used throughout the workflow file.
id: strings
shell: bash
run: |
echo "build-output-dir=${{ github.workspace }}/cmake-build" >> "$GITHUB_OUTPUT"
echo "py-build-output-dir=${{ github.workspace }}/build" >> "$GITHUB_OUTPUT"
echo "CMAKE_C_COMPILER=${{ matrix.c_compiler }}" >> "$GITHUB_ENV"
echo "CMAKE_CXX_COMPILER=${{ matrix.cpp_compiler }}" >> "$GITHUB_ENV"
echo "CMAKE_OPTIONS=${{ matrix.cmake_extra_options }}" >> "$GITHUB_ENV"

- name: Set up Homebrew
id: set-up-homebrew
uses: Homebrew/actions/setup-homebrew@master

- name: Install LLVM
run: |
brew install llvm ninja
echo "PATH=$(brew --prefix llvm)/bin:$PATH" >> "$GITHUB_ENV"

- name: Set up Python
uses: actions/setup-python@v5
with:
cache: 'pip'
python-version: '3.12'

- name: Install dependencies
run: pip install .
shell: bash

- name: Configure CMake
# Configure CMake in a 'build' subdirectory. `CMAKE_BUILD_TYPE` is only required if you are using a single-configuration generator such as make.
# See https://cmake.org/cmake/help/latest/variable/CMAKE_BUILD_TYPE.html?highlight=cmake_build_type
run: cmake -B "${{github.workspace}}/build" -DCMAKE_BUILD_TYPE=${{env.BUILD_TYPE}} -DCMAKE_EXPORT_COMPILE_COMMANDS=ON -DPYTHON_INCLUDE_DIR=$(python -c "from distutils.sysconfig import get_python_inc; print(get_python_inc())") -DPYTHON_LIBRARY=$(python -c "import distutils.sysconfig as sysconfig; print(sysconfig.get_config_var('LIBDIR'))") -DPYTHON_EXECUTABLE:FILEPATH=`which python` -DSTP_BUILD_COMPONENT_EXECUTABLE=True
shell: bash

- name: CMake Build
# Build CMake in a 'build' subdirectory.
run: cmake --build "${{github.workspace}}/build"
shell: bash

- name: Benchmark
run: python tools/benchmark.py -g
shell: bash

- name: Upload Graphs
uses: actions/[email protected]
with:
# Artifact name
name: Benchmark ${{ matrix.os }}
# A file, directory or wildcard pattern that describes what to upload
path: ${{ github.workspace }}/tools/figures
162 changes: 82 additions & 80 deletions .github/workflows/cmake-multi-platform.yml
Original file line number Diff line number Diff line change
@@ -1,110 +1,112 @@
---
# This starter workflow is for a CMake project running on multiple platforms. There is a different starter workflow if you just want a single platform.
# See: https://github.com/actions/starter-workflows/blob/main/ci/cmake-single-platform.yml
name: CMake on multiple platforms

on:
push:
branches: [ "develop" ]
pull_request:
branches: [ "main" ]
branches: ["develop"]
paths:
- "**.cpp"
- "**.hpp"
- "**.py"

pull_request:
branches: ["main"]
jobs:
build:
runs-on: ${{ matrix.os }}

strategy:
# Set fail-fast to false to ensure that feedback is delivered for all matrix combinations. Consider changing this to true when your workflow is stable.
fail-fast: false

# Set up a matrix to run the following 3 configurations:
# 1. <Windows, Release, latest MSVC compiler toolchain on the default runner image, default generator>
# 2. <Linux, Release, latest GCC compiler toolchain on the default runner image, default generator>
# 3. <Linux, Release, latest Clang compiler toolchain on the default runner image, default generator>
#
# To add more build types (Release, Debug, RelWithDebInfo, etc.) customize the build_type list.
matrix:
os: [ ubuntu-latest, windows-latest, macos-12, macos-13, macos-14 ]
build_type: [ Release ]
c_compiler: [ clang, cl ]
python_version: [ '3.10', '3.11', '3.12' ]
os: [ubuntu-latest, windows-latest, macos-12, macos-13, macos-14]
build_type: [Release]
c_compiler: [clang, cl]
python_version: ['3.10', '3.11', '3.12']
include:
- os: windows-latest
c_compiler: cl
cpp_compiler: cl
cmake_extra_options: ''
- os: ubuntu-latest
c_compiler: clang
cpp_compiler: clang++
cmake_extra_options: '-GNinja'
- os: windows-latest
c_compiler: cl
cpp_compiler: cl
cmake_extra_options: ''
- os: ubuntu-latest
c_compiler: clang
cpp_compiler: clang++
cmake_extra_options: '-GNinja'
# - os: macos-latest
# c_compiler: gcc
# cpp_compiler: g++
- os: macos-12
c_compiler: clang
cpp_compiler: clang++
cmake_extra_options: '-GNinja'
- os: macos-13
c_compiler: clang
cpp_compiler: clang++
cmake_extra_options: '-GNinja'
- os: macos-14
c_compiler: clang
cpp_compiler: clang++
cmake_extra_options: '-GNinja'

- os: macos-12
c_compiler: clang
cpp_compiler: clang++
cmake_extra_options: '-GNinja'
- os: macos-13
c_compiler: clang
cpp_compiler: clang++
cmake_extra_options: '-GNinja'
- os: macos-14
c_compiler: clang
cpp_compiler: clang++
cmake_extra_options: '-GNinja'
exclude:
- os: windows-latest
c_compiler: clang
- os: ubuntu-latest
c_compiler: cl
- os: macos-12
c_compiler: cl
- os: macos-13
c_compiler: cl
- os: macos-14
c_compiler: cl

- os: windows-latest
c_compiler: clang
- os: ubuntu-latest
c_compiler: cl
- os: macos-12
c_compiler: cl
- os: macos-13
c_compiler: cl
- os: macos-14
c_compiler: cl
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v4

- name: Set reusable strings
# Turn repeated input strings (such as the build output directory) into step outputs. These step outputs can be used throughout the workflow file.
id: strings
shell: bash
run: |
echo "build-output-dir=${{ github.workspace }}/cmake-build" >> "$GITHUB_OUTPUT"
echo "py-build-output-dir=${{ github.workspace }}/build" >> "$GITHUB_OUTPUT"
echo "CMAKE_C_COMPILER=${{ matrix.c_compiler }}" >> "$GITHUB_ENV"
echo "CMAKE_CXX_COMPILER=${{ matrix.cpp_compiler }}" >> "$GITHUB_ENV"
echo "CMAKE_OPTIONS=${{ matrix.cmake_extra_options }}" >> "$GITHUB_ENV"

- name: Set up Homebrew
if: ${{ matrix.os == 'macos-12' || matrix.os == 'macos-13' || matrix.os == 'macos-14' || matrix.os == 'ubuntu-latest'}}
id: set-up-homebrew
uses: Homebrew/actions/setup-homebrew@master

- name: Set reusable strings
# Turn repeated input strings (such as the build output directory) into step outputs. These step outputs can be used throughout the workflow file.
id: strings
shell: bash
run: |
echo "build-output-dir=${{ github.workspace }}/cmake-build" >> "$GITHUB_OUTPUT"
echo "py-build-output-dir=${{ github.workspace }}/build" >> "$GITHUB_OUTPUT"
echo "CMAKE_C_COMPILER=${{ matrix.c_compiler }}" >> "$GITHUB_ENV"
echo "CMAKE_CXX_COMPILER=${{ matrix.cpp_compiler }}" >> "$GITHUB_ENV"
echo "CMAKE_OPTIONS=${{ matrix.cmake_extra_options }}" >> "$GITHUB_ENV"
- name: Install LLVM
if: ${{ matrix.os == 'macos-12' || matrix.os == 'macos-13' || matrix.os == 'macos-14' || matrix.os == 'ubuntu-latest' }}
run: |
brew install llvm ninja
echo "PATH=$(brew --prefix llvm)/bin:$PATH" >> "$GITHUB_ENV"

- name: Set up Homebrew
if: ${{ matrix.os == 'macos-12' || matrix.os == 'macos-13' || matrix.os == 'macos-14' || matrix.os == 'ubuntu-latest'}}
id: set-up-homebrew
uses: Homebrew/actions/setup-homebrew@master
- name: Set up Python
uses: actions/setup-python@v5
with:
cache: 'pip'
python-version: ${{ matrix.python_version }}

- name: Install LLVM
if: ${{ matrix.os == 'macos-12' || matrix.os == 'macos-13' || matrix.os == 'macos-14' || matrix.os == 'ubuntu-latest' }}
run: |
brew install llvm ninja
echo "PATH=$(brew --prefix llvm)/bin:$PATH" >> "$GITHUB_ENV"
- name: Install dependencies
run: pip install .
shell: bash

- name: Set up Python
uses: actions/setup-python@v5
with:
cache: 'pip'
python-version: ${{ matrix.python_version }}
- name: Python Build
run: python setup.py build
shell: bash

- name: Install packages
run: pip install -r requirements.txt

- name: Python Build
run: python setup.py build

- name: Upload Build Artifact (Python)
uses: actions/[email protected]
with:
# Artifact name
name: PyBuild ${{ matrix.os }}-cp${{ matrix.python_version }}
# A file, directory or wildcard pattern that describes what to upload
path: ${{ steps.strings.outputs.py-build-output-dir }}
- name: Upload Build Artifact (Python)
uses: actions/[email protected]
with:
# Artifact name
name: PyBuild ${{ matrix.os }}-cp${{ matrix.python_version }}
# A file, directory or wildcard pattern that describes what to upload
path: ${{ steps.strings.outputs.py-build-output-dir }}
Loading
Loading