Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: voicevox_onnxruntimeのprepare.bashを使う #62

Merged
merged 1 commit into from
Dec 30, 2024
Merged

chore: voicevox_onnxruntimeのprepare.bashを使う #62

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 14 additions & 41 deletions .github/workflows/build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,6 @@ jobs:
--cmake_extra_defines
CMAKE_SYSTEM_NAME=Windows
CMAKE_SYSTEM_PROCESSOR=x86_64 # required for `cpuinfo`
Rust_CARGO_TARGET=x86_64-pc-windows-msvc
result_dir: build/Release
release_config: Release
- artifact_name: ${{ inputs.target || 'onnxruntime' }}-win-x64-dml
Expand All @@ -63,7 +62,6 @@ jobs:
--cmake_extra_defines
CMAKE_SYSTEM_NAME=Windows
CMAKE_SYSTEM_PROCESSOR=x86_64 # required for `cpuinfo`
Rust_CARGO_TARGET=x86_64-pc-windows-msvc
result_dir: build/Release
release_config: Release
- artifact_name: ${{ inputs.target || 'onnxruntime' }}-win-x64-cuda
Expand All @@ -79,7 +77,6 @@ jobs:
--cmake_extra_defines
CMAKE_SYSTEM_NAME=Windows
CMAKE_SYSTEM_PROCESSOR=x86_64 # required for `cpuinfo`
Rust_CARGO_TARGET=x86_64-pc-windows-msvc
result_dir: build/Release
release_config: Release
# https://github.com/VOICEVOX/onnxruntime-builder/issues/59
Expand All @@ -91,7 +88,6 @@ jobs:
# --cmake_extra_defines
# CMAKE_SYSTEM_NAME=Windows
# CMAKE_SYSTEM_PROCESSOR=x86 # required for `cpuinfo`
# Rust_CARGO_TARGET=i686-pc-windows-msvc
# result_dir: build/Release
# release_config: Release
- artifact_name: ${{ inputs.target || 'onnxruntime' }}-linux-x64
Expand All @@ -101,7 +97,6 @@ jobs:
--cmake_extra_defines
CMAKE_SYSTEM_NAME=Linux
CMAKE_SYSTEM_PROCESSOR=x86_64 # required for `cpuinfo`
Rust_CARGO_TARGET=x86_64-unknown-linux-gnu
result_dir: build
release_config: Release
- artifact_name: ${{ inputs.target || 'onnxruntime' }}-linux-x64-cuda
Expand All @@ -115,7 +110,6 @@ jobs:
--cmake_extra_defines
CMAKE_SYSTEM_NAME=Linux
CMAKE_SYSTEM_PROCESSOR=x86_64 # required for `cpuinfo`
Rust_CARGO_TARGET=x86_64-unknown-linux-gnu
result_dir: build
release_config: Release
- artifact_name: ${{ inputs.target || 'onnxruntime' }}-linux-armhf
Expand All @@ -130,7 +124,6 @@ jobs:
--cmake_extra_defines
CMAKE_SYSTEM_NAME=Linux
CMAKE_SYSTEM_PROCESSOR=armv7l # required for `cpuinfo`
Rust_CARGO_TARGET=armv7-unknown-linux-gnueabihf
result_dir: build
release_config: Release
- artifact_name: ${{ inputs.target || 'onnxruntime' }}-linux-arm64
Expand All @@ -145,7 +138,6 @@ jobs:
--cmake_extra_defines
CMAKE_SYSTEM_NAME=Linux
CMAKE_SYSTEM_PROCESSOR=aarch64 # required for `cpuinfo`
Rust_CARGO_TARGET=aarch64-unknown-linux-gnu
result_dir: build
release_config: Release
- artifact_name: ${{ inputs.target || 'onnxruntime' }}-osx-arm64
Expand All @@ -156,7 +148,6 @@ jobs:
CMAKE_SYSTEM_NAME=Darwin
CMAKE_OSX_ARCHITECTURES=arm64
CMAKE_SYSTEM_PROCESSOR=arm64 # required for `cpuinfo`
Rust_CARGO_TARGET=aarch64-apple-darwin
result_dir: build
release_config: Release
- artifact_name: ${{ inputs.target || 'onnxruntime' }}-osx-x86_64
Expand All @@ -167,7 +158,6 @@ jobs:
CMAKE_SYSTEM_NAME=Darwin
CMAKE_OSX_ARCHITECTURES=x86_64
CMAKE_SYSTEM_PROCESSOR=x86_64 # required for `cpuinfo`
Rust_CARGO_TARGET=x86_64-apple-darwin
result_dir: build
release_config: Release
- artifact_name: ${{ inputs.target || 'onnxruntime' }}-android-x64
Expand All @@ -179,7 +169,6 @@ jobs:
--cmake_extra_defines
CMAKE_SYSTEM_NAME=Android
CMAKE_SYSTEM_PROCESSOR=x86_64 # required for `cpuinfo`
Rust_CARGO_TARGET=x86_64-linux-android
result_dir: build
release_config: Release
- artifact_name: ${{ inputs.target || 'onnxruntime' }}-android-arm64
Expand All @@ -191,7 +180,6 @@ jobs:
--cmake_extra_defines
CMAKE_SYSTEM_NAME=Android
CMAKE_SYSTEM_PROCESSOR=aarch64 # required for `cpuinfo`
Rust_CARGO_TARGET=aarch64-linux-android
result_dir: build
release_config: Release
- artifact_name: ${{ inputs.target || 'onnxruntime' }}-ios-arm64
Expand All @@ -205,8 +193,6 @@ jobs:
--osx_arch arm64
--apple_deploy_target 16.0
--path_to_protoc_exe /usr/local/opt/protobuf@21/bin/protoc # Homebrewで入れた`protobuf@21`
--cmake_extra_defines
Rust_CARGO_TARGET=aarch64-apple-ios
result_dir: build/Release
release_config: Release-iphoneos
- artifact_name: ${{ inputs.target || 'onnxruntime' }}-ios-sim-arm64
Expand All @@ -220,8 +206,6 @@ jobs:
--osx_arch arm64
--apple_deploy_target 16.0
--path_to_protoc_exe /usr/local/opt/protobuf@21/bin/protoc # Homebrewで入れた`protobuf@21`
--cmake_extra_defines
Rust_CARGO_TARGET=aarch64-apple-ios-sim
result_dir: build/Release
release_config: Release-iphonesimulator
- artifact_name: ${{ inputs.target || 'onnxruntime' }}-ios-sim-x86_64
Expand All @@ -235,8 +219,6 @@ jobs:
--osx_arch x86_64
--apple_deploy_target 16.0
--path_to_protoc_exe /usr/local/opt/protobuf@21/bin/protoc # Homebrewで入れた`protobuf@21`
--cmake_extra_defines
Rust_CARGO_TARGET=x86_64-apple-ios
result_dir: build/Release
release_config: Release-iphonesimulator

Expand Down Expand Up @@ -394,33 +376,24 @@ jobs:
echo "CUDNN_HOME=$cudnn_path" >> "$GITHUB_ENV"
- name: Extract Rust toolchain and target triple
id: rust-toolchain-and-target-triple
- name: Run prepare.bash
id: prepare
if: steps.cache-build-result.outputs.cache-hit != 'true' && env.TARGET_LIBRARY == 'voicevox_onnxruntime'
Hiroshiba marked this conversation as resolved.
Show resolved Hide resolved
run: ./prepare.bash 1>${{ matrix.artifact_name }}_stdout.txt 2>${{ matrix.artifact_name }}_stderr.txt

- name: Encrypt the preparation log
if: failure() && steps.prepare.outcome == 'failure'
Comment on lines +384 to +385
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

（ただのリファクタリング案です）

これ、最後のステップで１回だけ実行するのでも良さそうですね！

すべての関連stepで./${{ matrix.artifact_name }}_{stdout,stderr}.txtに>>で追記していって、最後のステップでfailure()のときに./${{ matrix.artifact_name }}_{stdout,stderr}.txtをアップロードするようにしたらコードが１回で済むので便利そう。

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. checkout (今は> /dev/null 2>&1)
  2. prepare.bash
  3. ビルド

の三つがあるので、確かにまとめるとすっきりしそうですね。まあその時は見通し的に1.と2.のstep自体を3.の直前に持って来た方がよさそうなので、別PRの方がよさそうかなと思っています。

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ですね！　別PRがより良いと思います！

run: |
build_opts=(
${{ matrix.build_opts }}
)
echo "toolchain=$(cat ./rust-toolchain)" >> "$GITHUB_OUTPUT"
echo "target=$(sed -E 's/.*Rust_CARGO_TARGET=([a-z0-9_-]+).*/\1/' <<< "${build_opts[*]}")" >> "$GITHUB_OUTPUT"
for file in ./${{ matrix.artifact_name }}_{stdout,stderr}.txt; do
gpg -ef ./builder/build-log-pub.asc "$file"
done
- name: Set up Rust
if: steps.cache-build-result.outputs.cache-hit != 'true' && env.TARGET_LIBRARY == 'voicevox_onnxruntime'
uses: dtolnay/rust-toolchain@master
- name: Upload the encrypted preparation log
if: failure() && steps.prepare.outcome == 'failure'
uses: actions/upload-artifact@v4
with:
toolchain: ${{ steps.rust-toolchain-and-target-triple.outputs.toolchain }}
targets: ${{ steps.rust-toolchain-and-target-triple.outputs.target }}

- name: Install cargo-binstall
if: steps.cache-build-result.outputs.cache-hit != 'true' && env.TARGET_LIBRARY == 'voicevox_onnxruntime'
uses: taiki-e/install-action@cargo-binstall

- name: Install cxxbridge-cmd
if: steps.cache-build-result.outputs.cache-hit != 'true' && env.TARGET_LIBRARY == 'voicevox_onnxruntime'
run: |
md=$(cargo metadata --manifest-path ./vv_bin/Cargo.toml --format-version 1)
version=$(jq -r '.packages[] | select(.name == "cxx").version' <<< "$md")
cargo binstall "cxxbridge-cmd@$version" --no-confirm --log-level debug
name: preparation_${{ matrix.artifact_name }}
path: ${{ matrix.artifact_name }}_*.txt.gpg

- name: Configure build environment for non-x86_64 Linux
if: steps.cache-build-result.outputs.cache-hit != 'true' && startsWith(matrix.os, 'ubuntu') && matrix.linux_cross_arch
Expand Down