Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Java API:ビルドWorkflowを追加 #621

Merged
merged 40 commits into from
Oct 9, 2023
Merged
Show file tree
Hide file tree
Changes from 33 commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
2979fb9
Change: build-android.gradleに分離
sevenc-nanashi Sep 14, 2023
92555bd
Fix: Android版の構成を色々修正
sevenc-nanashi Sep 14, 2023
fb00418
Fix: 依存関係周りを修正
sevenc-nanashi Sep 14, 2023
3bd192c
Code: gradlew spotlessApply
sevenc-nanashi Sep 14, 2023
4186e79
Delete: 嘘コメントを削除
sevenc-nanashi Sep 14, 2023
55fc8ec
Change: 環境変数をOSとDEVICEに分離
sevenc-nanashi Sep 15, 2023
5baf16e
Merge: remote -> local
sevenc-nanashi Sep 15, 2023
81283e7
Merge: main -> fix/android-build
sevenc-nanashi Sep 15, 2023
5db3d3d
Change: バージョン周りを共通化
sevenc-nanashi Sep 15, 2023
a50ecbd
Fix: 依存関係を修正
sevenc-nanashi Sep 16, 2023
f092e36
Fix: 変数名を修正
sevenc-nanashi Sep 16, 2023
d3864d3
Add: Javaのビルドを追加
sevenc-nanashi Sep 17, 2023
3e52b1e
Add: featureを追加
sevenc-nanashi Sep 17, 2023
0c3b789
Fix: targetを修正
sevenc-nanashi Sep 17, 2023
04648de
Fix: wheelを一つだけに
sevenc-nanashi Sep 17, 2023
c1b911e
Change: keyを変更
sevenc-nanashi Sep 17, 2023
8f65c16
Fix: GH_TOKENを指定
sevenc-nanashi Sep 17, 2023
7114365
Fix: restore-keysを修正
sevenc-nanashi Sep 17, 2023
654449f
Fix: keysにtargetを含めるように
sevenc-nanashi Sep 17, 2023
bf27296
Fix: actionlintに従って修正
sevenc-nanashi Sep 17, 2023
2aa021b
Fix: ダウンロード周りを修正
sevenc-nanashi Sep 17, 2023
4594d6f
Add: treeを追加
sevenc-nanashi Sep 17, 2023
41c0988
Fix: 文法を修正
sevenc-nanashi Sep 17, 2023
eae0b85
Fix: ディレクトリ作成周りを修正
sevenc-nanashi Sep 17, 2023
15f23cf
Fix: globが適用されるように
sevenc-nanashi Sep 18, 2023
78ea083
Fix: read忘れ
sevenc-nanashi Sep 18, 2023
02c4be5
Change: Java 17に
sevenc-nanashi Sep 18, 2023
264235c
Fix: android版の校正を修正
sevenc-nanashi Sep 18, 2023
4db59ff
Change: libstdc++をstaticに
sevenc-nanashi Sep 18, 2023
502d06e
Merge: main -> add/java-build-workflow
sevenc-nanashi Sep 19, 2023
f958e5d
Merge: main -> add/java-build-workflow
sevenc-nanashi Sep 19, 2023
af7d583
Fix: SC2086を修正
sevenc-nanashi Sep 19, 2023
0c220de
Fix: libc++_shared.soを追加
sevenc-nanashi Sep 19, 2023
0e88e3c
Fix: キャッシュ周りを修正
sevenc-nanashi Sep 20, 2023
8c49cb5
Delete: Android版以外を削除
sevenc-nanashi Sep 30, 2023
53af4ff
Fix: includes -> contains
sevenc-nanashi Sep 30, 2023
d236789
Fix: Fix condition
sevenc-nanashi Sep 30, 2023
83f4d63
Change: set-versionの対象を変更
sevenc-nanashi Oct 8, 2023
bc7e246
Change: build_java_api -> build_java_package
sevenc-nanashi Oct 8, 2023
f9743f2
Delete: デバッグの痕跡を削除
sevenc-nanashi Oct 9, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
206 changes: 204 additions & 2 deletions .github/workflows/build_and_deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -257,6 +257,16 @@ jobs:
run: |
cargo set-version "$VERSION" --exclude voicevox_core_python_api --exclude download --exclude xtask
if ${{ !!matrix.whl_local_version }}; then cargo set-version "$VERSION+"${{ matrix.whl_local_version }} -p voicevox_core_python_api; fi
- name: cache target
uses: actions/cache@v3
if: github.event.inputs.is_production != 'true'
with:
path: |
~/.cargo
target
key: ${{ matrix.artifact_name }}-rust-${{ hashFiles('**/Cargo.lock') }}
restore-keys: |
${{ matrix.artifact_name }}-rust-
Copy link
Member

@Hiroshiba Hiroshiba Sep 20, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

結構早くなる感じでしょうか?
(キャッシュ、すぐバグにつながる印象があるのでちょっと避けたいなという思いがあります 😇 )

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

5分くらいですね。
ビルドにスピードが求められるのはWorkflowテストくらいなので製品版ビルドでは無効化しています

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

なるほどです、確かに5分短縮は嬉しいのですが・・・。
キャッシュのキーってこのリポジトリ全てで共通なグローバル変数で、かつ未来方向と過去方向にもちゃんと一意になっているかを意識し続けないといけないため、相当気を使うんですよね・・・。
これ以降のコードがキャッシュファイルがあることを想定して書く必要もあり、かつそうなっていない気がするので、できれば避けたい気持ちがあります。

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://github.com/Swatinem/rust-cacheではなくactions/cacheを使っている理由はあったりしますでしょうか?

(キーとしてはCargo.lock以外にもrust-toolchainだったりあるかなと思った次第で、rust-cacheはそれも含んでくれます)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ないですね、そっちにしてみました。(まぁキャッシュ自体を消す可能性もありますが)

- name: build voicevox_core_c_api
shell: bash
run: |
Expand Down Expand Up @@ -284,9 +294,21 @@ jobs:
else
build > /dev/null 2>&1
fi
echo "whl=$(find ./target/wheels -type f)" >> "$GITHUB_OUTPUT"
ls -l ./target/wheels
sevenc-nanashi marked this conversation as resolved.
Show resolved Hide resolved
echo "whl=$(find ./target/wheels -type f | head -1)" >> "$GITHUB_OUTPUT"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

これなんで必要になった感じでしょう 👀

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

分からないですね…なぜか落ちるようになりました

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

もしかしたらキャッシュが原因かなと思いました。やっぱキャッシュむずい!!!!!!!!!!!!!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

あー!その可能性はありますね
上でrm -rf ./target/wheels/*を走らせれば直りはするでしょうが、まぁ怖くなってきました

env:
ORT_USE_CUDA: ${{ matrix.use_cuda }}
- name: build voicevox_core_java_api
if: "!endsWith(matrix.target, 'ios')"
run: |
function build() {
cargo build -p voicevox_core_java_api -vv --features ${{ matrix.features }}, --target ${{ matrix.target }} --release
}
if ${{ github.event.inputs.is_production != 'true' }}; then
build
else
build > /dev/null 2>&1
fi
- name: Set ASSET_NAME env var
run: echo "ASSET_NAME=voicevox_core-${{ matrix.artifact_name }}-${{ env.VERSION }}" >> "$GITHUB_ENV"
- name: Organize artifact
Expand All @@ -301,6 +323,11 @@ jobs:
cp -v README.md "artifact/${{ env.ASSET_NAME }}/README.txt"
cp -vr model "artifact/${{ env.ASSET_NAME }}/"
echo "${{ env.VERSION }}" > "artifact/${{ env.ASSET_NAME }}/VERSION"

mkdir java_artifact
cp -v target/${{ matrix.target }}/release/libvoicevox_core_java_api.so java_artifact/ || true
cp -v target/${{ matrix.target }}/release/libvoicevox_core_java_api.dylib java_artifact/ || true
cp -v target/${{ matrix.target }}/release/voicevox_core_java_api.dll java_artifact/ || true
- name: Code signing (Windows)
if: startsWith(matrix.os, 'windows') && github.event.inputs.code_signing == 'true'
run: |
Expand All @@ -310,7 +337,7 @@ jobs:
CERT_PASSWORD: ${{ secrets.CERT_PASSWORD }}
- name: Upload artifact to build XCFramework
if: contains(matrix.target, 'ios')
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: voicevox_core-${{ matrix.target }}
path: artifact/${{ env.ASSET_NAME }}
Expand All @@ -336,6 +363,12 @@ jobs:
files: |-
${{ steps.build-voicevox-core-python-api.outputs.whl }}
target_commitish: ${{ github.sha }}
- name: Upload voicevox_core_java_api artifact
if: env.VERSION != '0.0.0' && !contains(matrix.target, 'ios')
uses: actions/upload-artifact@v3
with:
name: voicevox_core_java_api-${{ matrix.artifact_name }}
path: java_artifact

build_xcframework:
if: ${{ !(github.event_name != 'release' && github.event_name != 'workflow_dispatch') }} # !env.IS_SIMPLE_TEST と同じ
Expand Down Expand Up @@ -391,6 +424,175 @@ jobs:
${{ env.ASSET_NAME }}.zip
target_commitish: ${{ github.sha }}

build_java_api:
sevenc-nanashi marked this conversation as resolved.
Show resolved Hide resolved
runs-on: ubuntu-latest
if: ${{ !(github.event_name != 'release' && github.event_name != 'workflow_dispatch') }} # !env.IS_SIMPLE_TEST と同じ
needs:
- build_and_deploy
steps:
- uses: actions/checkout@v3
- name: Set up Rust
uses: ./.github/actions/rust-toolchain-from-file
- name: Set up Java
uses: actions/setup-java@v2
with:
java-version: "17"
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Android Gradle Pluginは17以上じゃないと動かなかったので。

distribution: "adopt"
- uses: nttld/setup-ndk@v1
id: setup-ndk
with:
ndk-version: r25b
- name: Install cargo-binstall
uses: taiki-e/install-action@cargo-binstall
- name: Install cargo-edit
run: cargo binstall cargo-edit@^0.11 --no-confirm
- name: set cargo version
run: |
cargo set-version "$VERSION" --exclude voicevox_core_python_api --exclude download --exclude xtask
sevenc-nanashi marked this conversation as resolved.
Show resolved Hide resolved
- name: "Download artifact (windows-x64-cpu)"
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

かなり混沌としてるので何とかしたい(何とかできなさそうな感じはする)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

github cli使ってダウンロードするところからgradle叩くところまでをシェルスクリプトにして実行、とかですかねぇ 😇
(本当はこっちのほうがローカルでも動かせるので良い)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

github cliは実行中のArtifactを落とせないっぽみがありました(記憶違いかも)

uses: actions/download-artifact@v3
with:
name: voicevox_core_java_api-windows-x64-cpu
path: artifact/windows-x64-cpu

- name: "Download artifact (windows-x64-directml)"
uses: actions/download-artifact@v3
with:
name: voicevox_core_java_api-windows-x64-directml
path: artifact/windows-x64-directml

- name: "Download artifact (windows-x64-cuda)"
uses: actions/download-artifact@v3
with:
name: voicevox_core_java_api-windows-x64-cuda
path: artifact/windows-x64-cuda

- name: "Download artifact (windows-x86-cpu)"
uses: actions/download-artifact@v3
with:
name: voicevox_core_java_api-windows-x86-cpu
path: artifact/windows-x86-cpu

- name: "Download artifact (linux-x64-cpu)"
uses: actions/download-artifact@v3
with:
name: voicevox_core_java_api-linux-x64-cpu
path: artifact/linux-x64-cpu

- name: "Download artifact (linux-x64-gpu)"
uses: actions/download-artifact@v3
with:
name: voicevox_core_java_api-linux-x64-gpu
path: artifact/linux-x64-gpu

- name: "Download artifact (linux-arm64-cpu)"
uses: actions/download-artifact@v3
with:
name: voicevox_core_java_api-linux-arm64-cpu
path: artifact/linux-arm64-cpu

- name: "Download artifact (android-arm64-cpu)"
uses: actions/download-artifact@v3
with:
name: voicevox_core_java_api-android-arm64-cpu
path: artifact/android-arm64-cpu

- name: "Download artifact (android-x86_64-cpu)"
uses: actions/download-artifact@v3
with:
name: voicevox_core_java_api-android-x86_64-cpu
path: artifact/android-x86_64-cpu

- name: "Download artifact (osx-arm64-cpu)"
uses: actions/download-artifact@v3
with:
name: voicevox_core_java_api-osx-arm64-cpu
path: artifact/osx-arm64-cpu

- name: "Download artifact (osx-x64-cpu)"
uses: actions/download-artifact@v3
with:
name: voicevox_core_java_api-osx-x64-cpu
path: artifact/osx-x64-cpu
- name: Print tree
run: tree artifact

- name: Build voicevoxcore
run: |
cat <<EOF | while read -r line; do
windows-x64-cpu|windows-x64
windows-x86-cpu|windows-x86
linux-x64-cpu|linux-x64
linux-arm64-cpu|linux-arm64
osx-x64-cpu|macos-x64
osx-arm64-cpu|macos-arm64
EOF
IFS='|' read -r artifact_name target <<< "$line"
mkdir "crates/voicevox_core_java_api/lib/src/main/resources/dll/${target}/"
cp -v "artifact/$artifact_name"/* "crates/voicevox_core_java_api/lib/src/main/resources/dll/${target}/"
done

cd crates/voicevox_core_java_api
OS=desktop DEVICE=cpu gradle publishToMavenLocal

- name: Build voicevoxcore-directml
run: |
cat <<EOF | while read -r line; do
windows-x64-directml|windows-x64
EOF
IFS='|' read -r artifact_name target <<< "$line"
cp -v "artifact/$artifact_name"/* "crates/voicevox_core_java_api/lib/src/main/resources/dll/${target}/"
done

cd crates/voicevox_core_java_api
OS=desktop DEVICE=directml gradle publishToMavenLocal

- name: Build voicevoxcore-cuda
run: |
cat <<EOF | while read -r line; do
windows-x64-cuda|windows-x64
linux-x64-gpu|linux-x64
EOF
IFS='|' read -r artifact_name target <<< "$line"
cp -v "artifact/$artifact_name"/* "crates/voicevox_core_java_api/lib/src/main/resources/dll/${target}/"
done

cd crates/voicevox_core_java_api
OS=desktop DEVICE=cuda gradle publishToMavenLocal

- name: Build voicevoxcore-android
run: |
rm -rf crates/voicevox_core_java_api/lib/src/main/resources/dll
cat <<EOF | while read -r line; do
android-arm64-cpu|arm64-v8a
android-x86_64-cpu|x86_64
EOF
IFS='|' read -r artifact_name target <<< "$line"
mkdir "crates/voicevox_core_java_api/lib/src/main/resources/jniLibs/${target}/"
cp -v "artifact/$artifact_name"/* "crates/voicevox_core_java_api/lib/src/main/resources/jniLibs/${target}/"
done

cp ${{ steps.setup-ndk.outputs.ndk-path }}/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/aarch64-linux-android/libc++_shared.so crates/voicevox_core_java_api/lib/src/main/resources/jniLibs/arm64-v8a/
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

stdc++_shared.soがないとロード時にクラッシュするので。

cp ${{ steps.setup-ndk.outputs.ndk-path }}/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/x86_64-linux-android/libc++_shared.so crates/voicevox_core_java_api/lib/src/main/resources/jniLibs/x86_64/

cd crates/voicevox_core_java_api
OS=android DEVICE=cpu gradle publishToMavenLocal

- name: Package
run: |
cd ~/.m2/repository
rm -rf dev || true
zip -r /tmp/java_packages.zip .

- name: Upload to Release
if: env.VERSION != '0.0.0' && env.SKIP_UPLOADING_RELEASE_ASSET == '0'
uses: softprops/action-gh-release@v1
with:
prerelease: true
tag_name: ${{ env.VERSION }}
files: |-
/tmp/java_packages.zip
target_commitish: ${{ github.sha }}
download_test:
sevenc-nanashi marked this conversation as resolved.
Show resolved Hide resolved
needs: [config, build_and_deploy]
if: needs.config.outputs.deploy == 'true'
Expand Down
3 changes: 3 additions & 0 deletions crates/voicevox_core_java_api/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,9 @@ publish.workspace = true
[lib]
crate-type = ["cdylib"]

[features]
directml = ["voicevox_core/directml"]

[dependencies]
android_logger = "0.13.1"
anyhow.workspace = true
Expand Down
20 changes: 20 additions & 0 deletions crates/voicevox_core_java_api/lib/build-android.gradle
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
plugins {
id 'com.android.library' version '8.1.1'
id 'maven-publish'
id 'org.jetbrains.kotlin.android' version '1.9.10'
}

Expand Down Expand Up @@ -53,4 +54,23 @@ android {
jniLibs.srcDirs = ["./src/main/resources/jniLibs"]
}
}
publishing {
singleVariant("release") {
withJavadocJar()
}
}
}

publishing {
publications {
maven(MavenPublication) {
groupId = 'jp.hiroshiba.voicevoxcore'
artifactId = gradle.ext.targetDevice == 'cpu' ? 'voicevoxcore-android' : "voicevoxcore-android-${gradle.ext.targetDevice}"
version = gradle.ext.version

afterEvaluate {
from components.release
}
}
}
}
14 changes: 14 additions & 0 deletions crates/voicevox_core_java_api/lib/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
plugins {
// Apply the java-library plugin for API and implementation separation.
id 'java-library'
id 'maven-publish'
id "com.diffplug.spotless" version "6.20.0"
}

Expand Down Expand Up @@ -47,6 +48,8 @@ java {
toolchain {
languageVersion = JavaLanguageVersion.of(8)
}

withJavadocJar()
}
tasks.named('test') {
// Use JUnit Platform for unit tests.
Expand All @@ -61,3 +64,14 @@ spotless {
googleJavaFormat()
}
}

publishing {
publications {
maven(MavenPublication) {
groupId = 'jp.hiroshiba.voicevoxcore'
artifactId = gradle.ext.targetDevice == 'cpu' ? 'voicevoxcore' : "voicevoxcore-${gradle.ext.targetDevice}"
version = gradle.ext.version
from components.java
}
}
}