Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Test automation] Fix pytest collection errors and schedule daily CI/CD workflow run #737

Open
wants to merge 59 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
59 commits
Select commit Hold shift + click to select a range
e972a04
Add workflow for GitHub Actions
ConnorHack Nov 6, 2024
9d56361
Update torchrun to pytest and checkpoint directory env var
ConnorHack Nov 6, 2024
02663d8
Remove Workplace Message notification Action
ConnorHack Nov 6, 2024
e200a42
Add aiosqlite as library dependency
ConnorHack Nov 6, 2024
307b2ca
Add RANK environment variable for torch.distributed
ConnorHack Nov 6, 2024
274b5a0
Add WORLD_SIZE env var for torch.distributed
ConnorHack Nov 6, 2024
f6b658e
Add torchrun specially for meta_reference
ConnorHack Nov 6, 2024
00829b8
Rename checkpoint directory env var
ConnorHack Nov 6, 2024
91b37c8
Remove old environment variables
ConnorHack Nov 6, 2024
b7f18eb
Merge branch 'meta-llama:main' into main
ConnorHack Nov 7, 2024
69215b6
Update checkpoint directory to include model
ConnorHack Nov 7, 2024
edfd92d
Merge branch 'meta-llama:main' into main
ConnorHack Nov 20, 2024
2795731
Update model name for mete-reference template
ConnorHack Nov 20, 2024
a5acb59
Merge branch 'meta-llama:main' into main
ConnorHack Nov 20, 2024
0ec4ddd
Merge branch 'meta-llama:main' into main
ConnorHack Nov 20, 2024
05f1041
Merge branch 'meta-llama:main' into main
ConnorHack Nov 21, 2024
16ffe19
Account for if a permitted model is None
ConnorHack Nov 21, 2024
490c5fb
Undo None check and temporarily move if model check before builder
ConnorHack Nov 21, 2024
0930234
Merge branch 'meta-llama:main' into main
ConnorHack Nov 21, 2024
19bc7e8
Merge branch 'meta-llama:main' into main
ConnorHack Nov 21, 2024
94bfd9a
Merge branch 'meta-llama:main' into main
ConnorHack Nov 21, 2024
318c988
Pre-emptively test llama stack RC
ConnorHack Nov 21, 2024
046eec9
Remove testing llama-stack RC
ConnorHack Nov 22, 2024
4136acc
Merge branch 'meta-llama:main' into main
ConnorHack Nov 22, 2024
4968797
Dynamically change provider in tests
ConnorHack Nov 22, 2024
25e23a1
Add debug statement for PROVIDER_ID
ConnorHack Nov 22, 2024
143e91f
Add manual provider back for testing
ConnorHack Nov 22, 2024
377896a
Remove testing llama-stack RC
ConnorHack Nov 22, 2024
1481a67
Test new provider name
ConnorHack Nov 22, 2024
0e9ed36
Remove unnecessary env vars
ConnorHack Nov 22, 2024
9c07e01
Fix syntax error
ConnorHack Nov 22, 2024
7f5e0dd
Refactor test run to support shorthand model names
ConnorHack Nov 22, 2024
d1d8f85
Update checkpointd directory setting
ConnorHack Nov 22, 2024
cbd69d0
Clean up checkpoint directory setting
ConnorHack Nov 22, 2024
8f60a3a
Clean up job names
ConnorHack Nov 22, 2024
0717104
Try shortening test formula
ConnorHack Nov 22, 2024
217f81b
Merge branch 'meta-llama:main' into main
ConnorHack Nov 25, 2024
e428b82
Revert test formula
ConnorHack Nov 25, 2024
cd0c80d
Add env vars debug printout
ConnorHack Nov 25, 2024
1912ff2
Temporarily make repo point to fork for PR testing
ConnorHack Nov 25, 2024
ac19743
Revert fork target back to main
ConnorHack Nov 25, 2024
8d83759
Add MODEL_CHECKPOINT_DIR check after update
ConnorHack Nov 25, 2024
bbea9bc
Revert provider / inference config back to mainline
ConnorHack Nov 25, 2024
fa4ce70
Merge branch 'meta-llama:main' into main
ConnorHack Dec 3, 2024
4b2e9d7
Merge branch 'meta-llama:main' into main
ConnorHack Dec 4, 2024
dc4e755
Temporarily disable PR runs until manual testing is complete
ConnorHack Dec 4, 2024
eeb8416
Merge branch 'meta-llama:main' into main
ConnorHack Jan 7, 2025
e5b3ff6
Merge branch 'meta-llama:main' into main
ConnorHack Jan 8, 2025
7e46e93
Update cicd branch to test removing ollama dependency
ConnorHack Dec 4, 2024
d2cef84
Fix renaming of checkout_reference from branch
ConnorHack Jan 8, 2025
5c14fdd
Test new import for SqliteKVStoreConfig
ConnorHack Jan 8, 2025
dc801e8
Merge branch 'meta-llama:main' into main
ConnorHack Jan 8, 2025
5942dff
Fix other ImportErrors
ConnorHack Jan 9, 2025
488b5d2
Update actions artifact versioning
ConnorHack Jan 9, 2025
0ac4d2f
Merge branch 'meta-llama:main' into main
ConnorHack Jan 9, 2025
df072c8
Schedule manual dispatch of workflow via cron scheduling
ConnorHack Jan 9, 2025
fa99fbf
Remove ollama dependency
ConnorHack Jan 9, 2025
e5da50f
Modify test call for test_model_registration.py within workflow
ConnorHack Jan 9, 2025
b61108c
Revert workflow back to original status for PR#737
ConnorHack Jan 10, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 6 additions & 7 deletions .github/workflows/gha_workflow_llama_stack_tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,10 @@ on:
# - 'llama_stack/**/*.py'
# - 'tests/**/*.py'

# Schedule cron job at 2:30 am EST every day of the week.
# Will run a manual workflow off of 'main' branch.
schedule:
- cron: '30 7 * * *'
workflow_dispatch:
inputs:
runner:
Expand Down Expand Up @@ -142,7 +146,7 @@ jobs:
id: checkout_repo
uses: actions/checkout@v4
with:
ref: ${{ inputs.branch }}
ref: ${{ inputs.checkout_reference }}

- name: "[DEBUG] Content of the repository after checkout"
id: debug_content_after_checkout
Expand Down Expand Up @@ -206,11 +210,6 @@ jobs:
sudo apt update -y
sudo apt install -y python3 python3-pip npm wget

- name: "Installing packages with 'curl'"
id: install_curl
run: |
curl -fsSL https://ollama.com/install.sh | sh

- name: "Installing packages with 'wget'"
id: install_wget
run: |
Expand Down Expand Up @@ -310,7 +309,7 @@ jobs:
- name: "PR - Upload Test Summary"
id: pr_test_summary_upload
if: github.event_name == 'pull_request_target'
uses: actions/upload-artifact@v3
uses: actions/upload-artifact@v4
with:
name: test-summary
path: test-summary.md
Expand Down
4 changes: 3 additions & 1 deletion llama_stack/providers/tests/agents/test_persistence.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,9 @@
from llama_stack.apis.agents import AgentConfig, Turn
from llama_stack.apis.inference import SamplingParams, UserMessage
from llama_stack.providers.datatypes import Api
from llama_stack.providers.utils.kvstore import kvstore_impl, SqliteKVStoreConfig
from llama_stack.providers.utils.kvstore import kvstore_impl
from llama_stack.providers.utils.kvstore.config import SqliteKVStoreConfig

from .fixtures import pick_inference_model

from .utils import create_agent_session
Expand Down
3 changes: 2 additions & 1 deletion llama_stack/providers/tests/inference/test_embeddings.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,8 @@

import pytest

from llama_stack.apis.inference import EmbeddingsResponse, ModelType
from llama_stack.apis.inference import EmbeddingsResponse
from llama_stack.apis.models import ModelType

# How to run this test:
# pytest -v -s llama_stack/providers/tests/inference/test_embeddings.py
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
# the root directory of this source tree.
import pytest

from llama_stack.apis.common.type_system import JobStatus
from llama_stack.apis.common.job_types import JobStatus
from llama_stack.apis.post_training import (
Checkpoint,
DataConfig,
Expand Down
Loading