-
Notifications
You must be signed in to change notification settings - Fork 41
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add simple and full knowledge pipeline functional tests #466
base: main
Are you sure you want to change the base?
Conversation
171ec5c
to
0cd8f58
Compare
0cd8f58
to
9e3133e
Compare
9e3133e
to
30515f0
Compare
30515f0
to
c78ad6e
Compare
c78ad6e
to
594702e
Compare
594702e
to
10e95a1
Compare
10e95a1
to
9a8f776
Compare
0c981d2
to
c518c82
Compare
c518c82
to
b451c06
Compare
b451c06
to
f7edd2b
Compare
This is a port of the old `scripts/test_knowledge.py` into functional tests that we can run with CI. These tests need to run on a GPU, so are marked as 'gpu' in pytest and only execute with a new py3-functional-gpu tox environment. This also adds a new workflow file to execute these tests on a GPU runner. Signed-off-by: Ben Browning <[email protected]>
f7edd2b
to
56ee2fe
Compare
Ok, almost ready to move this out of draft state. For now this only tests our end-to-end SDG simple and full pipelines for knowledge, but will be expanded in the future to cover skills, PDF knowledge docs, etc. The main focus here is to stub in an SDG CI job that runs with GPUs but does not have instructlab/instructlab installed, so that we can start working on a test suite for SDG independent of running InstructLab's e2e tests. We have SDG features to test that won't be exposed via the Tagging in @nathan-weinberg and @courtneypacheco as my go-to CI experts. If you have any general feedback on direction or changes to the approach I've taken here to add a new CI job for SDG, I'd love to hear it. To move this out of draft state I'll want to remove the temporary |
This is a port of the old
scripts/test_knowledge.py
into functional tests that we can run with CI. These tests need to run on a GPU, so are marked as 'gpu' in pytest and only execute with a new py3-functional-gpu tox environment. This also adds a new workflow file to execute these tests on a GPU runner.