Setting up hardware testing

[mwasilew]

This ticket is about setting up testing on hardware. Below there is a list of things to setup in order to start testing:

1. LAVA lab
   This is already setup. We host qcm6490 (qcs6490) and it can run boot test using OE build

2. qa-reports group and project
   I created group qcom. It currently only contains a test project that has restricted visibility. What is the name of the project we want to keep the test results in?

3. Setup github integration for result reporting
   The idea of test result reporting is to push the test status to the commit directly. This avoids running a github action for the whole duration of the test job (plus waiting time if there is a queue). Once tests are completed, the status will be pushed using github API. This can be done for a commit (cron builds) or PR. On PR it shows as an additional check. For this integration to work it is required to setup a token with the following permissions:

   • Read access to metadata
   • Read and Write access to commit statuses

   Token needs to be uploaded to qa-reports

4. Store build artifacts where LAVA can download them from
   LAVA can download with additional headers to avoid exposing artifacts to unauthorized download. Password or token has to be stored in LAVA master.

5. Define test jobs
   I think we should start with boot test on qcs6490. More tests can be added later. I think at this point we can't test with ethernet due to missing firmware. This reduces the number of tests we can actually run.

6. Adding test submission action to the workflow
   This can be done after github token is setup. Submit action repository is located here: https://github.com/foundriesio/qa-reports-action. I will start a PR on that shortly.

[mwasilew]

@ricardosalveti @doanac

[ricardosalveti]

2. qa-reports group and project
   I created group qcom. It currently only contains a test project that has restricted visibility. What is the name of the project we want to keep the test results in?

We can probably use 'quic-yocto' as a way to indicate that these are jobs coming out of this org.

5. Define test jobs
   I think we should start with boot test on qcs6490. More tests can be added later. I think at this point we can't test with ethernet due to missing firmware. This reduces the number of tests we can actually run.

Yes, boot test is already a great start.

6. Adding test submission action to the workflow
   This can be done after github token is setup. Submit action repository is located here: https://github.com/foundriesio/qa-reports-action. I will start a PR on that shortly.

Thanks for starting this up.

[sbanerjee-quic]

@mwasilew @ricardosalveti how to view the dashboard that displays the run result for boot test?

[mwasilew]

@sbanerjee-quic results will be posted somewhere here: https://qa-reports.foundries.io/qcom/. Right now there is nothing as we haven't started running any tests yet.

[mwasilew]

Ah, I forgot to mention there should be a link to test results on each commit we test - PR and cron testing.

[mwasilew]

I think this is mostly done. The last bit is checking schema on the lava job templates. I'll submit a PR today. Point 2 is not implemented at the moment and reporting is done in github actions using https://github.com/EnricoMi/publish-unit-test-result-action

[mwasilew]

This is completed for meta-qcom-hwe. I guess we need to replicate the same setup for meta-qcom now.