Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add performance benchmarks #45

Open
peddybeats opened this issue Apr 6, 2020 · 2 comments
Open

Add performance benchmarks #45

peddybeats opened this issue Apr 6, 2020 · 2 comments

Comments

@peddybeats
Copy link
Owner

Deliverables:

  1. A script that brings up the app, runs for N seconds and averages the # of inferences per second
  2. Configuration that runs this as a part of CI to ensure changes don't cause performance regressions
  3. A table in README that list inferences per second on {macbook pro, pixel 3, iphone 11, thinkpad} x {wasm, webgl, cpu}

Note, in CI, the webcam feed may need to be mocked using pre-recorded footage; if it proves to be too challenging, we may have to split 2 into an issue for later.

@Berkmann18
Copy link
Collaborator

Should we have Lighthouse as part of the CI? That way, metrics like performance and a11y can be automatically tested.

@peddybeats
Copy link
Owner Author

peddybeats commented Apr 7, 2020

I've never used Lighthouse, but just did a quick search and seems like an excellent fit!
https://github.com/treosh/lighthouse-ci-action

Also just noting in case it's useful, this is the page tfjs uses to do benchmarks:
https://github.com/tensorflow/tfjs/blob/master/tfjs-core/benchmarks/index.html

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants