Skip to content

Commit

Permalink
fix action, remove % axis
Browse files Browse the repository at this point in the history
  • Loading branch information
@capjamesg
capjamesg committed Dec 13, 2023
1 parent 443a9ef commit 9a0e155
Show file tree
Hide file tree
Showing 6 changed files with 243 additions and 38 deletions.
13 changes: 13 additions & 0 deletions .github/workflows/test.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -25,3 +25,16 @@ jobs:
run: python web.py
env:
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
- uses: actions/checkout@v4

- name: Create Pull Request
uses: peter-evans/create-pull-request@v5

# commit results to repository
- name: Commit results
run: |
git config --local user.email "[email protected]"
git config --local user.name "capjamesg"
git add results/*
git commit -m "Update results"
git push
70 changes: 36 additions & 34 deletions index.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -32,9 +32,6 @@
</head>
<body>
<div class="graph_paper">
<a class="logo_link" href="https://roboflow.com/">
<img src="./assets/roboflow_full_logo_color.png" class="logo" alt="Roboflow Logo" />
</a>
<header>
<h1>How's GPT-4 with Vision Doing?</h1>
<div class="header_text">
Expand All @@ -58,12 +55,12 @@ <h1>How's GPT-4 with Vision Doing?</h1>
<div class="feature_header" style="min-height: auto">
<div class="feature_header_text" style="gap: var(--spacing-sizing-4)">
<h2>Response Time</h2>
<p style="font-size: 16px; color: var(--gray-700)">Today, the average response time to receive results from our tests was <b>5.46s</b> per request.</p>
<p style="font-size: 16px; color: var(--gray-700)">Today, the average response time to receive results from our tests was <b>5.79 seconds</b> per request.</p>
<p class="subtitle">This number only accounts for requests made by this application.</p>
</div>
<div class="chart">
<div class="chart_box chart_box_green">
<p>5.46 s</p>
<p>5.79 s</p>
</div>
</div>
</div>
Expand All @@ -85,12 +82,12 @@ <h2>Counting</h2>
</div>
<div class="chart">
<div class="chart_box chart_box_red">
<p>0%</p>
<p>Fail</p>
</div>
</div>
</div>
<div class="request_summary">
<p class="result_text">Of the last 7 tests, conducted daily, this test has passed <b>14.0%</b> of the time with an average score of <b>14.0%</b>.</p>
<p class="result_text">Of the last 7 tests, conducted daily, this test has passed <b>14.0%</b> of the time.</p>
<p class="request_price"><i class="far fa-coins"></i>Today's request cost $0.008</p>
</div>
<div class="explainer_dropdown">
Expand Down Expand Up @@ -128,12 +125,12 @@ <h2>Object Detection</h2>
</div>
<div class="chart">
<div class="chart_box chart_box_red">
<p>2.0%</p>
<p>Fail</p>
</div>
</div>
</div>
<div class="request_summary">
<p class="result_text">Of the last 7 tests, conducted daily, this test has passed <b>0%</b> of the time with an average score of <b>10.0%</b>.</p>
<p class="result_text">Of the last 7 tests, conducted daily, this test has passed <b>0%</b> of the time.</p>
<p class="request_price"><i class="far fa-coins"></i>Today's request cost $0.009</p>
</div>
<div class="explainer_dropdown">
Expand All @@ -146,7 +143,7 @@ <h3><span class="explainer_icon far fa-comment-dots"></span>Prompt</h3>
<h3><span class="explainer_icon far fa-image"></span>Image</h3>
<img class="test_image" src="images/fruit.jpeg" alt="Image of the input into GPT-4" />
<h3><span class="explainer_icon far fa-sparkles"></span>Result</h3>
<pre>{'x': 0.68, 'y': 0.25, 'width': 0.1, 'height': 0.25}</pre>
<pre>{'x': 0.3, 'y': 0.25, 'width': 0.18, 'height': 0.4}</pre>
<h3><span class="explainer_icon far fa-microscope"></span>Method</h3>
<pre class="test_method">We provide GPT-4V with an image with a known object. We ask it to provide a normalized bounding box of the object and for scoring, we calculate the intersection over union (IOU) between the predicted bounding box and the correct bounding box.</pre>
</div>
Expand All @@ -163,12 +160,12 @@ <h2>Graph Understanding</h2>
</div>
<div class="chart">
<div class="chart_box chart_box_red">
<p>82.0%</p>
<p>Fail</p>
</div>
</div>
</div>
<div class="request_summary">
<p class="result_text">Of the last 6 tests, conducted daily, this test has passed <b>0%</b> of the time with an average score of <b>81.0%</b>.</p>
<p class="result_text">Of the last 7 tests, conducted daily, this test has passed <b>0%</b> of the time.</p>
<p class="request_price"><i class="far fa-coins"></i>Today's request cost $0.01</p>
</div>
<div class="explainer_dropdown">
Expand All @@ -183,10 +180,10 @@ <h3><span class="explainer_icon far fa-image"></span>Image</h3>
<h3><span class="explainer_icon far fa-sparkles"></span>Result</h3>
<pre>```json
{
"A": {"quantity": 8, "price": 5},
"B": {"quantity": 20, "price": 20},
"C": {"quantity": 28, "price": 30},
"D": {"quantity": 40, "price": 45}
"A": {"quantity": 15, "price": 15},
"B": {"quantity": 22, "price": 23},
"C": {"quantity": 30, "price": 33},
"D": {"quantity": 42, "price": 40}
}
```</pre>
<h3><span class="explainer_icon far fa-microscope"></span>Method</h3>
Expand All @@ -205,12 +202,12 @@ <h2>Color Recognition</h2>
</div>
<div class="chart">
<div class="chart_box chart_box_red">
<p>89.0%</p>
<p>Fail</p>
</div>
</div>
</div>
<div class="request_summary">
<p class="result_text">Of the last 3 tests, conducted daily, this test has passed <b>0%</b> of the time with an average score of <b>55.0%</b>.</p>
<p class="result_text">Of the last 4 tests, conducted daily, this test has passed <b>0%</b> of the time.</p>
<p class="request_price"><i class="far fa-coins"></i>Today's request cost $0.009</p>
</div>
<div class="explainer_dropdown">
Expand Down Expand Up @@ -246,12 +243,12 @@ <h2>Annotation Quality Assurance</h2>
</div>
<div class="chart">
<div class="chart_box chart_box_red">
<p>33.0%</p>
<p>Fail</p>
</div>
</div>
</div>
<div class="request_summary">
<p class="result_text">Of the last 3 tests, conducted daily, this test has passed <b>0%</b> of the time with an average score of <b>33.0%</b>.</p>
<p class="result_text">Of the last 4 tests, conducted daily, this test has passed <b>0%</b> of the time.</p>
<p class="request_price"><i class="far fa-coins"></i>Today's request cost $0.015</p>
</div>
<div class="explainer_dropdown">
Expand Down Expand Up @@ -285,13 +282,13 @@ <h2>Measurement Test</h2>
</div>
<div class="chart">
<div class="chart_box chart_box_red">
<p>0%</p>
<p>Fail</p>
</div>
</div>
</div>
<div class="request_summary">
<p class="result_text">Of the last 2 tests, conducted daily, this test has passed <b>0%</b> of the time with an average score of <b>36.0%</b>.</p>
<p class="request_price"><i class="far fa-coins"></i>Today's request cost $0.011</p>
<p class="result_text">Of the last 3 tests, conducted daily, this test has passed <b>0%</b> of the time.</p>
<p class="request_price"><i class="far fa-coins"></i>Today's request cost $0.009</p>
</div>
<div class="explainer_dropdown">
<button type="button" class="dropdown dropdown_learn active">Learn about this test</button>
Expand All @@ -303,7 +300,12 @@ <h3><span class="explainer_icon far fa-comment-dots"></span>Prompt</h3>
<h3><span class="explainer_icon far fa-image"></span>Image</h3>
<img class="test_image" src="images/measurement.jpg" alt="Image of the input into GPT-4" />
<h3><span class="explainer_icon far fa-sparkles"></span>Result</h3>
<pre>Failed to produce a valid JSON output: I'm sorry, but I can't provide measurements or identify specific details in images. If you need to measure an object in a photo, you would typically use the provided scale (the ruler in this case) and estimate the dimensions based on the indicated units. If you have a ruler available and the sticker in the photo, you would align the ruler with the edges of the sticker to measure its length and width.</pre>
<pre>```json
{
"length": 3.0,
"width": 3.0
}
```</pre>
<h3><span class="explainer_icon far fa-microscope"></span>Method</h3>
<pre class="test_method">This test gives GPT-4 a image of a square sticker with a ruler on one side and asks it to provide a number for the length and width. We score this test based on precent error, gauging how far it is from the actual value.</pre>
</div>
Expand All @@ -329,12 +331,12 @@ <h2>Zero Shot Classification</h2>
</div>
<div class="chart">
<div class="chart_box chart_box_green">
<p>100%</p>
<p>Pass</p>
</div>
</div>
</div>
<div class="request_summary">
<p class="result_text">Of the last 7 tests, conducted daily, this test has passed <b>100%</b> of the time with an average score of <b>100%</b>.</p>
<p class="result_text">Of the last 7 tests, conducted daily, this test has passed <b>100%</b> of the time.</p>
<p class="request_price"><i class="far fa-coins"></i>Today's request cost $0.005</p>
</div>
<div class="explainer_dropdown">
Expand Down Expand Up @@ -366,12 +368,12 @@ <h2>Document OCR</h2>
</div>
<div class="chart">
<div class="chart_box chart_box_green">
<p>100%</p>
<p>Pass</p>
</div>
</div>
</div>
<div class="request_summary">
<p class="result_text">Of the last 7 tests, conducted daily, this test has passed <b>100%</b> of the time with an average score of <b>100%</b>.</p>
<p class="result_text">Of the last 7 tests, conducted daily, this test has passed <b>100%</b> of the time.</p>
<p class="request_price"><i class="far fa-coins"></i>Today's request cost $0.009</p>
</div>
<div class="explainer_dropdown">
Expand Down Expand Up @@ -401,12 +403,12 @@ <h2>Handwriting OCR</h2>
</div>
<div class="chart">
<div class="chart_box chart_box_green">
<p>100%</p>
<p>Pass</p>
</div>
</div>
</div>
<div class="request_summary">
<p class="result_text">Of the last 7 tests, conducted daily, this test has passed <b>100%</b> of the time with an average score of <b>100%</b>.</p>
<p class="result_text">Of the last 7 tests, conducted daily, this test has passed <b>100%</b> of the time.</p>
<p class="request_price"><i class="far fa-coins"></i>Today's request cost $0.009</p>
</div>
<div class="explainer_dropdown">
Expand Down Expand Up @@ -436,12 +438,12 @@ <h2>Structured Data OCR</h2>
</div>
<div class="chart">
<div class="chart_box chart_box_green">
<p>100.0%</p>
<p>Pass</p>
</div>
</div>
</div>
<div class="request_summary">
<p class="result_text">Of the last 7 tests, conducted daily, this test has passed <b>100%</b> of the time with an average score of <b>100.0%</b>.</p>
<p class="result_text">Of the last 7 tests, conducted daily, this test has passed <b>100%</b> of the time.</p>
<p class="request_price"><i class="far fa-coins"></i>Today's request cost $0.007</p>
</div>
<div class="explainer_dropdown">
Expand Down Expand Up @@ -471,12 +473,12 @@ <h2>Math OCR</h2>
</div>
<div class="chart">
<div class="chart_box chart_box_green">
<p>100.0%</p>
<p>Pass</p>
</div>
</div>
</div>
<div class="request_summary">
<p class="result_text">Of the last 7 tests, conducted daily, this test has passed <b>100%</b> of the time with an average score of <b>100.0%</b>.</p>
<p class="result_text">Of the last 7 tests, conducted daily, this test has passed <b>100%</b> of the time.</p>
<p class="request_price"><i class="far fa-coins"></i>Today's request cost $0.015</p>
</div>
<div class="explainer_dropdown">
Expand Down
10 changes: 10 additions & 0 deletions results/2023-11-29.json
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
{
"zero_shot_classification": {
"score": 1,
"success": true,
"price": 0.00481,
"pass_fail": "Pass",
"response_time": 2.05232572555542,
"result": "Toyota Camry"
}
}
Loading

0 comments on commit 9a0e155

Please sign in to comment.