Skip to content

Commit

Permalink
Push dev branch build
Browse files Browse the repository at this point in the history
  • Loading branch information
Naeemkh committed Feb 2, 2025
1 parent e69a253 commit 0e65737
Show file tree
Hide file tree
Showing 4 changed files with 15 additions and 15 deletions.
2 changes: 1 addition & 1 deletion docs/contents/core/benchmarking/benchmarking.html
Original file line number Diff line number Diff line change
Expand Up @@ -812,7 +812,7 @@ <h3 data-number="12.2.3" class="anchored" data-anchor-id="domain-specific-benchm
</section>
<section id="ai-benchmarks-system-model-and-data" class="level2 page-columns page-full" data-number="12.3">
<h2 data-number="12.3" class="anchored" data-anchor-id="ai-benchmarks-system-model-and-data"><span class="header-section-number">12.3</span> AI Benchmarks: System, Model, and Data</h2>
<p>The evolution of benchmarks reaches its apex in machine learning, reflecting a journey that parallels the field’s development towards domain-specific applications. Early machine learning benchmarks focused primarily on algorithmic performance, measuring how well models could perform specific tasks <span class="citation" data-cites="lecun1998gradient">(<a href="../references.html#ref-lecun1998gradient" role="doc-biblioref">Lecun et al. 1998</a>)</span>. As machine learning applications scaled and computational demands grew, the focus expanded to include system performance and hardware efficiency <span class="citation" data-cites="jouppi2017datacenter">(<a href="../references.html#ref-jouppi2017datacenter" role="doc-biblioref">Jouppi et al. 2017</a>)</span>. Most recently, the critical role of data quality has emerged as the third essential dimension of evaluation <span class="citation" data-cites="gebru2018datasheets">(<a href="../references.html#ref-gebru2018datasheets" role="doc-biblioref"><strong>gebru2018datasheets?</strong></a>)</span>.</p>
<p>The evolution of benchmarks reaches its apex in machine learning, reflecting a journey that parallels the field’s development towards domain-specific applications. Early machine learning benchmarks focused primarily on algorithmic performance, measuring how well models could perform specific tasks <span class="citation" data-cites="lecun1998gradient">(<a href="../references.html#ref-lecun1998gradient" role="doc-biblioref">Lecun et al. 1998</a>)</span>. As machine learning applications scaled and computational demands grew, the focus expanded to include system performance and hardware efficiency <span class="citation" data-cites="jouppi2017datacenter">(<a href="../references.html#ref-jouppi2017datacenter" role="doc-biblioref">Jouppi et al. 2017</a>)</span>. Most recently, the critical role of data quality has emerged as the third essential dimension of evaluation <span class="citation" data-cites="gebru2021datasheets">(<a href="../references.html#ref-gebru2021datasheets" role="doc-biblioref">Gebru et al. 2021</a>)</span>.</p>
<div class="no-row-height column-margin column-container"><div id="ref-jouppi2017datacenter" class="csl-entry" role="listitem">
Jouppi, Norman P., Cliff Young, Nishant Patil, David Patterson, Gaurav Agrawal, Raminder Bajwa, Sarah Bates, et al. 2017. <span>“In-Datacenter Performance Analysis of a Tensor Processing Unit.”</span> <em>ACM SIGARCH Computer Architecture News</em> 45 (2): 1–12. <a href="https://doi.org/10.1145/3140659.3080246">https://doi.org/10.1145/3140659.3080246</a>.
</div></div><p>What sets AI benchmarks apart from traditional performance metrics is their inherent variability—introducing accuracy as a fundamental dimension of evaluation. Unlike conventional benchmarks, which measure fixed, deterministic characteristics like computational speed or energy consumption, AI benchmarks must account for the probabilistic nature of machine learning models. The same system can produce different results depending on the data it encounters, making accuracy a defining factor in performance assessment. This distinction adds complexity, as benchmarking AI systems requires not only measuring raw computational efficiency but also understanding trade-offs between accuracy, generalization, and resource constraints.</p>
Expand Down
12 changes: 6 additions & 6 deletions docs/contents/core/efficient_ai/efficient_ai.html
Original file line number Diff line number Diff line change
Expand Up @@ -801,10 +801,10 @@ <h2 data-number="9.2" class="anchored" data-anchor-id="ai-efficiency-evolution">
<li><p><strong>Compute Efficiency:</strong> Compute efficiency addresses the effective utilization of computational resources, including energy and hardware infrastructure.</p></li>
<li><p><strong>Data Efficiency:</strong> Data efficiency emphasizes optimizing the amount and quality of data required to achieve desired performance.</p></li>
</ul>
<div id="fig-evolution-efficiency" class="quarto-float quarto-figure quarto-figure-center anchored" data-fig-env="figure" data-fig-pos="htb">
<div id="fig-evolution-efficiency" class="quarto-float quarto-figure quarto-figure-center anchored" data-fig-pos="htb" data-fig-env="figure">
<figure class="quarto-float quarto-float-fig figure">
<div aria-describedby="fig-evolution-efficiency-caption-0ceaefa1-69ba-4598-a22c-09a6ac19f8ca">
<div class="sourceCode" id="fig-evolution-efficiency" data-fig-env="figure" data-fig-pos="htb"><pre class="sourceCode tikz code-with-copy"><code class="sourceCode latex"><span id="fig-evolution-efficiency-1"><a href="#fig-evolution-efficiency-1" aria-hidden="true" tabindex="-1"></a><span class="kw">\begin</span>{<span class="ex">tikzpicture</span>}[font=<span class="fu">\small\sf</span>,node distance=2mm]</span>
<div class="sourceCode" id="fig-evolution-efficiency" data-fig-pos="htb" data-fig-env="figure"><pre class="sourceCode tikz code-with-copy"><code class="sourceCode latex"><span id="fig-evolution-efficiency-1"><a href="#fig-evolution-efficiency-1" aria-hidden="true" tabindex="-1"></a><span class="kw">\begin</span>{<span class="ex">tikzpicture</span>}[font=<span class="fu">\small\sf</span>,node distance=2mm]</span>
<span id="fig-evolution-efficiency-2"><a href="#fig-evolution-efficiency-2" aria-hidden="true" tabindex="-1"></a><span class="fu">\usetikzlibrary</span>{positioning,arrows.meta,arrows,calc}</span>
<span id="fig-evolution-efficiency-3"><a href="#fig-evolution-efficiency-3" aria-hidden="true" tabindex="-1"></a><span class="fu">\tikzset</span>{</span>
<span id="fig-evolution-efficiency-4"><a href="#fig-evolution-efficiency-4" aria-hidden="true" tabindex="-1"></a> Box/.style={inner xsep=1pt,</span>
Expand Down Expand Up @@ -1033,10 +1033,10 @@ <h3 data-number="9.3.1" class="anchored" data-anchor-id="defining-ml-system-effi
<h3 data-number="9.3.2" class="anchored" data-anchor-id="interdependencies-between-efficiency-dimensions"><span class="header-section-number">9.3.2</span> Interdependencies Between Efficiency Dimensions</h3>
<p>The efficiency of machine learning systems is inherently a multifaceted challenge that encompasses model design, computational resources, and data utilization. These dimensions—algorithmic efficiency, compute efficiency, and data efficiency—are deeply interdependent, forming a dynamic ecosystem where improvements in one area often ripple across the others. Understanding these interdependencies is crucial for building scalable, cost-effective, and high-performing systems that can adapt to diverse application demands.</p>
<p>This interplay is best captured through a conceptual visualization. <a href="#fig-interdependece" class="quarto-xref">Figure&nbsp;<span>9.4</span></a> illustrates how these efficiency dimensions overlap and interact with each other in a simple Venn diagram. Each circle represents one of the efficiency dimensions, and their intersections highlight the areas where they influence one another, which we will explore next.</p>
<div id="fig-interdependece" class="quarto-float quarto-figure quarto-figure-center anchored" data-fig-env="figure" data-fig-pos="htb">
<div id="fig-interdependece" class="quarto-float quarto-figure quarto-figure-center anchored" data-fig-pos="htb" data-fig-env="figure">
<figure class="quarto-float quarto-float-fig figure">
<div aria-describedby="fig-interdependece-caption-0ceaefa1-69ba-4598-a22c-09a6ac19f8ca">
<div class="sourceCode" id="fig-interdependece" data-fig-env="figure" data-fig-pos="htb"><pre class="sourceCode tikz code-with-copy"><code class="sourceCode latex"><span id="fig-interdependece-1"><a href="#fig-interdependece-1" aria-hidden="true" tabindex="-1"></a><span class="fu">\scalebox</span>{0.8}{<span class="co">%</span></span>
<div class="sourceCode" id="fig-interdependece" data-fig-pos="htb" data-fig-env="figure"><pre class="sourceCode tikz code-with-copy"><code class="sourceCode latex"><span id="fig-interdependece-1"><a href="#fig-interdependece-1" aria-hidden="true" tabindex="-1"></a><span class="fu">\scalebox</span>{0.8}{<span class="co">%</span></span>
<span id="fig-interdependece-2"><a href="#fig-interdependece-2" aria-hidden="true" tabindex="-1"></a><span class="kw">\begin</span>{<span class="ex">tikzpicture</span>}[font=<span class="fu">\small\sf</span>,scale=1.25,line width=0.75pt]</span>
<span id="fig-interdependece-3"><a href="#fig-interdependece-3" aria-hidden="true" tabindex="-1"></a><span class="fu">\usetikzlibrary</span>{arrows.meta,calc,positioning,angles}</span>
<span id="fig-interdependece-4"><a href="#fig-interdependece-4" aria-hidden="true" tabindex="-1"></a><span class="fu">\def\firstcircle</span>{(0,0) circle (1.5cm)}</span>
Expand Down Expand Up @@ -1136,10 +1136,10 @@ <h4 class="anchored" data-anchor-id="sustainability-reinforces-efficiency">Susta
<h4 class="anchored" data-anchor-id="the-virtuous-cycle-of-machine-learning-systems">The Virtuous Cycle of Machine Learning Systems</h4>
<p>Efficiency, scalability, and sustainability are deeply interconnected, forming a virtuous cycle that propels machine learning systems toward broader impact. Efficient systems enable scalable deployments, which amplify their sustainability benefits. In turn, sustainable practices drive the need for more efficient designs, ensuring the cycle continues. This interplay creates systems that are not only technically impressive but also socially and environmentally responsible, aligning AI innovation with the needs of a global community.</p>
<p><a href="#fig-virtuous-cycle" class="quarto-xref">Figure&nbsp;<span>9.5</span></a> below illustrates the virtuous cycle of machine learning systems. It highlights how efficiency drives scalability, scalability fosters sustainability, and sustainability reinforces efficiency.</p>
<div id="fig-virtuous-cycle" class="quarto-float quarto-figure quarto-figure-center anchored" data-fig-env="figure" data-fig-pos="htb">
<div id="fig-virtuous-cycle" class="quarto-float quarto-figure quarto-figure-center anchored" data-fig-pos="htb" data-fig-env="figure">
<figure class="quarto-float quarto-float-fig figure">
<div aria-describedby="fig-virtuous-cycle-caption-0ceaefa1-69ba-4598-a22c-09a6ac19f8ca">
<div class="sourceCode" id="fig-virtuous-cycle" data-fig-env="figure" data-fig-pos="htb"><pre class="sourceCode tikz code-with-copy"><code class="sourceCode latex"><span id="fig-virtuous-cycle-1"><a href="#fig-virtuous-cycle-1" aria-hidden="true" tabindex="-1"></a><span class="kw">\begin</span>{<span class="ex">tikzpicture</span>}[font=<span class="fu">\small\sf</span>,node distance=1pt,line width=0.75pt]</span>
<div class="sourceCode" id="fig-virtuous-cycle" data-fig-pos="htb" data-fig-env="figure"><pre class="sourceCode tikz code-with-copy"><code class="sourceCode latex"><span id="fig-virtuous-cycle-1"><a href="#fig-virtuous-cycle-1" aria-hidden="true" tabindex="-1"></a><span class="kw">\begin</span>{<span class="ex">tikzpicture</span>}[font=<span class="fu">\small\sf</span>,node distance=1pt,line width=0.75pt]</span>
<span id="fig-virtuous-cycle-2"><a href="#fig-virtuous-cycle-2" aria-hidden="true" tabindex="-1"></a><span class="fu">\usetikzlibrary</span>{arrows.meta,calc,positioning,angles}</span>
<span id="fig-virtuous-cycle-3"><a href="#fig-virtuous-cycle-3" aria-hidden="true" tabindex="-1"></a><span class="fu">\def\ra</span>{40mm}</span>
<span id="fig-virtuous-cycle-4"><a href="#fig-virtuous-cycle-4" aria-hidden="true" tabindex="-1"></a></span>
Expand Down
Loading

0 comments on commit 0e65737

Please sign in to comment.