Riallto v1.1 (#38)

This PR contains contributions for the v1.1 release of Riallto.

* Beta Linux support
    + Installation scripts for Ubuntu 24.04 using the 6.10 Linux kernel.
    + A docker based installation flow for Linux
* Upgrade flows for Ryzen AI SW to v1.1
* Hawkpoint support
* bfloat16 support
* Improvements for combining multiple kernels.
* Improvements to nputop (Windows)
* Visualisation fixes
* Fixes to sequence codegen

---------
Co-authored-by: Shane Fleming <[email protected]>
Co-authored-by: Mario Ruiz <[email protected]>
Co-authored-by: skalade <[email protected]>

.github/workflows/linux_ci_script.sh

@@ -0,0 +1,18 @@
+#!/bin/bash
+#
+# Copyright (C) 2023 Advanced Micro Devices, Inc. All rights reserved.
+# SPDX-License-Identifier: MIT
+
+docker container stop riallto_ci || true
+docker container wait riallto_ci || true
+
+docker run -dit --rm --name riallto_ci \
+        --cap-add=NET_ADMIN \
+        -v $(pwd):/workspace \
+        --device=/dev/accel/accel0:/dev/accel/accel0 \
+        -w /workspace \
+        riallto:latest \
+        /bin/bash 
+
+docker exec -i riallto_ci /bin/bash -c "source ~/.bashrc && cd /workspace/ && python3 -m pip install . && python3 -m pytest ./tests"
+

.github/workflows/nix_buildAndRun.yml

@@ -0,0 +1,44 @@
+name: Linux Riallto CI Testing 
+
+on:
+  push:
+    branches:
+      - main
+    paths-ignore:
+      - 'README.md'
+      - 'CONTRIBUTING.md'
+      - 'LICENSE**'
+      - 'notebooks/**'
+      - 'scripts/utils/**'
+      - 'scripts/wsl/**'
+      - '**/*.ipynb'
+      - '.github/ISSUE_TEMPLATE/**'
+      - '.github/*.md'
+  pull_request:
+    types: [assigned, opened, synchronize, reopened]
+    paths-ignore:
+      - 'README.md'
+      - 'CONTRIBUTING.md'
+      - 'LICENSE**'
+      - 'notebooks/**'
+      - 'scripts/utils/**'
+      - 'scripts/wsl/**'
+      - '**/*.ipynb'
+      - '.github/ISSUE_TEMPLATE/**'
+      - '.github/*.md'
+  workflow_dispatch:
+
+jobs:
+  birman-tests:
+    name: not-birman-15-linux-tests
+    runs-on: self-hosted-linux
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v2
+        with:
+          fetch-depth: 1
+
+      - name: run pytests 
+        run: |
+          ls ./
+          ./.github/workflows/linux_ci_script.sh 

.github/workflows/win_buildAndRun.yml

@@ -31,7 +31,7 @@ on:
 jobs:
   birman-tests:
     name: not-birman-15-tests
-    runs-on: self-hosted
+    runs-on: self-hosted-windows
     steps:
       - name: Checkout
         uses: actions/checkout@v2

.gitignore

@@ -11,3 +11,10 @@ install/installed
 install/tarballs/*.tar.gz
 
 *-checkpoint.ipynb
+
+tests/images/*.svg
+
+/*.json
+/*.xclbin
+/*.mlir
+/*.seq

CITATION.cff

@@ -30,6 +30,6 @@ authors:
 - family-names: "Joshua"
   given-names: "Lu"
 title: "Riallto: an open-source exploration framework for first time users of the AMD Ryzen AI NPU"
-version: 1.0
-date-released: 2021-12-12
-url: "https://github.com/AMDResearch/Riallto"
+version: 1.1
+date-released: 2023-12-12
+url: "https://github.com/AMDResearch/Riallto"

README.md

@@ -4,15 +4,15 @@ Riallto is an open source project that aims to bridge the gap between newcomers
 
 The educational examples and software in this repository will enable you to interact with the NPU and learn how to write custom applications with this novel accelerator.
 
-Note that NPUs can also be referred to as Inference Processing Units, or IPUs. You may see the NPU referred to as an "IPU" in some Ryzen AI documentation. Once the Ryzen AI driver is installed on your computer, you will see a device named "IPU" in the Windows Device Manager."
+Note that NPUs can also be referred to as Inference Processing Units, or IPUs. You may see the NPU referred to as an "IPU" in some Ryzen AI documentation. On Windows once the Ryzen AI driver is installed on your computer, you will see a device named "IPU" in the Windows Device Manager."
 
-## Quick Start
+## Quick Start (Windows)
 
 The Riallto installer is separated into 'Lite' and 'Full' installation versions. The 'Lite' version has all the essential packages required for the Riallto and ONNX runtimes on the NPU to get started with prebuilt vision applications. The 'Full' version enables developers to write their own custom applications, which requires additional tools.
 
 The steps below will allow you to get quickly started using the 'Lite' version of Riallto. For details on requirements and installation for the 'Full' version, follow the instructions in the [Install Riallto](https://riallto.ai/install-riallto.html) guide.
 
-* [Download and unzip the the v10.1109.8.100 IPU driver](https://account.amd.com/en/forms/downloads/ryzen-ai-software-platform-xef.html?filename=ipu_stack_rel_silicon_1.0.zip). You will need to create an AMD account if you do not have one.
+* [Download and unzip the the v10.1109.8.128 IPU driver](https://account.amd.com/en/forms/downloads/ryzen-ai-software-platform-xef.html?filename=ipu_stack_rel_silicon_prod_1.1.zip). You will need to create an AMD account if you do not have one.
 * Open a Windows Command Prompt, or Powershell, as administrator.
 * Change directory to the unzipped IPU driver directory, then run the `amd_install_kipudrv.bat` file. Press Enter if prompted.
 * Confirm the IPU driver has installed correctly by running the following command in a Powershell terminal and checking the output matches that of the figure below:
@@ -23,7 +23,7 @@ The steps below will allow you to get quickly started using the 'Lite' version o
 
 ![IPU driver dialog](docs/images/ipu_driver.png)
 
-* [Download and unzip the Riallto installer](https://www.xilinx.com/bin/public/openDownload?filename=Riallto-v1.0.zip). 
+* [Download and unzip the Riallto installer](https://www.amd.com/bin/public/amdOpenDownload?filename=Riallto-v1.1.zip). 
 * Navigate to the `installer.exe` using the File Explorer and run as administrator, making sure the 'Lite' install option is selected when prompted, as shown in the figure below. 
 
 ![Riallto installer options](docs/images/installer.png)
@@ -34,6 +34,10 @@ The installer places a shortcut on the desktop, which allows you to launch Riall
 
 Note that the notebooks from sections 1, 2, 3, and 5 can be run using the 'Lite' install option. A 'Full' install is required to run the notebooks from section 4. See instructions in the [Install Riallto](https://riallto.ai/install-riallto.html) guide for more details.
 
+## Quick Start (Linux)
+
+Along with Windows support there is also support to install Riallto in Ubuntu 24.04 provided you upgrade to the 6.10 Linux Kernel. Steps for the installation process for this can be found [here](./scripts/linux).
+
 ## Contribute
 
 Contributions to this repository are welcome. Please refer to [CONTRIBUTING.md](CONTRIBUTING.md) for how to improve Riallto.

notebooks/1_0_Introduction.ipynb

@@ -74,6 +74,8 @@
     "\n",
     "Demonstrates the Ryzen AI NPU in action, running real-time AI workloads in Microsoft's Windows Studio Effects (WSE).\n",
     "\n",
+    "**This is only supported on Windows.**\n",
+    "\n",
     "**[1. Windows Studio Effects demo](2_1_MS_Windows_Studio_Effects.ipynb)**\n"
    ]
   },
@@ -135,6 +137,8 @@
     "\n",
     "Introduces the PyTorch-ONNX flow for running AI applications on the Ryzen AI NPU.\n",
     "\n",
+    "**This is currently not supported on Linux.** \n",
+    "\n",
     "**[1. Machine Learning Inference with PyTorch and ONNX](5_1_pytorch_onnx_inference.ipynb)**\n",
     "\n",
     "**[2. Machine Learning re-training with PyTorch and ONNX](5_2_pytorch_onnx_re-train.ipynb)**\n",
@@ -184,7 +188,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.2"
+   "version": "3.10.6"
   },
   "widgets": {
    "application/vnd.jupyter.widget-state+json": {

notebooks/2_1_MS_Windows_Studio_Effects.ipynb

@@ -13,11 +13,23 @@
     "\n",
     "* Demonstrate the NPU in action using the Windows Studio Effects\n",
     "\n",
+    "\n",
     "## References\n",
     "\n",
     "**[Windows Studio Effects](https://support.microsoft.com/en-us/windows/manage-cameras-with-camera-settings-in-windows-11-97997ed5-bb98-47b6-a13d-964106997757#ID0EDBBF)**\n",
     "    \n",
-    "---\n"
+    "---"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "<div class=\"alert alert-box alert-warning\">\n",
+    "\n",
+    "This is only supported on Windows and is not supported on the Linux release of Riallto\n",
+    "\n",
+    "</div>"
    ]
   },
   {
@@ -219,7 +231,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.2"
+   "version": "3.10.6"
   }
  },
  "nbformat": 4,

notebooks/4_1_software_framework.ipynb

@@ -52,7 +52,7 @@
     "* **AIEtools**\n",
     "    * These are the compilation tools used to build the Ryzen AI NPU application. \n",
     "\n",
-    "The AIETools tools are Linux based. For Windows laptops, they run in [Windows Subsystem for Linux 2 (WSL 2)](https://learn.microsoft.com/en-us/windows/wsl/about). If you have installed Riallto and are reading this material on your laptop as a Jupyter notebook, WSL 2 should have been already installed and enabled on your system. "
+    "The AIETools tools are Linux based. For Windows laptops, they run in [Windows Subsystem for Linux 2 (WSL 2)](https://learn.microsoft.com/en-us/windows/wsl/about). If you have installed Riallto and are reading this material on your Windows laptop as a Jupyter notebook, WSL 2 should have been already installed and enabled on your system. WSL 2 is not required for the Linux installation. On Linux the AIETools tools are contained within a docker container."
    ]
   },
   {
@@ -62,7 +62,7 @@
    },
    "source": [
     "<div class=\"alert alert-box alert-info\">\n",
-    "To check that the WSL 2 instance, where the <em>Riallto</em> tools are installed, is enabled on your system, run the following cell:\n",
+    "On Windows to check that the WSL 2 instance, where the <em>Riallto</em> tools are installed, is enabled on your system, run the following cell (Note: this will not work on a Linux installation of Riallto):\n",
     "</div>"
    ]
   },
@@ -103,6 +103,32 @@
     "The output of this cell should report the OS version where the 'Riallto' tools are installed. If you do not see this, or if you installed the **Lite** version of Riallto, please refer to the [Riallto installation instructions](https://www.riallto.ai/install-riallto.html) to install the **Full** version of Riallto."
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "<div class=\"alert alert-box alert-info\">\n",
+    "On Ubuntu to check that your Linux Kernel version is >6.10 which <em>Riallto</em> requires, run the following cell:\n",
+    "</div>"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "6.10.0-061000rc2-generic\n"
+     ]
+    }
+   ],
+   "source": [
+    "! uname -r"
+   ]
+  },
   {
    "cell_type": "markdown",
    "metadata": {},
@@ -280,7 +306,7 @@
        "\n",
        "This will construct a passthrough npu.build.Kernel object that can be used within\n",
        "a callgraph to construct a complete application. \n",
-       "\u001b[1;31mFile:\u001b[0m      c:\\users\\riallto\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\npu\\magic.py"
+       "\u001b[1;31mFile:\u001b[0m      c:\\users\\shane\\appdata\\local\\riallto\\riallto_venv\\lib\\site-packages\\npu\\magic.py"
       ]
      },
      "metadata": {},
@@ -476,7 +502,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.2"
+   "version": "3.12.3"
   },
   "widgets": {
    "application/vnd.jupyter.widget-state+json": {

notebooks/5_1_pytorch_onnx_inference.ipynb

@@ -14,6 +14,7 @@
     "* Show the ONNX model generation and inference flow on the NPU\n",
     "    \n",
     "* Deploy a quantized ResNet-50 model onto Ryzen AI NPU for inference\n",
+    "\n",
     "      \n",
     "## References\n",
     "\n",
@@ -30,6 +31,17 @@
     "---\n"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "<div class=\"alert alert-box alert-warning\">\n",
+    "\n",
+    "This is not currently supported on the Linux release of Riallto.\n",
+    "\n",
+    "</div>"
+   ]
+  },
   {
    "cell_type": "markdown",
    "metadata": {},
@@ -850,7 +862,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.2"
+   "version": "3.10.6"
   },
   "widgets": {
    "application/vnd.jupyter.widget-state+json": {

notebooks/5_2_pytorch_onnx_re-train.ipynb

@@ -15,6 +15,7 @@
     "\n",
     "* Learn how to quantize an ONNX model to run inference on the NPU\n",
     "\n",
+    "\n",
     "## References\n",
     "\n",
     "**[Ryzen AI Software Platform](https://ryzenai.docs.amd.com/en/latest/getstartex.html)**\n",
@@ -26,6 +27,17 @@
     "---\n"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "<div class=\"alert alert-box alert-warning\">\n",
+    "\n",
+    "This is not currently supported on the Linux release of Riallto.\n",
+    "\n",
+    "</div>"
+   ]
+  },
   {
    "cell_type": "markdown",
    "metadata": {},
@@ -698,7 +710,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.2"
+   "version": "3.10.6"
   },
   "widgets": {
    "application/vnd.jupyter.widget-state+json": {

npu/__init__.py

@@ -26,13 +26,16 @@
 """
 
 from .utils.test_device import get_driver_version, version_to_tuple
+import platform
 
 __supported_driver__ = "10.1109.8.100"
-__installed_driver__ = get_driver_version()
 
-if version_to_tuple(__installed_driver__) < version_to_tuple(__supported_driver__):
-    raise ValueError(f"""Detected driver: {__installed_driver__}, supported driver version is >={__supported_driver__}, 
-              go to https://riallto.ai/prerequisites-driver.html for driver setup instructions.""")
+if platform.system() == 'Windows':
+    __installed_driver__ = get_driver_version()
+
+    if version_to_tuple(__installed_driver__) < version_to_tuple(__supported_driver__):
+        raise ValueError(f"""Detected driver: {__installed_driver__}, supported driver version is >={__supported_driver__},
+                  go to https://riallto.ai/prerequisites-driver.html for driver setup instructions.""")
 
 from .repr_dict import ReprDict
 from .magic import kernel_magic