Adding docs

intel · Feb 19, 2024 · ede8e16 · ede8e16
1 parent c60a95a
commit ede8e16
Show file tree

Hide file tree

Showing 43 changed files with 3,224 additions and 0 deletions.
diff --git a/docs/.gitignore b/docs/.gitignore
@@ -0,0 +1 @@
+book
diff --git a/docs/book.toml b/docs/book.toml
@@ -0,0 +1,6 @@
+[book]
+authors = ["el-tipton"]
+language = "en"
+multilingual = False
+src = "src"
+title = "HBFA-FL: Host-Based Firmware Analyzer - Fuzzing Lite"
diff --git a/docs/images/Step3_Select_Boot_Manager.png b/docs/images/Step3_Select_Boot_Manager.png
diff --git a/docs/images/Step4_Select_EFI_Shell.png b/docs/images/Step4_Select_EFI_Shell.png
diff --git a/docs/images/Step5_EFI_Shell_Press_Escape.png b/docs/images/Step5_EFI_Shell_Press_Escape.png
diff --git a/docs/images/Ubuntu_boot_select_firmware_settings.png b/docs/images/Ubuntu_boot_select_firmware_settings.png
diff --git a/docs/images/hbfa_covreport.png b/docs/images/hbfa_covreport.png
diff --git a/docs/images/hbfa_covreport2.png b/docs/images/hbfa_covreport2.png
diff --git a/docs/images/hbfa_crash_AAA.png b/docs/images/hbfa_crash_AAA.png
diff --git a/docs/images/hbfa_crashes.png b/docs/images/hbfa_crashes.png
diff --git a/docs/images/hbfa_hangs.png b/docs/images/hbfa_hangs.png
diff --git a/docs/images/hbfa_helloworld_AAAAs.png b/docs/images/hbfa_helloworld_AAAAs.png
diff --git a/docs/images/hbfa_helloworld_debug_Acrash1.png b/docs/images/hbfa_helloworld_debug_Acrash1.png
diff --git a/docs/images/hbfa_helloworld_debug_Acrash2.png b/docs/images/hbfa_helloworld_debug_Acrash2.png
diff --git a/docs/images/hbfa_helloworld_debug_Acrash3.png b/docs/images/hbfa_helloworld_debug_Acrash3.png
diff --git a/docs/images/hbfa_helloworld_extended.png b/docs/images/hbfa_helloworld_extended.png
diff --git a/docs/images/hbfa_helloworld_list.png b/docs/images/hbfa_helloworld_list.png
diff --git a/docs/images/hbfa_helloworld_list2.png b/docs/images/hbfa_helloworld_list2.png
diff --git a/docs/images/hbfa_helloworld_reprocrash.png b/docs/images/hbfa_helloworld_reprocrash.png
diff --git a/docs/images/hbfa_shell_helloworld.png b/docs/images/hbfa_shell_helloworld.png
diff --git a/docs/images/hbfa_shell_ls.png b/docs/images/hbfa_shell_ls.png
diff --git a/docs/images/hbfa_shell_map.png b/docs/images/hbfa_shell_map.png
diff --git a/docs/images/hbfa_source_UefiHostTestPkg.dsc.png b/docs/images/hbfa_source_UefiHostTestPkg.dsc.png
diff --git a/docs/images/hbfa_source_helloworld.c.png b/docs/images/hbfa_source_helloworld.c.png
diff --git a/docs/images/hbfa_source_testhelloworld.c.png b/docs/images/hbfa_source_testhelloworld.c.png
diff --git a/docs/images/hbfa_summary.png b/docs/images/hbfa_summary.png
diff --git a/docs/src/README.md b/docs/src/README.md
@@ -0,0 +1,31 @@
+# Introduction
+
+The Host-based Firmware Analyzer - Fuzzing Lite (HBFA-FL) is based off of forking the original HBFA from the [edk2-staging repository](https://github.com/tianocore/edk2-staging/tree/HBFA). The original release of HBFA is described in the white-paper: ["Using Host-based Firmware Analysis to Improve Platform Resiliency"](https://www.intel.com/content/dam/develop/external/us/en/documents/intel-usinghbfatoimproveplatformresiliency-820238.pdf). The original release authors were Brian Richardson, Chris Wu, Jiewen Yao, and Vincent J. Zimmer.
+
+The goal for HBFA-FL is to update the original codebase, enhancing some features and removing/streamlining some others, with a focus on functionality for Linux environments. To that end, HBFA-FL removes much of the original HBFA code base, including support for Windows and unit-testing; we retain support for fuzzing with AFL and LibFuzzer. This repository is effectively a fork from the original version of HBFA at this commit : [ead8f4f8eefdb2eb762184e1a5809c43d6908952](https://github.com/tianocore/edk2-staging/commit/ead8f4f8eefdb2eb762184e1a5809c43d6908952)
+
+## Version
+
+The most recent release for HBFA-FL is version 0.11. The release is an initial release with a focus on functionality modern Linux, for AFL and LibFuzzer fuzzing support, and enabling additional sanitizers and coverage options for HBFA.
+
+### Supported Features
+
+- Command-line interfaces to support building and running of fuzzing
+- Execute fuzzing harnesses from common fuzzing frameworks (e.g. AFL and LibFuzzer)
+- Incorporation of Address Sanitizer (ASAN), Memory Sanitizer (MSAN), and the Undefined Behavior Sanitizer (UBSAN)
+- Generating code coverage reports (GCOV/LCOV and Profraw)
+
+### Features Not Fully Implemented
+
+- Support for symbolic execution with KLEE/STP with HBFA-FL has not yet been fully updated.
+
+### Features Not Supported from the Original HBFA
+
+- Windows support has been removed
+- GUI support has been removed
+
+## Getting Started
+
+To get started using HBFA-FL, we recommend taking a look at the documentation, starting [here](./SUMMARY.md).
+
+Return to [Summary](./SUMMARY.md) | Next [&gt;&gt;](./setup/README.md)
diff --git a/docs/src/SUMMARY.md b/docs/src/SUMMARY.md
@@ -0,0 +1,21 @@
+# Summary
+
+[Introduction](README.md)
+
+## User Guide
+
+- [Setting Up](setup/README.md)
+  - [For Linux](setup/linux.md)
+- [Getting started with fuzzing in HBFA-FL](fuzzing/README.md)
+  - [Where to create and save a fuzzing test case harness](harness/wheretoharness.md)
+  - [Creating and Compiling New Test Cases](fuzzing/building.md)
+  - [Fuzzing test harnesses included with HBFA-FL](harness/includedfuzzharnesses.md)
+  - [Fuzzing with AFL: RunAFL.py](fuzzingwithAFL.md)
+  - [Fuzzing with LibFuzzer: RunLibFuzzer.py](./fuzzingwithLibFuzzer.md)
+  - [Generating fuzzing summary and coverage data reports](./generatingCoverageReports.md)
+- [Tutorials](tutorials/README.md)
+  - [HBFA-FL: Writing a fuzzing harness](tutorials/writingafuzzingharness.md)
+
+## Archived Documentation
+
+- [Original HBFA Documentation](archive/originalHBFA/README.md)
diff --git a/docs/src/archive/originalHBFA/Doc/User Guide - How-to-Add-New-Case.pdf b/docs/src/archive/originalHBFA/Doc/User Guide - How-to-Add-New-Case.pdf
diff --git a/docs/src/archive/originalHBFA/Doc/User Guide - How-to-Run-Test-Case.pdf b/docs/src/archive/originalHBFA/Doc/User Guide - How-to-Run-Test-Case.pdf
diff --git a/docs/src/archive/originalHBFA/README.md b/docs/src/archive/originalHBFA/README.md
@@ -0,0 +1,11 @@
+# Archive for documentation
+
+Several documents related to HBFA are referenced externally or archived locally in this repository in this folder. A Table of these documents is included in the following.
+
+| Document | Reference link |
+| -------- | -------------- |
+| The original HBFA release in the whitepaper | [Link](https://www.intel.com/content/dam/develop/external/us/en/documents/intel-usinghbfatoimproveplatformresiliency-820238.pdf) |
+| The original HBFA 'User Guide - How-to-Add-New-Case.pdf' | [Link](./Doc/User%20Guide%20-%20How-to-Add-New-Case.pdf) |
+
+
+Return to [Summary](../../SUMMARY.md)
diff --git a/docs/src/fuzzing/README.md b/docs/src/fuzzing/README.md
@@ -0,0 +1,12 @@
+# Getting started with fuzzing in HBFA-FL
+
+Fuzzing with AFL and LibFuzzer are supported in HBFA-FL. For a given test-harness, the source can be compiled and fuzzed with either AFL or LibFuzzer. To do this, you may directly invoke the build process from EDK-II or you may leverage the included 'RunAFL.py' and 'RunLibFuzzer.py' scripts (recommended). Additionally, several fuzzing test-harnesses are included with HBFA-FL and can be helpful for fuzzing, or as a good reference for developing other fuzzing harnesses.
+
+- For information on creating and building test cases, see these sections: [Where to create and save a fuzzing test case harness](../harness/wheretoharness.md) and [Creating and Compiling New Test Cases](./building.md)
+- For a full-tutorial on creating fuzzing harnesses for HBFA-FL, see the tutorial [HBFA-FL: Writing a fuzzing harness](../tutorials/writingafuzzingharness.md)
+- For more information on the fuzzing test-harnesses included with HBFA-FL, see this [section](../harness/includedfuzzharnesses.md)
+- To get started fuzzing with AFL, see [Fuzzing with AFL](fuzzingwithAFL.md)
+- To get started fuzzing with LibFuzzer, see [Fuzzing with LibFuzzer](./fuzzingwithLibFuzzer.md)
+- To learn more on how to generate Fuzzing Report and Coverage data for your fuzzing, see [Generating fuzzing summary and coverage data reports](./generatingCoverageReports.md)
+
+[&lt;&lt;](../setup/linux.md) Back | Return to [Summary](../SUMMARY.md) | Next [&gt;&gt;](./building.md)
diff --git a/docs/src/fuzzing/building.md b/docs/src/fuzzing/building.md
@@ -0,0 +1,32 @@
+# Creating and Compiling New Test Cases
+
+The original HBFA documentation provides some helpful information on how to to about adding new test cases/harnesses. For more information, see [Host-based Firmware Analyzer User Guide: How to Add New Case](https://github.com/tianocore/edk2-staging/blob/HBFA/HBFA/Doc/User%20Guide%20-%20How-to-Add-New-Case.pdf). Further, a [tutorial](../tutorials/writingafuzzingharness.md) is included in HBFA-FL, which gives details on creating a fuzzing test harness for a vulnerable (fictitious) testHelloWorld UEFI shell program.
+
+## Compiling Test Cases
+
+The recommended approach with HBFA is to use the included RunAFL.py and RunLibFuzzer.py scripts. These scripts will invoke the build commands needed (see the sections: [Using RunAFL.py to build and fuzzing a module](#b-using-runaflpy-to-build-and-fuzzing-a-module) and [Using LibFuzzer.py to build and fuzzing a module](#b-using-libfuzzerpy-to-build-and-fuzzing-a-module). However, one can directly invoke the build system in EDK-II.
+
+For building a test module in EDK-II, some of the important files used in the HBFA environment are described in the following. (Noting, the file paths are releative to the base of the edk2 repository.)
+
+| Filename | Description |
+| -------- | ----------- |
+| edk2/BaseTools/BinWrappers/PosixLike/build | When invoking 'build' from the CLI in HBFA, this Bash script is ran and acts as a wrapper to invoke a Python-based build script 'build.py' for the HBFA environment in this Docker image.|
+| edk2/BaseTools/Source/Python/build/build.py | Primary script used to orchestrate  building a platform or a module for EDK-II |
+
+When building a test module in HBFA, an invocation of the 'build' command may be done similar to that shown in the following.
+
+```console
+build -p UefiHostFuzzTestCasePkg/UefiHostFuzzTestCasePkg.dsc -m UefiHostFuzzTestCasePkg/TestCase/MdeModulePkg/Library/BaseBmpSupportLib/TestBmpSupportLib.inf -a X64 -b DEBUG -t AFL --conf /root/hbfa_workspace/hbfa-fl/HBFA/UefiHostFuzzTestPkg/Conf -t GCC5
+```
+
+The 'build' script has many options and features (e.g. see the output from ```build -h```). Some useful flags used (and available in the HBFA environment are)
+
+| Build CLI option | Purpose | Notes/Available options |
+| ---------------- | ------- | ----------------- |
+| -p | To specify the platform (.dsc) file name | In this case, the platform should be the HBFA, UefiHostFuzzTestCasePkg.dsc file. |
+| -m | To specify the test case the module specified by the INF file name argument | This should point to the test module file you have created and with to build/fuzz (or a pre-built test-case) |
+| -a | To specify the target architecture | Per HBFA documentation, only 'X64' is supported for LibFuzzer. Use of 'IA32' is ok for AFL in HBFA. |
+| -t | This is used to specify the toolchain | Note multiple targets can be specified (e.g. ```-t AFL -t GCC5```) |
+| --conf | the customized Conf directory | For HBFA, this should be set as 'hbfa-fl/HBFA/UefiHostFuzzTestPkg/Conf' |
+
+[&lt;&lt;](./README.md) Back | Return to [Summary](../SUMMARY.md) | Next [&gt;&gt;](../harness/includedfuzzharnesses.md)
diff --git a/docs/src/fuzzing/fuzzingwithAFL.md b/docs/src/fuzzing/fuzzingwithAFL.md
@@ -0,0 +1,107 @@
+# Fuzzing with AFL: RunAFL.py
+
+To demonstrate using the RunAFL.py script for fuzzing, we'll build and fuzz the TestBmpSupportLib test-case included with HBFA-FL. First, it is important to understand some of the required arguments for the RunAFL.py script. To that end, the help contents for the script are:
+
+```console
+# RunAFL.py -h
+usage: RunAFL.py [-h] [-a {IA32,X64,ARM,AARCH64}] [-b BUILDTARGET] [-m MODULEFILE] [-i INPUTSEED] [-o OUTPUT] [-c {rawcommand,manual}]
+
+options:
+  -h, --help            show this help message and exit
+  -a {IA32,X64,ARM,AARCH64}, --arch {IA32,X64,ARM,AARCH64}
+                        ARCHS is one of list: IA32, X64, ARM or AARCH64, which overrides target.txt's TARGET_ARCH definition.
+  -b BUILDTARGET, --buildtarget BUILDTARGET
+                        Using the TARGET to build the platform, overriding target.txt's TARGET definition.
+  -m MODULEFILE, --module MODULEFILE
+                        Build the module specified by the INF file name argument.
+  -i INPUTSEED, --input INPUTSEED
+                        Test input seed path.
+  -o OUTPUT, --output OUTPUT
+                        Test output path for AFL.
+  -c {rawcommand,manual}, --commandline {rawcommand,manual}
+                        This specifies how the fuzzer is initiated from command-line for Linux-based distributions. Specify either: 'rawcommand' or 'manual'. Using 'rawcommand' will directly initiate the AFL fuzzer
+                        after building the test module and is recommended for automated approaches. Alternatively, 'manual' may be used to simply build the test case and print out a command that the user can run to
+                        start the fuzzer (this will preserve the fuzzing display for AFL.
+```
+
+Here, we will need to reference a few files when running the script. For the TestBmpSupportLib test case, the following files should be referenced as arguments to the RunAFL.py command. **Note**, the full paths for the files should be specified.
+
+| File Location | Description |
+| ------------- | ----------- |
+| hbfa-fl/HBFA/UefiHostFuzzTestCasePkg/TestCase/MdeModulePkg/Library/BaseBmpSupportLib/TestBmpSupportLib.inf | This file is used with the '-m' option. This is the component description file for the test-case/model. This file will reference the sources, dependencies, etc. for this test-case. |
+| hbfa-fl/HBFA/UefiHostFuzzTestCasePkg/Seed/BMP/Raw | This folder is used with the '-i' option (for the input seed(s)). This directory contains several Bitmap image files (.BMP) that will serve as a fuzzing corpus. |
+
+Additionally, one must specify the target architecture (this example will target 'X64'), via the option '-a'. Further, the output directory for fuzzing session output should be provided via the '-o' option. If not specified, when RunAFL.py is invoked, the default assumption/value for the '--commandline' argument will be 'rawcommand', meaning the RunAFL.py script will automatically launch AFL after building the fuzzing harness.
+
+Putting all this together, the fuzzing test-case can be built and ran as shown:
+
+```console
+# RunAFL.py -a X64 -m /root/hbfa_workspace/hbfa-fl/HBFA/UefiHostFuzzTestCasePkg/TestCase/MdeModulePkg/Library/BaseBmpSupportLib/TestBmpSupportLib.inf -i /root/hbfa_workspace/hbfa-fl/HBFA/UefiHostFuzzTestCasePkg/Seed/BMP/Raw -o /tmp/fuzz_RunAFL_TestBmpSupportLib
+Start build Test Module:
+build -p UefiHostFuzzTestCasePkg/UefiHostFuzzTestCasePkg.dsc -m UefiHostFuzzTestCasePkg/TestCase/MdeModulePkg/Library/BaseBmpSupportLib/TestBmpSupportLib.inf -a X64 -b DEBUG -t AFL --conf /root/hbfa_workspace/hbfa-fl/HBFA/UefiHostFuzzTestPkg/Conf -t GCC5
+Build Successfully !!!
+
+Start run AFL test:
+afl-fuzz -i /root/hbfa_workspace/hbfa-fl/HBFA/UefiHostFuzzTestCasePkg/Seed/BMP/Raw -o /tmp/fuzz_RunAFL_TestBmpSupportLib /root/hbfa_workspace/Build/UefiHostFuzzTestCasePkg/DEBUG_AFL/X64/TestBmpSupportLib @@
+afl-fuzz 2.52b by <[email protected]>
+
+[+] Looks like we're not running on a tty, so I'll be a bit less verbose.
+...
+[+] Here are some useful stats:
+
+
+
+    Test case count : 8 favored, 0 variable, 10 total
+
+       Bitmap range : 23 to 94 bits (average: 64.50 bits)
+
+        Exec timing : 135 to 172 us (average: 147 us)
+
+
+
+[*] No -t option specified, so I'll use exec timeout of 20 ms.
+
+[+] All set and ready to roll!
+
+[*] Entering queue cycle 1.
+
+[*] Fuzzing test case #0 (10 total, 0 uniq crashes found)...
+
+[*] Fuzzing test case #1 (36 total, 0 uniq crashes found)...
+
+[*] Fuzzing test case #2 (37 total, 0 uniq crashes found)...
+...
+```
+
+Here, we note your full-path to the hbfa-fl directly may differ. Alternatively, if you wish to simply build the test case and run the fuzzer later (or with other options), you can invoke the RunAFL.py script with the '--commandline' argument 'manual' as show in the following.
+
+```console
+# RunAFL.py -a X64 -m /root/hbfa_workspace/hbfa-fl/HBFA/UefiHostFuzzTestCasePkg/TestCase/MdeModulePkg/Library/BaseBmpSupportLib/TestBmpSupportLib.inf -i /root/hbfa_workspace/hbfa-fl/HBFA/UefiHostFuzzTestCasePkg/Seed/BMP/Raw -o /tmp/fuzz_RunAFL_TestBmpSupportLib --commandline manual
+Start build Test Module:
+build -p UefiHostFuzzTestCasePkg/UefiHostFuzzTestCasePkg.dsc -m UefiHostFuzzTestCasePkg/TestCase/MdeModulePkg/Library/BaseBmpSupportLib/TestBmpSupportLib.inf -a X64 -b DEBUG -t AFL --conf /root/hbfa_workspace/hbfa-fl/HBFA/UefiHostFuzzTestPkg/Conf -t GCC5
+Build Successfully !!!
+
+Start run AFL test:
+Run this command to initiate the fuzzer: afl-fuzz -i /root/hbfa_workspace/hbfa-fl/HBFA/UefiHostFuzzTestCasePkg/Seed/BMP/Raw -o /tmp/fuzz_RunAFL_TestBmpSupportLib /root/hbfa_workspace/Build/UefiHostFuzzTestCasePkg/DEBUG_AFL/X64/TestBmpSupportLib @@
+```
+
+As can be seen in the output, a command is printed that can be copied and used to run the built-test case with 'afl-fuzz'. Adjust and run the command as needed to initiate a fuzzing session. E.g. for the present example, the following command will kick off the fuzzing session.
+
+```console
+# afl-fuzz -i /root/hbfa_workspace/hbfa-fl/HBFA/UefiHostFuzzTestCasePkg/Seed/BMP/Raw -o /tmp/fuzz_RunAFL_TestBmpSupportLib /root/hbfa_workspace/Build/UefiHostFuzzTestCasePkg/DEBUG_AFL/X64/TestBmpSupportLib @@
+```
+
+The fuzzing session can be terminated by pressing 'Ctrl+c'.
+
+### Examining the AFL output directory
+
+In the example provided in the prior sub-section, the fuzzing output was set to output to the directory '/tmp/fuzz_RunAFL_TestBmpSupportLib'. A file listing of this output directory is shown in the following.
+
+```console
+[root@92fd93545422 hbfa_workspace]# ls /tmp/fuzz_RunAFL_TestBmpSupportLib
+crashes  fuzz_bitmap  fuzzer_stats  hangs  plot_data  queue
+```
+
+This is the standard output directories and files per the AFL fuzzer. For more information on the output, see [Fuzzing with afl-fuzz](https://afl-1.readthedocs.io/en/latest/fuzzing.html). Some useful folder locations include the 'crashes' directory, which will include an input file suitable for triggering any of the crashes discovered. Likewise, the queue directory will contain input files (test cases) that will reach each code path discovered.
+
+[&lt;&lt;](../harness/includedfuzzharnesses.md) Back | Return to [Summary](../SUMMARY.md) | Next [&gt;&gt;](./fuzzingwithLibFuzzer.md)