Skip to content

Commit

Permalink
update README.md
Browse files Browse the repository at this point in the history
  • Loading branch information
Bodong-Yang committed Dec 25, 2023
1 parent d0555d8 commit 7e2c77b
Showing 1 changed file with 73 additions and 52 deletions.
125 changes: 73 additions & 52 deletions tools/images_packer/README.md
Original file line number Diff line number Diff line change
@@ -1,15 +1,28 @@
# Offline OTA image builder
# OTA images packer

`tools.offline_ota_image_builder` is a helper package for bundling offline OTA image from the OTA images build by ota-metadata(or images compatible with OTA image specification).
Built image can be used as external cache source for otaproxy, or used by full offline OTA helper util to trigger fully offline OTA update.
`tools.images_packer` is a helper package for bundling multiple images into one single images bundle.
The input OTA images are expected to be built by ota-metadata(or images compatible with OTA image specification).

**NOTE**: full offline OTA helper util is not yet implemented.
This package provide the following two packing mode, each for different purposes:
1. `build-cache-src`: with this mode, the built image can be recognized and utilized by otaproxy to reduce network downloading during OTA update.

Check failure on line 7 in tools/images_packer/README.md

View workflow job for this annotation

GitHub Actions / markdown_lint_check

Lists should be surrounded by blank lines [Context: "1. `build-cache-src`: with thi..."]

It provides the following features:
The following features are provided in this mode:

1. build offline OTA image from the OTA images built by ota-metadata(or images compatible with OTA image specification),
2. export built offline OTA image as tar archive,
3. prepare and export built image onto specific block device as external cache source device for otaproxy.
1. build external cache source image,

2. export the built image as tar archive,

3. prepare and export built image onto specific block device as external cache source device recognized by otaproxy.

2. `build-offline-ota-imgs-bundle`: with this mode, the built image can be used by the `fully offline OTA helper` script to trigger fullly offline OTA locally without any network.

The following feature are provided in this mode:

1. build offline OTA image,

2. export the built image as tar archive.

**NOTE**: full offline OTA helper util is not yet implemented.

<!--- Please check documentation at [OTA cache mechanism design for otaproxy](https://tier4.atlassian.net/l/cp/yzmnPx9T) for more details. -->

Expand All @@ -36,7 +49,7 @@ Example OTA image layout:

Please also refere to [ota-metadata](https://github.com/tier4/ota-metadata) repo for the implementation of OTA image layout specification.

## Offline OTA image layout
## OTA image bundle layout

The rootfs of the built image has the following layout:

Expand All @@ -55,11 +68,13 @@ The rootfs of the built image has the following layout:
```

This layout is compatible with external cache source's layout, which means any offline OTA image can natually directly be used as external cache source recognized by otaproxy(but user still needs to prepare the device either by themselves and extract the offline OTA image rootfs onto the prepared device, or prepare the device with this **offline_ota_image_builder** package).
Both external cache source image and offline OTA image bundle use this image bundle layout.
It means that offline OTA image can also be used as external cache source recognized by otaproxy(but user still needs to prepare the device either by themselves and extract the offline OTA image rootfs onto the prepared device, or prepare the device with this **images_packer** package).


Check failure on line 74 in tools/images_packer/README.md

View workflow job for this annotation

GitHub Actions / markdown_lint_check

Multiple consecutive blank lines [Expected: 1; Actual: 2]
<!--- Please check [External cache source](https://tier4.atlassian.net/wiki/spaces/WEB/pages/2813984854/OTA+cache+mechanism+design+for+otaproxy#External-cache-source) section of the doc for more details. --->

## Offline OTA image Manifest schema
## OTA image bundle Manifest schema

Offline OTA image's rootfs contains a `manifest.json` file with a single JSON object in it. This file includes the information related to this offline OTA image, including the bundled OTA images' metadata.

Expand All @@ -84,7 +99,7 @@ Offline OTA image's rootfs contains a `manifest.json` file with a single JSON ob

<!--- Please check [Build external cache source](https://tier4.atlassian.net/wiki/spaces/WEB/pages/2813984854/OTA+cache+mechanism+design+for+otaproxy#Build-external-cache-source) section for more details. --->

## How to use
## How to use `images_packer`

### Prepare the dependencies

Expand All @@ -111,65 +126,71 @@ This image builder requires latest ota-client to be installed/accessable. The re
(venv) $ pip install ota-client
```

### Build image and export
### Examples of building and exporting image

#### Builder usage

```text
usage: offline_ota_image_builder [-h] --image <ECU_NAME>:<IMAGE_PATH>[:<IMAGE_VERSION>]
[-o <OUTPUT_PATH>] [-w <DEVICE>] [--confirm-write-to]
#### Usage 1: Build external cache source image and export it as tar archive

Helper script that builds offline OTA image with given OTA image(s) as external cache source or for
offline OTA use.
options:
-h, --help show this help message and exit
--image <ECU_NAME>:<IMAGE_PATH>[:<IMAGE_VERSION>]
OTA image for <ECU_ID> as tar archive(compressed or uncompressed), this
option can be used multiple times to include multiple images.
-o <OUTPUT_PATH>, --output <OUTPUT_PATH>
save the generated image rootfs into tar archive to <OUTPUT_PATH>.
-w <DEVICE>, --write-to <DEVICE>
write the image to <DEVICE> and prepare the device as external cache source
device.
--force-write-to prepare <DEVICE> as external cache source device without inter-active confirmation,
only valid when used with -w option.
```bash
# current folder layout: venv ota-client
(venv) $ cd ota-client
(venv) $ sudo -E env PATH=$PATH python3 -m tools.image_packer build-cache-src --image-dir=./images_dir --output=t2.tar
```

Execute the image builder by directly calling it from the source code. This package requires `root` permission to run(required by extracting OTA image and preparing external cache source device).

Option `--image <ECU_NAME>:<IMAGE_PATH>[:<IMAGE_VERSION>]` is used to specify OTA image to be included. This option can be used multiple times to specify multiple OTA images.

Option `--write-to <DEVICE>` is used to prepare external cache source device used by otaproxy. The specified device will be formatted as `ext4`, fslabel with `ota_cache_src`, and be exported with the built offline OTA image's rootfs. If this package is used in a non-interactive script, option `--force-write-to` can be used to bypass interactive confirmation.
This will build the external cache source image with images listed in `./images_dir`, and export the built image as `t2.tar` tar archive.

Option `--output <OUTPUT>` specify where to save the exported tar archive of built image.
User must at least specifies one of `--write-to` and `--output`, or specifies them together.
#### Usage 1: Build offline OTA image and export it as tar archive
#### Usage 2: Build offline OTA images bundle and export it as tar archive

```bash
# current folder layout: venv ota-client
(venv) $ cd ota-client
(venv) $ sudo -E env PATH=$PATH python3 -m tools.offline_ota_image_builder --image=p1:p1_image.tgz:ver_20230808 --image=p2:p2_image.tgz:ver_20230808 --output=t2.tar
(venv) $ sudo -E env PATH=$PATH python3 -m tools.image_packer build-offline-ota-imgs-bundle --image=p1:p1_image.tgz:ver_20230808 --image=p2:p2_image.tgz:ver_20230808 --output=t2.tar
```

This will build the offline OTA image with `p1_image.tgz` and `p2_image.tgz`, which are for ECU `p1` and `p2`, and export the built image as `t2.tar` tar archive.

### Usage 2: Build the offline OTA image and create external cache source dev
#### Usage 3: Build the external cache source image and create external cache source dev

```bash
# current folder layout: venv ota-client
(venv) $ cd ota-client
(venv) $ sudo -E env PATH=$PATH python3 -m tools.offline_ota_image_builder --image=p1:p1_image.tgz:ver_20230808 --image=p2:p2_image.tgz:ver_20230808 --write-to=/dev/<target_dev>
(venv) $ sudo -E env PATH=$PATH python3 -m tools.image_packer build-cache-src --image-dir=./images_dir --write-to=/dev/<target_dev>
```

This will build the offline OTA image with `p1_image.tgz` and `p2_image.tgz`, which are for ECU `p1` and `p2`, and then prepare the `/dev/<target_dev>` as external cache source device(ext4 filesystem labelled with `ota_cache_src`) with image rootfs exported to the filesystem on `/dev/<target_device>`.
This will build the external cache source image OTA images listed in the `./images_dir`, and prepare the `/dev/<target_dev>` as external cache source device(ext4 filesystem labelled with `ota_cache_src`) with image rootfs exported to the filesystem on `/dev/<target_device>`.

### Usage 3: Build the offline OTA image, export it as tar archive and prepare external cache source dev
## CLI Usage reference

```bash
# current folder layout: venv ota-client
(venv) $ cd ota-client
(venv) $ sudo -E env PATH=$PATH python3 -m tools.offline_ota_image_builder --image=p1:p1_image.tgz:ver_20230808 --image=p2:p2_image.tgz:ver_20230808 --output=t2.tar --write-to=/dev/<target_dev>
### `build-cache-src` subcommand

```text
usage: images_packer build-cache-src [-h] [--image <IMAGE_FILE_PATH>] [--image-dir <IMAGE_FILES_DIR>] [-o <OUTPUT_FILE_PATH>] [-w <DEVICE>] [--force-write-to]
build external OTA cache source recognized and used otaproxy.
optional arguments:
-h, --help show this help message and exit
--image <IMAGE_FILE_PATH>
(Optional) OTA image to be included in the external cache source, this argument can be specified multiple times to include multiple images.
--image-dir <IMAGE_FILES_DIR>
(Optional) Specify a dir of OTA images to be included in the cache source.
-o <OUTPUT_FILE_PATH>, --output <OUTPUT_FILE_PATH>
save the generated image bundle tar archive to <OUTPUT_FILE_PATH>.
-w <DEVICE>, --write-to <DEVICE>
(Optional) write the external cache source image to <DEVICE> and prepare the device, and then prepare the device to be used as external cache source storage.
--force-write-to (Optional) prepare <DEVICE> as external cache source device without inter-active confirmation, only valid when used with -w option.
```

### `build-offline-ota-imgs-bundle` subcommand

```text
usage: images_packer build-offline-ota-imgs-bundle [-h] --image <ECU_NAME>:<IMAGE_PATH>[:<IMAGE_VERSION>] [-o <OUTPUT_FILE_PATH>]
build OTA image bundle for offline OTA use.
optional arguments:
-h, --help show this help message and exit
--image <ECU_NAME>:<IMAGE_PATH>[:<IMAGE_VERSION>]
OTA image for <ECU_ID> as tar archive(compressed or uncompressed), this option can be used multiple times to include multiple images.
-o <OUTPUT_FILE_PATH>, --output <OUTPUT_FILE_PATH>
save the generated image bundle tar archive to <OUTPUT_FILE_PATH>.
```

0 comments on commit 7e2c77b

Please sign in to comment.