Merge pull request #950 from OpenC3/key_concepts

Add Key Concepts page

docs.openc3.com/docs/configuration/protocols.md

@@ -64,6 +64,14 @@ The Length Protocol depends on a length field at a fixed location in the defined
 | Max Length                   | The maximum allowed value in the length field                                                                                                                                                                                                                                                                                                                                                                                | No       | nil (no maximum length)  |
 | Fill Length and Sync Pattern | Setting this flag to true causes the length field and sync pattern (if present) to be filled automatically on outgoing packets.                                                                                                                                                                                                                                                                                              | No       | false                    |
 
+The most confusing aspect of the Length Protocol is calculating the Length Value Offset. This is especially true in the commonly used CCSDS Space Packet Protocol. The best way to illustrate this is with an example. Suppose you have CCSDS Space Packets prepended with a Sync Pattern of 0x1ACFFC1D. This would look like the following:
+
+| Sync (4 bytes) | Header (4 bytes) | Length (2 bytes) | Data (4 bytes) |
+| -------------- | ---------------- | ---------------- | -------------- |
+| 0x1ACFFC1D     | 0x0001CADB       | 0x0003           | 0xDEADBEEF     |
+
+In this case the total length of the packet is 14 bytes: **4 + 4 + 2 + 4 = 14**. With 4 bytes of data, the length field is 3 because in CCSDS the length field is calculated as (data length - 1). So how would we calculate the Length Value Offset? COSMOS reads all the bytes in the packet (including the Sync Pattern) so the total length is 14 bytes. The length field is 3 so the Length Value Offset (offset to apply to the length field value) should be 11 (**3 + 11 = 14**).
+
 ### Terminated Protocol
 
 The Terminated Protocol delineates packets using termination characters found at the end of every packet. It continuously reads data until the termination characters are found at which point it returns the packet data. For example, all the packets using the interface are followed by 0xABCD. This data can either be a part of each packet that is kept or something which is known only by the Terminated Protocol and simply thrown away.

docs.openc3.com/docs/getting-started/gettingstarted.md

@@ -41,7 +41,7 @@ This guide assumes we're on Windows and COSMOS is installed in C:\COSMOS. Adjust
 
 1. Before creating your own configuration you should uninstall the COSMOS Demo so you're working with a clean COSMOS system. Click the Admin button and the PLUGINS tab. Then click the Trash can icon next to openc3-cosmos-demo to delete it. When you go back to the Command and Telemetry Server you should have a blank table with no interfaces.
 
-1. If you followed the [Installation Guide](installation.md) you should already be inside a cloned [openc3-project](https://github.com/OpenC3/cosmos-project). Inside this project it's recommended to edit the README.md ([Markdown](https://www.markdownguide.org/)) to describe your program / project.
+1. If you followed the [Installation Guide](installation.md) you should already be inside a cloned [openc3-project](https://github.com/OpenC3/cosmos-project) which is in your PATH (necessary for openc3.bat / openc3.sh to be resolved). Inside this project it's recommended to edit the README.md ([Markdown](https://www.markdownguide.org/)) to describe your program / project.
 
 1. Now we need to create a plugin. Plugins are how we add targets and microservices to COSMOS. Our plugin will contain a single target which contains all the information defining the packets (command and telemetry) that are needed to communicate with the target. Use the COSMOS plugin generator to create the correct structure:
 

docs.openc3.com/docs/getting-started/installation.md

@@ -34,15 +34,15 @@ Docker by default will break idle (no data) connections after a period of 5 minu
 
 Since the COSMOS 5.0.9 release we recommend using the project template to get started.
 
-        git clone https://github.com/OpenC3/cosmos-project.git tutorial
+        git clone https://github.com/OpenC3/cosmos-project.git
 
 :::info Offline Installation
 
   <p style={{"margin-bottom": 20 + 'px'}}>If you need to install in an offline environment you should first see if you're able to directly use the COSMOS containers. If so you can first save the containers:</p>
 
   <p style={{"margin-bottom": 20 + 'px'}}><code>./openc3.sh util save</code></p>
 
-  <p style={{"margin-bottom": 20 + 'px'}}>By default this will download the 'latest' images and create tar files in the 'tmp' directory which you can transfer to your offline environment. You can pass a release after 'save' to download a specific release (e.g. util save 5.0.9). Transfer the tar files to your offline environment's project 'tmp' dir and  import them with:</p>
+  <p style={{"margin-bottom": 20 + 'px'}}>By default this will download the 'latest' images and create tar files in the 'tmp' directory which you can transfer to your offline environment. You can pass a release after 'save' to download a specific release (e.g. util save 5.12.0). Transfer the tar files to your offline environment's project 'tmp' dir and  import them with:</p>
 
   <p style={{"margin-bottom": 20 + 'px'}}><code>./openc3.sh util load</code></p>
 :::
@@ -55,7 +55,7 @@ The COSMOS containers are designed to work and be built in the presence of an SS
 
 Increasingly organizations are using some sort of SSL decryptor device which can cause curl and other command line tools like git to have SSL certificate problems. If installation fails with messages that involve "certificate", "SSL", "self-signed", or "secure" this is the problem. IT typically sets up browsers to work correctly but not command line applications. Note that the file extension might not be .pem, it could be .pem, crt, .ca-bundle, .cer, .p7b, .p7s, or potentially something else.
 
-The workaround is to get a proper local certificate file from your IT department that can be used by tools like curl (for example mine is at C:\Shared\Ball.pem). Doesn't matter just somewhere with no spaces.
+The workaround is to get a proper local certificate file from your IT department that can be used by tools like curl (for example C:\Shared\Ball.pem). Doesn't matter just somewhere with no spaces.
 
 Then set the following environment variables to that path (ie. C:\Shared\Ball.pem)
 
@@ -70,6 +70,8 @@ You will need to create new ones with the names above and set their value to the
 
 ### RUN
 
+Add the locally cloned project directory to your path so you can directly use the batch file or shell script. In Windows this would be adding "C:\openc3-project" to the PATH. In Linux you would edit your shell's rc file and export the PATH. For example, on a Mac add the following to ~/.zshrc: `export PATH=~/cosmos-project:$PATH`.
+
 Run `openc3.bat run` (Windows), or `./openc3.sh run` (linux/Mac).
 
 Note, you can edit the .env file and change OPENC3_TAG to a specific release (e.g. 5.0.9) rather than 'latest'.

docs.openc3.com/docs/getting-started/key_concepts.md

@@ -0,0 +1,84 @@
+---
+sidebar_position: 4
+title: Key Concepts
+---
+
+# OpenC3 COSMOS Key Concepts
+
+## Projects
+
+The main COSMOS [repo](https://github.com/OpenC3/cosmos) contains all the source code used to build and run COSMOS. However, users (not developers) of COSMOS should use the COSMOS [project](https://github.com/OpenC3/cosmos-project) to launch COSMOS. The project consists of the [openc3.sh](https://github.com/OpenC3/cosmos-project/blob/main/openc3.sh) and [openc3.bat](https://github.com/OpenC3/cosmos-project/blob/main/openc3.bat) files for starting and stopping COSMOS, the [compose.yaml](https://github.com/OpenC3/cosmos-project/blob/main/compose.yaml) for configuring the COSMOS containers, and the [.env](https://github.com/OpenC3/cosmos-project/blob/main/.env) file for setting runtime variables. Additionally, the COSMOS project contains user modifiable config files for both Redis and Traefik.
+
+## Containerization
+
+### Images
+
+Per [Docker](https://docs.docker.com/get-started/overview/#images), "An image is a read-only template with instructions for creating a Docker container." The base operating system COSMOS uses is called [Alpine Linux](https://www.alpinelinux.org/). It is a simple and compact image with a full package system that allows us to install our dependencies. Starting with Alpine, we create a [Dockerfile](https://docs.docker.com/engine/reference/builder/) to add Ruby and Python and a few other packages to create our own docker image. We further build upon that image to create a NodeJS image to support our frontend and additional images to support our backend.
+
+### Containers
+
+Per [Docker](https://www.docker.com/resources/what-container/), "a container is a standard unit of software that packages up code and all its dependencies so the application runs quickly and reliably from one computing environment to another." Also per [Docker](https://docs.docker.com/guides/walkthroughs/what-is-a-container/), "A container is an isolated environment for your code. This means that a container has no knowledge of your operating system, or your files. It runs on the environment provided to you by Docker Desktop. Containers have everything that your code needs in order to run, down to a base operating system." COSMOS utilizes containers to provide a consistent runtime environment. Containers make it easy to deploy to local on-prem servers, cloud environments, or air-gapped networks.
+
+The COSMOS Open Source containers consist of the following:
+
+| Name                                     | Description                                                                                            |
+| ---------------------------------------- | ------------------------------------------------------------------------------------------------------ |
+| cosmos-openc3-cosmos-init-1              | Copies files to Minio and configures COSMOS then exits                                                 |
+| cosmos-openc3-operator-1                 | Main COSMOS container that runs the interfaces and target mircoservices                                |
+| cosmos-openc3-cosmos-cmd-tlm-api-1       | Rails server that provides all the COSMOS API endpoints                                                |
+| cosmos-openc3-cosmos-script-runner-api-1 | Rails server that provides the Script API endpoints                                                    |
+| cosmos-openc3-redis-1                    | Serves the static target configuration                                                                 |
+| cosmos-openc3-redis-ephemeral-1          | Serves the [streams](https://redis.io/docs/data-types/streams) containing the raw and decomutated data |
+| cosmos-openc3-minio-1                    | Provides a S3 like bucket storage interface and also serves as a static webserver for the tool files   |
+| cosmos-openc3-traefik-1                  | Provides a reverse proxy and load balancer with routes to the COSMOS endpoints                         |
+
+The container list for [Enterprise COSMOS](https://openc3.com/enterprise) consists of the following:
+
+| Name                                  | Description                                                                                   |
+| ------------------------------------- | --------------------------------------------------------------------------------------------- |
+| cosmos-enterprise-openc3-metrics-1    | Rails server that provides metrics on COSMOS performance                                      |
+| cosmos-enterprise-openc3-keycloak-1   | Single-Sign On service for authentication                                                     |
+| cosmos-enterprise-openc3-postgresql-1 | SQL Database for use by Keycloak                                                              |
+| openc3-nfs \*                         | Network File System pod only for use in Kubernetes to share code libraries between containers |
+
+### Docker Compose
+
+Per [Docker](https://docs.docker.com/compose/), "Compose is a tool for defining and running multi-container Docker applications. With Compose, you use a YAML file to configure your application's services. Then, with a single command, you create and start all the services from your configuration." OpenC3 uses compose files to both build and run COSMOS. The [compose.yaml](https://github.com/OpenC3/cosmos-project/blob/main/compose.yaml) is where ports are exposed and environment variables are used.
+
+### Environment File
+
+COSMOS uses an [environment file](https://docs.docker.com/compose/environment-variables/env-file/) along with Docker Compose to pass environment variables into the COSMOS runtime. This [.env](https://github.com/OpenC3/cosmos-project/blob/main/.env) file consists of simple key value pairs that contain the version of COSMOS deployed, usernames and passwords, and much more.
+
+### Kubernetes
+
+Per [Kubernetes.io](https://kubernetes.io/), "Kubernetes, also known as K8s, is an open-source system for automating deployment, scaling, and management of containerized applications. It groups containers that make up an application into logical units for easy management and discovery." [COSMOS Enterprise](https://openc3.com/enterprise) provides [Helm charts](https://helm.sh/docs/topics/charts/) for easy deployment to Kubernetes in various cloud environments.
+
+COSMOS Enterprise also provides [Terraform](https://www.terraform.io/) scripts to deploy COSMOS infrastructure on various cloud environments.
+
+## Frontend
+
+### Vue.js
+
+The COSMOS frontend is fully browser native and is implented in the Vue.js framework. Per [Vue.js](https://vuejs.org/guide/introduction.html), "Vue is a JavaScript framework for building user interfaces. It builds on top of standard HTML, CSS, and JavaScript and provides a declarative and component-based programming model that helps you efficiently develop user interfaces, be they simple or complex." COSMOS utilizes Vue.js and the [Vuetify](https://vuetifyjs.com/en/) Component Framework UI library to build all the COSMOS tools which run in the browser of your choice.
+
+### Single-Spa
+
+While COSMOS itself is written in Vue.js, we utilize a technology called [single-spa](https://single-spa.js.org/) to allow COSMOS developers to create applications in any javascript framework they choose. Single-spa is a micro frontend framework and acts as a top level router to render the application being requested. COSMOS provides sample applications ready to plug into single-spa in Angular, React, Svelt, and Vue.
+
+### Astro UX
+
+Per [AstroUXDS](https://www.astrouxds.com/), "The Astro Space UX Design System enables developers and designers to build rich space app experiences with established interaction patterns and best practices." COSMOS utilizes the Astro design guidelines for color, typograpy, and iconograpy. In some cases, e.g. [Astro Clock](https://www.astrouxds.com/components/clock/), COSMOS directly incorporates Astro components.
+
+## Backend
+
+### Redis
+
+[Redis](https://redis.io/) is an in-memory data store with support for strings, hashes, lists, sets, sorted sets, streams, and more. COSMOS uses Redis to store both our configuration and data. If you look back at our [container list](/docs/getting-started/key_concepts#containers) you'll notice two redis containers: cosmos-openc3-redis-1 and cosmos-openc3-redis-ephemeral-1. The ephemeral container contains all the real-time data pushed into [Redis streams](https://redis.io/docs/data-types/streams/). The other redis container contains COSMOS configuration that is meant to persist. [COSMOS Enterprise](https://openc3.com/enterprise) provides helm charts that setup [Redis Cluster](https://redis.io/docs/management/scaling/) to perform horizontal scaling where data is shared across multiple Redis nodes.
+
+### MinIO
+
+[MinIO](https://min.io/) is a high-performance, S3 compatible object store. COSMOS uses this storage technology to host both the COSMOS tools themselves and the long term log files. [COSMOS Enterprise](https://openc3.com/enterprise) deployed in a cloud environment uses the available cloud native bucket storage technology, e.g. AWS S3, GCP Buckets, and Azure Blob Storage. Using bucket storage allows COSMOS to directly serve the tools as a static website and thus we don't need to deploy Tomcat or Nginx for example.
+
+### Ruby on Rails
+
+The COSMOS API and Script Runner backends are powered by [Ruby on Rails](https://rubyonrails.org/). Rails is a web application development framework written in the Ruby programming language. Rails (and our familiarity with Ruby) allows us to write less code while accomplishing more than many other languages and frameworks.

docs.openc3.com/docs/getting-started/podman.md

@@ -1,5 +1,5 @@
 ---
-sidebar_position: 5
+sidebar_position: 6
 title: Podman
 ---
 

docs.openc3.com/docs/getting-started/requirements.md

@@ -1,5 +1,5 @@
 ---
-sidebar_position: 4
+sidebar_position: 5
 title: Requirements and Design
 ---