Skip to content

Commit

Permalink
HDDS-11631. Fix review comments
Browse files Browse the repository at this point in the history
  • Loading branch information
ptlrs committed Jan 10, 2025
1 parent 787bfa6 commit d5b9206
Showing 1 changed file with 90 additions and 34 deletions.
124 changes: 90 additions & 34 deletions docs/08-developer-guide/02-run/02-docker-compose.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,30 +2,64 @@
sidebar_label: Docker Compose
---

<!-- cspell:words xzf -->

# Running Ozone From Docker Compose

import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';

This guide explains how to run Apache Ozone using Docker Compose, either with locally built sources or pre-built images.
This guide walks you through the process of building Apache Ozone from source and running it using Docker Compose. This approach is particularly useful for development, testing, and understanding Ozone's architecture.

## Prerequisites

- [Docker Engine](https://www.docker.com/products/docker-desktop/) 20.10.0 or higher
- [Docker Compose](https://docs.docker.com/compose/install/) V2
- Built Ozone distribution (if running from local build)
Before you begin, ensure you have installed:

## Running Ozone
- [Docker Engine](https://docs.docker.com/engine/install/) - Latest stable version
- [Docker Compose](https://docs.docker.com/compose/install/) - Latest stable version

Follow the [Try Ozone with Docker](/docs/02-quick-start/01-installation/01-docker.md) quick start guide to run Ozone via Docker Compose
## Building and Deploying Ozone

## Container Diagram
### Step 1: Build from Source

First, build Ozone from source following our [Build with Maven](/docs/08-developer-guide/01-build/02-maven.md) guide.

### Step 2: Locate Docker Compose Files

Navigate to the compose directory in your build output:

```bash
cd ./hadoop-ozone/dist/target/ozone-<VERSION>/compose/ozone
```

### Step 3: Configure Your Deployment (Optional)

This image shows the containers that will be created by the `docker compose up -d` command when running the default `ozone` Docker Compose configuration as listed under the [Try Ozone with Docker](/docs/02-quick-start/01-installation/01-docker.md) quick start guide.
You can customize your Ozone deployment by modifying the configuration parameters in the `docker-compose.yaml` file:

<!-- cspell:word DN -->
1. **Common Configurations**: Located under the `x-common-config` section
2. **Service-Specific Settings**: Found under the `environment` section of individual services

Example configuration modification:

```yaml
# Example configuration modifications
x-common-config:
environment:
OZONE-SITE.XML_ozone.scm.container.size: 1GB
OZONE-SITE.XML_ozone.scm.block.size: 256MB
```
### Step 4: Launch the Cluster
Start your Ozone cluster:
```bash
docker compose up -d --scale datanode=3
```

This command creates a fully functional Apache Ozone cluster using the `ozone-docker-runner` base image, which mounts your locally compiled Ozone binaries.

## Container Diagram

This image shows the containers that will be created by the `docker compose up -d` command when running the default Docker Compose configuration under `/compose/ozone` .

```mermaid
graph TB
Expand Down Expand Up @@ -62,52 +96,74 @@ graph TB
class scm,om manager;
```

## Cluster Configuration
## Cluster Components

### Default Services
### Core Services

The default Docker Compose configuration includes:
Your Ozone cluster includes the following components:

- Storage Container Manager (SCM)
- Ozone Manager (OM)
- S3 Gateway
- Recon (Monitoring Service)
- Datanodes
- HttpFS
- **Storage Container Manager (SCM)**: Manages storage containers and block allocation
- **Ozone Manager (OM)**: Handles namespace operations and metadata management
- **S3 Gateway**: Provides S3-compatible API access
- **Recon**: Monitoring and management service
- **DataNodes**: Distributed storage nodes
- **HttpFS**: HTTP-based filesystem interface

## Cluster Management

Common Docker Compose commands:
### Essential Commands

Here are the key commands for managing your Ozone cluster:

```bash
# Start the cluster
docker compose up -d

# Stop the cluster
# Stop and remove all containers
docker compose down

# View service logs
# Monitor service logs
docker compose logs -f [service_name]

# Scale data nodes
# Scale the number of datanodes
docker compose up -d --scale datanode=3

# Check service status
# Check cluster status
docker compose ps
```

## Next Steps
### CLI Access

This page explains the Docker Compose configuration for a basic Ozone cluster.
You can next explore some of the other Docker Compose configurations that are available under the compose directory.
Access the Ozone command-line interface from any ozone container:

```bash
# Enter the Ozone Manager container or any other container
docker compose exec om bash

# Run Ozone commands
ozone
```

## Advanced Configurations

The compose directory includes several specialized configurations for different use cases:

| Configuration | Purpose |
|----------------|-----------------------------------------|
| ozone-ha | High availability deployment setup |
| ozonesecure | Security features with SSL and Kerberos |
| ozone-topology | Rack-aware deployment configuration |
| upgrade | Non-rolling upgrade testing environment |

To explore these configurations:

```bash
cd hadoop-ozone/dist/target/ozone-*-SNAPSHOT/compose/
```

| Docker Compose configuration | Description |
|--------------|-------------|
| ozone-ha | Explore Ozone high availability with this configuration |
| ozone-secure | Explore various SSL certificate and Kerberos configurations |
| topology | Explore the rack-aware configuration |
| upgrade | Explore the non-rolling upgrade configuration |
## Next Steps

After setting up your development cluster:

1. Explore the [Ozone CLI documentation](/docs/04-user-guide/02-clients/01-ozone.md)
2. Experiment with the various compose configurations for specific use cases

0 comments on commit d5b9206

Please sign in to comment.