-
Notifications
You must be signed in to change notification settings - Fork 1
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Update tag versions. Update examples. Update Readme.
- Loading branch information
1 parent
f956231
commit 970ddf9
Showing
8 changed files
with
165 additions
and
153 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,11 +1,11 @@ | ||
# Kubernetes logging by Loki stack (Loki+Promtail) | ||
|
||
Terraform module for logging your kubernetes cluster resources. | ||
Terraform module for deploy Loki logging to your kubernetes cluster, with multi cloud storage support. | ||
|
||
## Wokrflow | ||
|
||
Module creates all necessary resources for logging important containers inside your kubernetes cluster. Previously you need to have Grafana to see your logs. Loki is only separate Data Source for Grafana. | ||
Module supports logging to AWS s3 bucket, GCP storage, Azure blob storage and Kubernetes Persistent Volume. | ||
Module supports different storages for logs: AWS S3 bucket, GCP GCS Bucket, Azure Blob storage and Kubernetes Persistent Volume. | ||
|
||
## Software Requirements | ||
|
||
|
@@ -16,92 +16,71 @@ Helm provider | >= 2.1.0 | |
Kubernetes provider | >= v2.0.1 | ||
|
||
## Usage | ||
#### AWS with S3 as storage | ||
``` | ||
# AWS s3 Loki Logging | ||
module "aws_s3_loki_stack" { | ||
source = "terraform-iaac/loki-stack/kubernetes" | ||
loki_node_selector = { | ||
(local.node_spot_label_key) = false | ||
(local.node_multi_az_label_key) = true | ||
} | ||
# In case if IRSA is enabled | ||
# In case if IRSA is enabled. IRSA must have S3 RW Policy access. | ||
# Otherwise, your instance must have S3 RW Policy attached. | ||
loki_service_account_annotations = { | ||
"eks.amazonaws.com/role-arn" = "arn:aws:iam::123456789:role/loki-logging" | ||
} | ||
loki_resources = { | ||
request_cpu = "50m" | ||
request_memory = "100Mi" | ||
} | ||
provider_type = "aws" | ||
s3_name = "s3-bucket-loki-logs" | ||
s3_region = "us-east-1" | ||
} | ||
# Google Cloud Storage Loki Logging | ||
``` | ||
#### GCP with GCS as storage | ||
``` | ||
module "gcs_loki_stack" { | ||
source = "terraform-iaac/loki-stack/kubernetes" | ||
loki_node_selector = { | ||
(local.node_spot_label_key) = false | ||
(local.node_multi_az_label_key) = true | ||
} | ||
# In case if Workload Identity is enabled. | ||
# Otherwise, your node must have RW permissions to GCS. | ||
loki_service_account_annotations = { | ||
"iam.gke.io/gcp-service-account" = "[email protected]" | ||
} | ||
promtail_resources = { | ||
request_cpu = "20m" | ||
request_memory = "50Mi" | ||
} | ||
provider_type = "gcp" | ||
gcs_bucket_name = "k8s-logging" | ||
gcs_bucket_name = "gcs-bucket-loki-logs" | ||
} | ||
# Azure Loki Logging | ||
``` | ||
#### Azure with Blob as storage | ||
``` | ||
module "azure_loki_stack" { | ||
source = "terraform-iaac/loki-stack/kubernetes" | ||
loki_node_selector = { | ||
(local.node_spot_label_key) = false | ||
(local.node_multi_az_label_key) = true | ||
} | ||
provider_type = "azure" | ||
storage_account_name = "kuberneteslogging" | ||
storage_account_access_key = "super-secret-key" | ||
container_name = "logs" | ||
} | ||
``` | ||
|
||
# Local Loki Logging | ||
module "local_loki_stack" { | ||
source = "terraform-iaac/loki-stack/kubernetes" | ||
loki_node_selector = { | ||
(local.node_spot_label_key) = false | ||
(local.node_multi_az_label_key) = true | ||
} | ||
#### PV local as storage | ||
``` | ||
module "pv_local_loki_stack" { | ||
source = "terraform-iaac/loki-stack/kubernetes" | ||
provider_type = "local" | ||
persistent_volume_name = kubernetes_persistent_volume.test.metadata.0.name | ||
persistent_volume_size = "4Gi" // We recommend to use a bit smaller value than Persistent Volume have (example: current_PV*0.9) | ||
pvc_storage_class_name = "default" | ||
pvc_access_modes = ["ReadWriteOnce"] | ||
persistent_volume_name = kubernetes_persistent_volume.pv_loki.metadata.0.name | ||
persistent_volume_size = "4Gi" | ||
} | ||
``` | ||
### Note: ***provider_type*** support only ***aws, azure, gcp or local*** value. Every value require own variables (see ***locals*** section in varaibles.tf file) | ||
### Note: ***provider_type*** supports only ***aws, azure, gcp or local*** value. Every value require own variables (see ***locals*** section in varaibles.tf file or check examples.) | ||
|
||
## Inputs | ||
|
||
Name | Description | Type | Default | Example | Required | ||
--- | --- |----------| --- |--- |--- | ||
namespace | Name of namespace where you want to deploy loki-stack | `string` | `monitoring` | n/a | no | ||
create_namespace | Create namespace by module? true or false | `bool` | true | n/a | no | ||
loki_resources | Compute Resources required by loki container. CPU/RAM requests | `map` | `{}` | <pre>{<br> request_cpu = "20m"<br> request_memory = "50Mi"<br>}</pre> | no | ||
promtail_resources | Compute Resources required by promtail container. CPU/RAM requests | `map` | `{}` | <pre>{<br> request_cpu = "20m"<br> request_memory = "50Mi"<br>}</pre> | no | ||
Name | Description | Type | Default | Example | Required | ||
--- | --- |----------|-------------------------------------------------------------------------------|--- |--- | ||
namespace | Name of namespace where you want to deploy loki-stack | `string` | `monitoring` | n/a | no | ||
create_namespace | Create namespace by module? true or false | `bool` | true | n/a | no | ||
loki_resources | Compute Resources required by loki container. CPU/RAM requests | `map` | <pre>{<br> request_cpu = "50m"<br> request_memory = "100Mi"<br>}</pre> | <pre>{<br> request_cpu = "20m"<br> request_memory = "50Mi"<br>}</pre> | no | ||
promtail_resources | Compute Resources required by promtail container. CPU/RAM requests | `map` | <pre>{<br> request_cpu = "20m"<br> request_memory = "50Mi"<br>}</pre> | <pre>{<br> request_cpu = "20m"<br> request_memory = "50Mi"<br>}</pre> | no | ||
|
||
### Loki variables | ||
Name | Description | Type | Default | Example | Required | ||
|
@@ -125,7 +104,7 @@ Name | Description | Type | Default | Example | Required | |
--- | --- | --- | --- |--- |--- | ||
provider_type | Choose what type of provider you want (aws, azure, gcp and local) | `string` | n/a | `azure` | yes | ||
|
||
### ***AWS s3*** | ||
### ***AWS S3*** | ||
Name | Description | Type | Default | Example | Required | ||
--- | --- | --- | --- |--- |--- | ||
s3_region | AWS region where s3 locate | `string` | `null` | `us-east-1` | no | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,14 @@ | ||
# AWS S3 Loki Logging | ||
module "aws_s3_loki_stack" { | ||
source = "terraform-iaac/loki-stack/kubernetes" | ||
|
||
# In case if IRSA is enabled. IRSA must have S3 RW Policy access. | ||
# Otherwise, your instance must have S3 RW Policy attached. | ||
loki_service_account_annotations = { | ||
"eks.amazonaws.com/role-arn" = "arn:aws:iam::123456789:role/loki-logging" | ||
} | ||
|
||
provider_type = "aws" | ||
s3_name = "s3-bucket-loki-logs" | ||
s3_region = "us-east-1" | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,8 @@ | ||
module "azure_loki_stack" { | ||
source = "terraform-iaac/loki-stack/kubernetes" | ||
|
||
provider_type = "azure" | ||
storage_account_name = "kuberneteslogging" | ||
storage_account_access_key = "super-secret-key" | ||
container_name = "logs" | ||
} |
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,41 @@ | ||
# Google Cloud Storage Loki Logging | ||
# Create GCP Service Account for LOKI | ||
resource "google_service_account" "loki" { | ||
account_id = "loki" | ||
display_name = "Loki Logging" | ||
description = "SA for Loki Logging with GCS Access" | ||
} | ||
|
||
# Assign permission to service account | ||
resource "google_storage_bucket_access_control" "loki" { | ||
bucket = "gcs-bucket-loki-logs" | ||
role = "WRITER" | ||
entity = "user-${google_service_account.loki.email}" | ||
} | ||
resource "google_storage_bucket_iam_member" "loki" { | ||
bucket = "gcs-bucket-loki-logs" | ||
role = "roles/storage.objectAdmin" | ||
member = "serviceAccount:${google_service_account.loki.email}" | ||
} | ||
resource "google_service_account_iam_binding" "loki_workload_identity" { | ||
service_account_id = google_service_account.loki.name | ||
|
||
role = "roles/iam.workloadIdentityUser" | ||
|
||
members = [ | ||
"serviceAccount:${var.gke_workload_identity_pool}[${module.gcs_loki_stack.namespace}/${module.gcs_loki_stack.loki_service_account_name}]", | ||
] | ||
} | ||
|
||
module "gcs_loki_stack" { | ||
source = "terraform-iaac/loki-stack/kubernetes" | ||
|
||
# In case if Workload Identity is enabled. | ||
# Otherwise, your node must have RW permissions to GCS. | ||
loki_service_account_annotations = { | ||
"iam.gke.io/gcp-service-account" = google_service_account.loki.email | ||
} | ||
|
||
provider_type = "gcp" | ||
gcs_bucket_name = "gcs-bucket-loki-logs" | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,27 @@ | ||
resource "kubernetes_persistent_volume" "pv_loki" { | ||
metadata { | ||
name = "loki-pv" | ||
} | ||
spec { | ||
access_modes = ["ReadWriteOnce"] | ||
capacity = { | ||
storage = "4Gi" | ||
} | ||
storage_class_name = "default" | ||
persistent_volume_reclaim_policy = "Retain" | ||
persistent_volume_source { | ||
# ... | ||
} | ||
} | ||
} | ||
|
||
# PV Local Loki Logging | ||
module "pv_local_loki_stack" { | ||
source = "terraform-iaac/loki-stack/kubernetes" | ||
|
||
provider_type = "local" | ||
pvc_storage_class_name = "default" | ||
pvc_access_modes = ["ReadWriteOnce"] | ||
persistent_volume_name = kubernetes_persistent_volume.pv_loki.metadata.0.name | ||
persistent_volume_size = "4Gi" | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.