lab done

lab7.md

@@ -0,0 +1,59 @@
+# Lab 7: Monitoring and Logging
+
+## Overview
+
+In this lab, you will become familiar with a logging stack that includes Promtail, Loki, and Grafana. Your goal is to create a Docker Compose configuration and configuration files to set up this logging stack.
+
+## Task 1: Logging Stack Setup
+
+**6 Points:**
+
+1. Study the Logging Stack:
+   - Begin by researching the components of the logging stack:
+     - [Grafana Webinar: Loki Getting Started](https://grafana.com/go/webinar/loki-getting-started/)
+     - [Loki Overview](https://grafana.com/docs/loki/latest/overview/)
+     - [Loki GitHub Repository](https://github.com/grafana/loki)
+
+2. Create a Monitoring Folder:
+   - Start by creating a new folder named `monitoring` in your project directory.
+
+3. Docker Compose Configuration:
+   - Inside the `monitoring` folder, prepare a `docker-compose.yml` file that defines the entire logging stack along with your application.
+   - To assist you in this task, refer to these resources for sample Docker Compose configurations:
+     - [Example Docker Compose Configuration from Loki Repository](https://github.com/grafana/loki/blob/main/production/docker-compose.yaml)
+     - [Promtail Configuration Example](https://github.com/black-rosary/loki-nginx/blob/master/promtail/promtail.yml) (Adapt it as needed)
+
+4. Testing:
+   - Verify that the configured logging stack and your application work as expected.
+
+## Task 2: Documentation and Reporting
+
+**6 Points:**
+
+1. Logging Stack Report:
+   - Create a new file named `LOGGING.md` to document how the logging stack you've set up functions.
+   - Provide detailed explanations of each component's role within the stack.
+
+2. Screenshots:
+   - Capture screenshots that demonstrate the successful operation of your logging stack.
+   - Include these screenshots in your `LOGGING.md` report for reference.
+
+## Bonus Task: Additional Configuration
+
+**2.5 Points:**
+
+1. Integrating Your Extra App:
+   - Extend the `docker-compose.yml` configuration to include your additional application.
+
+2. Configure Stack for Comprehensive Logging:
+   - Modify the logging stack's configuration to collect logs from all containers defined in the `docker-compose.yml`.
+   - Include screenshots in your `LOGGING.md` report to demonstrate your success.
+
+### Guidelines
+
+- Ensure that your documentation in `LOGGING.md` is well-structured and comprehensible.
+- Follow proper naming conventions for files and folders.
+- Use code blocks and Markdown formatting where appropriate.
+- Create pull requests (PRs) as needed: from your fork to the main branch of this repository, and from your fork's branch to your fork's master branch.
+
+> Note: Thoroughly document your work, and ensure the logging stack functions correctly. Utilize the bonus points opportunity to enhance your understanding and the completeness of your setup.

monitoring/LOGGING.md

@@ -0,0 +1,36 @@
+# Logging Stack Report
+
+In this document, I will provide a detailed overview of the logging stack that has been set up, including its components and their respective roles.
+
+## Components
+
+### 1. Grafana
+
+**Role**: Grafana is an open-source analytics and monitoring platform that allows you to query, visualize, alert on, and understand your metrics no matter where they are stored. In the logging stack, Grafana serves as the dashboarding tool where logs and metrics can be visualized.
+
+### 2. Loki
+
+**Role**: Loki is a horizontally scalable, highly available, multi-tenant log aggregation system inspired by Prometheus. It is designed to be very cost-effective and easy to operate. Loki allows you to query logs using the same labels that you're already using with Prometheus, enabling easy integration with Grafana for visualization.
+
+### 3. Promtail
+
+**Role**: Promtail is an agent that ships the contents of local logs to a private Loki instance or Grafana Cloud. It is usually deployed to every machine that has applications needed to be logged. Promtail is responsible for tailing log files, extracting and parsing log lines, and sending them to Loki.
+
+## Functioning
+
+The logging stack functions as follows:
+
+1. **Promtail Configuration**: Promtail is configured to tail log files from specific directories on each host. It extracts labels from log entries and sends them to Loki for storage and indexing.
+
+2. **Loki Configuration**: Loki receives log entries from Promtail and stores them. It indexes the logs based on the labels extracted by Promtail, making it efficient to query logs based on various dimensions.
+
+3. **Grafana Configuration**: Grafana is configured to query data from Loki and visualize log data using dashboards. It allows users to create custom dashboards and panels to monitor log data in real-time.
+
+## Conclusion
+
+The logging stack consisting of Grafana, Loki, and Promtail provides a robust solution for log aggregation, storage, and visualization. Grafana serves as the frontend dashboarding tool, while Loki stores logs efficiently and allows for querying based on labels. Promtail ensures that log data is collected from various sources and sent to Loki for storage. This setup enables effective monitoring and troubleshooting of applications and systems.
+
+## Screenshots
+
+![Grafana Dashboard](grafana1.png)
+![Grafana Dashboard](grafana2.png)

monitoring/docker-compose.yml

@@ -0,0 +1,37 @@
+version: '3'
+
+services:
+  loki:
+    image: grafana/loki:latest
+    ports:
+      - "3100:3100"
+    command: -config.file=/etc/loki/local-config.yaml
+    volumes:
+      - ./loki-config:/etc/loki
+
+  promtail:
+    image: grafana/promtail:latest
+    volumes:
+      - /var/log:/var/log
+      - ./promtail-config:/etc/promtail
+    command: -config.file=/etc/promtail/promtail-config.yaml
+
+  grafana:
+    image: grafana/grafana:latest
+    ports:
+      - "3000:3000"
+    depends_on:
+      - loki
+    environment:
+      - GF_AUTH_ANONYMOUS_ENABLED=true
+      - GF_AUTH_ANONYMOUS_ORG_NAME=MainOrg
+
+  app_python:
+    build: ../app_python
+    ports:
+      - "5001:5000"
+    volumes:
+      - ../app_python:/app_python
+    depends_on:
+      - loki
+      - promtail

monitoring/loki-config/local-config.yaml

@@ -0,0 +1,25 @@
+auth_enabled: false
+
+server:
+  http_listen_port: 3100
+
+ingester:
+  lifecycler:
+    address: 127.0.0.1
+    ring:
+      kvstore:
+        store: inmemory
+    final_sleep: 0s
+
+  chunk_idle_period: 1h
+  chunk_retain_period: 30s
+
+schema_config:
+  configs:
+    - from: 2021-08-01
+      store: boltdb-shipper
+      object_store: filesystem
+      schema: v11
+      index:
+        prefix: index_
+        period: 168h

monitoring/promtail-config/promtail-config.yaml

@@ -0,0 +1,18 @@
+server:
+  http_listen_port: 9080
+  grpc_listen_port: 0
+
+positions:
+  filename: /tmp/positions.yaml
+
+clients:
+  - url: http://loki:3100/loki/api/v1/push
+
+scrape_configs:
+  - job_name: system
+    static_configs:
+      - targets:
+          - localhost
+        labels:
+          job: varlogs
+          __path__: /var/log/*log