Merge branch 'streamlit' into develop

.devcontainer/devcontainer.json

@@ -0,0 +1,33 @@
+{
+  "name": "Python 3",
+  // Or use a Dockerfile or Docker Compose file. More info: https://containers.dev/guide/dockerfile
+  "image": "mcr.microsoft.com/devcontainers/python:1-3.11-bullseye",
+  "customizations": {
+    "codespaces": {
+      "openFiles": [
+        "README.md",
+        "streamlit_app.py"
+      ]
+    },
+    "vscode": {
+      "settings": {},
+      "extensions": [
+        "ms-python.python",
+        "ms-python.vscode-pylance"
+      ]
+    }
+  },
+  "updateContentCommand": "[ -f packages.txt ] && sudo apt update && sudo apt upgrade -y && sudo xargs apt install -y <packages.txt; [ -f requirements.txt ] && pip3 install --user -r requirements.txt; pip3 install --user streamlit; echo '✅ Packages installed and Requirements met'",
+  "postAttachCommand": {
+    "server": "streamlit run streamlit_app.py --server.enableCORS false --server.enableXsrfProtection false"
+  },
+  "portsAttributes": {
+    "8501": {
+      "label": "Application",
+      "onAutoForward": "openPreview"
+    }
+  },
+  "forwardPorts": [
+    8501
+  ]
+}

.github/workflows/ci.yml

@@ -2,7 +2,7 @@ name: ci
 on:
   push:
     branches:
-      - 'develop'
+      - '*'
 permissions:
   contents: write
 jobs:
@@ -38,7 +38,7 @@ jobs:
           python-version: 3.x
       - name: Install Dependencies
         run: |
-          pip install mypy requests types-requests pandas-stubs types-psycopg2
+          pip install mypy requests types-requests pandas-stubs types-psycopg2 types-Flask
       - name: mypy
         run: |
           mypy src/
@@ -59,18 +59,18 @@ jobs:
         run: |
           pylint src/
 
-  pytest:
-    runs-on: ubuntu-latest
-    name: pytest
-    steps:
-      - uses: actions/checkout@v2
-      - name: Set up Python 3.x
-        uses: actions/setup-python@v2
-        with:
-          python-version: 3.x
-      - name: Install Dependencies
-        run: |
-          pip install pytest requests pandas yfinance
-      - name: pytest
-        run: |
-          pytest tests/
+  # pytest:
+  #   runs-on: ubuntu-latest
+  #   name: pytest
+  #   steps:
+  #     - uses: actions/checkout@v2
+  #     - name: Set up Python 3.x
+  #       uses: actions/setup-python@v2
+  #       with:
+  #         python-version: 3.x
+  #     - name: Install Dependencies
+  #       run: |
+  #         pip install pytest requests pandas yfinance psycopg2-binary
+  #     - name: pytest
+  #       run: |
+  #         pytest tests/

.gitignore

@@ -14,6 +14,7 @@
 
 # Personal files
 parameters.py
+*.dump
 
 # Python runtime
 *.pyc

README.md

@@ -2,14 +2,15 @@
 
 StockTracker is a Python application that help analyse and visualise stock performance data for a list of companies. It provides a user-friendly GUI for comparing stock prices, viewing historical trends, and staying updated with the latest news.
 
+Please head to [StockTracker](https://wleong1.github.io/StockTracker/) for more detailed explanation.
+
 ## Table of Contents
 
 - [Project Description](#project-description)
 - [Technologies](#technologies)
 - [Installation](#installation)
-- [Usage](#usage)
 - [Screenshots](#screenshots)
-- [Tests](#tests)
+- [Tests (Deprecated)](#tests)
 - [Contributing](#contributing)
 - [License](#license)
 
@@ -25,49 +26,32 @@ StockTracker is designed to simplify stock analysis and decision-making. It fetc
 ## Technologies
 
 - Python 3.10
-- PyQt5
-- Bokeh
-- Matplotlib
-- Numpy
+- Flask
+- Plotly
 - Pandas
-- Unittest
+- psycopg2-binary
+- Streamlit
+- yfinance
+- PostgreSQL
 
 ## **Installation**
 
-1. Clone the project repository using Git.
-2. Ensure that your pip package manager is up to date by running the command 
-```pip install --upgrade pip```.
-3. Install the required modules by executing the command 
-```pip install -r ./requirements.txt```
-
-## **Usage**
-
-1. Download the stock data from the following source: https://www.kaggle.com/datasets/camnugent/sandp500.
-2. Extract the downloaded file and navigate to the extracted folder.
-3. Locate the directory named _individual_stocks_5yr_ within the first _individual_stocks_5yr_ directory and copy it.
-4. Paste the _individual_stocks_5yr_ directory into the __StockTracker__ repository.
-5. Within the _src_ directory of the project, create a new Python file named _parameter.py_.
-6. Inside _parameter.py_, create two variables: __NEWS_API_KEY__ and __ALPHA_VANTAGE_API_KEY__.
-7. Assign your __News API__ and __Alpha Vantage API__ keys to the respective variables in _parameter.py_.
-8. Run the _main.py_ file to start the __StockTracker__ application.
-
-Note: 
-- This project relies on external data sources and requires valid API keys to access the necessary information. Make sure to provide your own API keys to ensure proper functionality.
-- The demonstration of the GUI functionality utilises older data. To ensure you have the most recent graph representation, replace the provided data with updated information.
+Please refer to the [how-to guide](https://wleong1.github.io/StockTracker/how-to-guides/) for detailed installation instructions.
 
 ## **Screenshots**
 
-![image](https://github.com/wleong1/StockTracker/assets/122815453/bd739475-5fe7-4342-99ef-a28674e004f9)
+![image](./docs/assets/user-interface-diagram.png)
 
 The image showcases the GUI, featuring elements such as the company selection dropdown menu, latest closing price, current news, and historical performance.
 
-## **Tests**
+## **Tests (Deprecated)**
 
 StockTracker incorporates various tests to ensure its stability and reliability. To execute these tests, follow these steps:
 
 1. Go to the tests directory.
-2. Replace the sample_data files with a subset of files you intend to test, maintaining a similar naming convention.
-3. Adjust the test scripts as necessary.
+2. Run the tests.
+
+NOTE: This section is deprecated due to the migration from csv files to PostgreSQL database.
 
 ## **Contributing**
 

db_dir/Dockerfile

@@ -0,0 +1,9 @@
+FROM postgres
+
+# Install dependencies
+RUN apt-get update
+RUN apt-get install -y nfs-common
+RUN apt-get install rpcbind
+
+# Copy the shell script into the image
+COPY db_init.sh /docker-entrypoint-initdb.d/init-user-db.sh

db_dir/db_init.sh

@@ -0,0 +1,7 @@
+#!/bin/bash
+
+# Creates an empty stocks database
+psql -U postgres -c "CREATE DATABASE stocks;"
+
+# Restore the data into the stocks database
+pg_restore -U postgres -d stocks /nfs/db.dump

docker-compose.yml

@@ -0,0 +1,64 @@
+version: '3'
+
+# Microservices
+services:
+  # Postgresql database container
+  db:
+    build: ./db_dir
+    container_name: stocks-postgres
+    privileged: true
+    ports:
+      - 5433:5432
+    environment:
+      - POSTGRES_USER=postgres
+      - POSTGRES_PASSWORD=123456
+    volumes:
+      - nfs-volume:/nfs
+    networks: 
+      - stocks-network
+    healthcheck:
+      test: ["CMD-SHELL", "pg_isready -h stocks-postgres -U postgres -p 5432 | grep 'accepting connections'"]
+      interval: 5s
+      timeout: 5s
+      retries: 5
+
+  # Core container
+  core:
+    build: ./src
+    container_name: core-modules
+    ports:
+      - 5000:5000
+    volumes:
+      - ./src:/usr/workspace
+    networks:
+      - stocks-network
+    depends_on:
+      db:
+        condition: service_healthy
+
+  # Streamlit app container
+  web:
+    build: ./streamlit
+    container_name: streamlit-container
+    ports: 
+      - 8501:8501
+    volumes:
+      - ./streamlit:/app
+    networks:
+      - stocks-network
+    depends_on:
+        - core
+
+# Volume for NFS
+volumes:
+  nfs-volume:
+    driver: local
+    driver_opts:
+      type: "nfs"
+      o: "addr=${HOST_IP},nolock,soft,rw,nfsvers=4"
+      device: ":${PWD}/network_directory"
+
+# Shared network for microservices
+networks:
+  stocks-network:
+    name: stocks-network

docs/explanation.md

@@ -1,15 +1,57 @@
-This part of the project documentation focuses on an
-**understanding-oriented** approach. You'll get a
-chance to read about the background of the project,
-as well as reasoning about how it was implemented.
-
-> **Note:** Expand this section by considering the
-> following points:
-
-- Give context and background on your library
-- Explain why you created it
-- Provide multiple examples and approaches of how
-    to work with it
-- Help the reader make connections
-- Avoid writing instructions or technical descriptions
-    here
+# StockTracker
+## Compatibility
+
+This project has been tested on:
+
+- WSL 2 on Windows running Ubuntu 22.04
+- Ubuntu 22.04
+
+## Motivation
+
+The motivation behind this project is to improve the developer's financial literacy and learn software engineering best practices.
+
+## Project Structure
+
+The main components of this project are:
+
+- *Web app*: Built using Python Streamlit
+- *Business logic*: Implemented using Python and Flask
+- *Database*: PostgreSQL
+
+## Technologies Used
+
+- Python 3.10
+- Flask
+- Plotly
+- Pandas
+- psycopg2-binary
+- Streamlit
+- yfinance
+- PostgreSQL
+
+## User Interface
+
+### User Interface
+![alt text](./assets/user-interface-diagram.png)
+
+The image above illustrates what a user would see if the project is set up successfully.
+
+## Architecture
+
+### Architecture Diagram
+
+![alt text](./assets/architecture-diagram.png)
+
+The idea behind the project is to split each microservice into respective containers:
+
+- Web app container
+- Business logic container
+- Database container
+
+An ***NFS server*** is set up to allow sharing of PostgreSQL backup files (*.dump*) with the PostgreSQL ***database container***, which acts as the *NFS client*. Any user interactions with the ***web app container*** will send requests via *RESTful API* to the ***business logic container***, which in turn sends the requests to the ***database container*** via the *database connection*.
+
+## Demo
+
+A version of the web app that contains part of the database has been deployed. Feel free to give that a go.
+
+[Demo link](https://wleong1-stocktracker.streamlit.app/)