Skip to content

Latest commit

 

History

History
122 lines (88 loc) · 3.12 KB

README.md

File metadata and controls

122 lines (88 loc) · 3.12 KB

🤖 AILIVE - Local Training with Stable-Baselines3

This project trains a reinforcement learning agent using Proximal Policy Optimization (PPO) from the Stable-Baselines3 library. The project includes automatic model saving, uploading models to a remote server, and resuming training from previously saved models.

Features

  • Uses PPO algorithm from Stable-Baselines3
  • Supports TensorBoard logging
  • Automatically saves models at regular intervals
  • Uploads saved models to a remote server via pre-signed URLs
  • Resumes training from the latest saved model

Prerequisites

  • Python 3.7 or later
  • pip for package management

Installation

  1. Clone the repository:

    git clone https://github.com/ailiveco/local-trainings.git
cd local-trainings

  2. Create a virtual environment (recommended):

    python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

  3. Install the required packages:

    pip install -r requirements.txt

Configuration

Update the following constants in main.py as needed:

  • API Key: Replace AILIVE_SECRET_APIKEY with your valid API key.

    AILIVE_SECRET_APIKEY = "EXAMPLEKEY-zero-walking"

  • Save Interval: Adjust the frequency of model saving.

    SAVE_INTERVAL = 500_000  # Save every 500,000 steps

  • Total Training Timesteps: Set the total number of timesteps for training.

    TOTAL_TIMESTEPS = 10_000_000  # Train for 10 million steps

Usage

  1. Run the main script to start training:

    python main.py

  2. TensorBoard logs are saved in the specified directory. To visualize logs:

    tensorboard --logdir=./sessions/<agent_name>/<skill_name>/tensorboard

  3. Models are saved in the models/ directory inside the session folder.

Project Structure

.
├── main.py                  # Main script for training the agent
├── requirements.txt         # List of dependencies
├── README.md                # Project documentation
└── sessions/                # Directory for logs and saved models
    └── <agent_name>/        # Agent-specific folder
        └── <skill_name>/    # Skill-specific folder
            ├── tensorboard/ # TensorBoard logs
            └── models/      # Saved models

Key Functions

get_presigned_url()

Fetches a pre-signed URL for uploading models to a remote server.

upload_model(file_path)

Uploads a model file to the server using the pre-signed URL.

save_model(model, step_count)

Saves the model locally and uploads it to the server.

load_latest_model(model)

Loads the latest saved model for resuming training.

main()

Sets up the environment, initializes the PPO model, and manages the training loop.

Notes

  • Ensure your API key is valid and matches the expected format.
  • TensorBoard must be installed to use logging features.
  • The Humanoid-v5 environment is used as an example; you can replace it with any supported environment.

License

This project is licensed under the MIT License.