Skip to content

Releases: rlworkgroup/garage

2018.10

31 Oct 18:32
@ryanjulian ryanjulian
v2018.10.0
564e4c2
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
2018.10

The Reinforcement Learning Working Group is proud to announce the 2018.10 release of garage.

We are actively seeking new contributors. If you use garage, please consider submitting a PR with your algorithm or improvements to the framework.

Summary

This release's life began as a maintenance fork of rllab. The original authors of rllab, current maintainers, and heavy users conferred about the future of the project. We reached a consensus to continue development of rllab under the new name "garage," and to organize future development within a GitHub organization which is detached from any particular institution. We named this organization the Reinforcement Learning Working Group.

Most changes in this release concern stability, dependency updates, platform support, testing, and maintainability. We added many pieces of automation which are invisible to everyday users, but greatly assist in speeding garage development and keeping the framework stable. We have made many attempts to remove code which we did not think we could support in the future, though some unstable parts (e.g. viskit, EC2 support in garage.misc.instrument) remain and should be treated with caution. We welcome PRs for features which need updates and improvements.

We finished building out the TensorFlow tree, added a few algorithms in TensorFlow (e.g. PPO, TRPO, DDPG with HER), and promoted it out of sandbox into the main tree as garage.tf.

Likewise, we moved all Theano-specific modules into their own subtree (garage.theano) to separate the framework-agnostic and framework-specific parts of garage clearly.

New features include TensorBoard support in the logger, support for dm_control environments, and a general implementation of dynamics randomization for MuJoCo-based environments.

Users migrating their projects from rllab should consult the migration instructions below.

Please see the CHANGELOG for detailed information on the changes in this release.

Who should use this release, and how

Users who want to base a project on a semi-stable version of this software, and are not interested in bleeding-edge features (e.g. PyTorch support) should use the release branch and tags. We also recommend existing rllab users migrate their code to this release ASAP.

Platform support

This release has been tested extensively on Ubuntu 16.04. We have also used it successfully on Ubuntu 18.04 and on macOS 10.12, 10.13, and 10.14.

Maintenance Plan

We plan on supporting this branch until at least June 2019. Our support will come mostly in the form of attempting to reproduce and fix critical user-reported bugs, conducting quality control on user-contributed PRs to the release branch, and releasing new versions when fixes are committed.

We haven no intention of performing proactive maintenance such as dependency upgrades, nor new features, tests, platform support, or documentation. However, we welcome PRs to the maintenance branch (release-2018.10) from contributors wishing see these enhancements to this version of the software.

Hotfixes

We will post backwards-compatible hotfixes for this release to the branch release-2018.10. New hotfixes will also trigger a new release tag which complies with semantic versioning, i.e. the first hotfix release would be tagged v2018.10.1, the second would be tagged v2018.10.2, etc.

We will not add new features, nor remove existing features from the branch release-2018.10 unless it is absolutely necessary for the integrity of the software.

Next release

v2018.10 marks the first in what will hopefully be a long line of regular releases. We hope to release 2-3 times per year, approximately aligned with the North American academic calendar. We hope to release next around early February 2019, e.g. v2019.02.

See Looking forward for more information on what to expect in the next release.

Migrating from rllab

garage is based on a predecessor project called rllab. Migrating from rllab to garage should be mostly painless, but not completely automatic. Some classes and functions from rllab have been renamed, moved, or had their signatures changed. Very few have been removed. Follow the process below to migrate.

  1. Install the garage conda environment
    Execute the installation script for Linux or macOS. This will create a separate conda environment named "garage" will, so there won't be any conflicts with a previous installation of a "rllab" environment. However, be aware that both scripts try to install miniconda, so there could be conflicts if you already have a different conda installation. If you're not using conda for other purpose, the best option is to remove it as indicated here. For a more granular installation of garage, read the installation scripts and only execute those commands that are required for your system.

  2. Rebase or retarget your repository on garage
    This step will be very specific to your project. Essentially, get garage into your PYTHONPATH, e.g. by moving your rllab sandbox into garage/sandbox, or by editing your environment configuration.

  3. Replace rllab package imports with garage package imports
    e.g. from rllab.core import Serializable becomes from garage.core import Serializable. Note that some import paths have changed. Please check the CHANGELOG for hints at where to look for changes which affect your project

  4. Run your launcher files and tests, and make sure everything is working as-expected. Fix errors as you find them.

Looking forward

The next release of garage will focus primarily on two goals: PyTorch support and splitting garage into federated packages.

Our goal is to make the software foundation for reproducible reinforcement learning research. That requires good maintenance, stability, and widespread adoption. We believe breaking garage up is necessary to ensure the future maintainability of the project, and to speed adoption.

The next release will likely bring many breaking changes to garage, along with a new federated project structure which splits what is currently called garage into a family of several Python packages with independent repositories, development infrastructure, dependency management, and documentation. The federated packages will be pip dependencies which may be downloaded and used in any project.

Today, using garage is an all-or-nothing choice for a prospective user. He or she may either buy into the entire ecosystem — algorithms, experiment runner, conda environment, custom environments, plotter, etc. — or use none of it at all.

Our goal for the next release is to take the first steps towards unbundling garage into a family easy-to-adopt Python packages with well-designed interfaces which allow them to easily work together, or be used separately. Stay tuned to this GitHub repository for details and proposals, and to give your own input.

Here's an example of how a federated garage might be split into packages:

Experiment runner

  • Experiment runner framework and algorithm interface (parallel samplers, logging, live plotting, deployment support for local, EC2, GCP targets, etc.). Contains abstractions for defining experiments, running them, monitoring them, collecting results, and visualizing results.

    Likely based on lagom and/or ray

Algorithm libraries

  • Library of numpy-based RL algorithms, math utilities for RL, and useful algorithm base classes

  • Library of TensorFlow-based algorithms

  • Library of PyTorch-based RL algorithms

    Note: we plan on removing Theano support by the next release.

Environments

  • gym.Env wrappers and dependency management for popular environments (e.g. dm_control, ALE, pybullet)
  • Custom single-task environments
  • Custom multi-task/meta-learning environments and supporting wrappers
  • Custom robotics environments, and bindings to ROS for real robot execution

Utilities

  • A small library of Python types for RL (e.g. garage.spaces)
  • Experiment results visualization toolkit (e.g. viskit)

Distribution Repository (garage)
garage becomes an application repository which pulls all of the above together into a single environment, and demonstrates how to use them. New projects can use garage as a template or as an upstream.

Contributors to this release

Assets 2
Loading