PyPI v0.15.0

Features

• Add daemon Services Mode (daemon --services-mode) where the daemon spins a task in its own docker and verifies start-up and shut-down. This allows multiple tasks to be launched simultaneously on the same machine (currently in CPU mode only), where each task service will register itself as a worker for the lifetime of the task
• Enhance build --docker mode
  • Add --install-globally option to install required packages in the docker's system python
  • Add --entry-point option to allow automatic task cloning when running the docker
• Support PyTorch Nightly builds using the agent.torch_nightly configuration flag. If true, the agent looks for a nightly build when a stable torch wheel is not found
• Add environment variables support for git user/password
  • Using TRAINS_AGENT_GIT_USER/TRAINS_AGENT_GIT_PASS
  • Pass git credentials to dockerized experiment execution
• Support running code from module (i.e. -m in execution entry point)
• Add daemon --create-queue to automatically create a queue and use it if queue name doesn't exist in the server
• Move --gpus and --cpu-only to worker args (used by daemon, execute and build)

Bug Fixes

• Fix init wizard, correctly display the input servers #19
• Fix version control links in requirements when using conda
• Fix build --docker mode standalone docker execution
• Improve docker host-mount support, use TRAINS_AGENT_DOCKER_HOST_MOUNT environment variable
• Support pip v20.1 local/http package reference in pip freeze
• Fix detached mode to correctly use cache folder slots
• Fix CUDA_VISIBLE_DEVICES should never be set to "all" (Trains Slack channel thread)
• Do not monitor GPU when running with --cpu-only

PyPI v0.14.1

Features and Bug Fixes

• Add daemon detached mode (--detached, -d) that runs the agent as daemon in the background and returns immediately
• Auto mount ~/.git-credentials into docker container (if file exists)
• Add TRAINS_AGENT_EXTRA_PYTHON_PATH environment variable to allow adding additional python path during experiment execution (helpful when using extra un-tracked modules)
• Fix "run as user" feature (using TRAINS_AGENT_EXEC_USER environment variable)
• Fix PyTorch support to ignore minor versions when looking for package to install/download
• Fix experiment execution output handling

PyPI v0.14.0

Features and Bug Fixes

• Add support for trains-agent execute --id <experiment-id> --docker that allows executing a specific experiment inside a docker container
• Add support for trains-agent execute --id <template-experiment-id> --clone that clones the provided experiment and executes the cloned experiment
• Add support for APIClient.models.delete() to allow programmatically deleting a model clearml/clearml-server#32
• Add daemon support for passing storage-related OS environment variables to experiments executed inside a docker container (supported by trains>=0.13.3):
  • AWS: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_DEFAULT_REGION
  • Azure: AZURE_STORAGE_ACCOUNT and AZURE_STORAGE_KEY
  • Google: GOOGLE_APPLICATION_CREDENTIALS
• Fix git checkout with submodules clearml/clearml#112
• Prefer docker image from command line over the one specified in experiment

PyPI v0.13.3

Features and Bug Fixes

• Allow providing queue names instead of queue IDs in daemon mode
• Docker mode improvements
  • Support running as a specific user inside a docker using the TRAINS_AGENT_EXEC_USER environment flag
  • Pass correct GPU limit when skipping gpus flag
  • Add --force-current-version daemon command-line flag
• Add K8s/trains glue service example
• Added K8s support in daemon mode
  • Running inside a K8s pod
  • Mounting dockerized experiment folders to host
  • Allow a specific network for the docker
• Add default storage environment vars (for AWS, GS and Azure) to generated agent configuration
• Improve Unicode/UTF stdout handling

PyPI v0.13.2

Features and Bug Fixes

• Pre-install numpy if it exists in the requirements
• Add experiment archiving example
• Add .bashrc reloading before running trains-agent in the AWS dynamic cluster management service
• Add support for pulling recursive git modules as as well as main project
• Limit virtualenv version to <20 due to an import issue in v20.0.0
• Fix pip install/upgrade with limit in conda
• Fix daemon monitor to not stop experiments if network is down

PyPI v0.13.1

Features

• Add poetry support
• Add agent.docker_force_pull config option to force docker pull before running an experiment using a docker image
• Add agent.extra_docker_arguments and agent.extra_docker_shell_script config options for extra docker parameters and shell script which will always be used when running an experiment using a docker image
• Add agent.package_manager.pip_version config option to limit pip version used in virtual env. By default, this value is set to <20 due to a pip version instability
• Add support for pip "editable" packages syntax in experiment requirements (e.g. -e git://github.com/my-repo, see here)
• Add support for git repositories without ".git" suffix (e.g. Azure Repos)
• Improve conda support
• Fix cases where virtualenv was installed but unavailable as a shell command
• Fix logging monitor to make sure logs are sent even in case an exception occurs inside the monitor itself

PyPI v0.13.0

Features

• Add support for docker pre-installed pytorch versions that do not exist on PyPI/PyTorch.org
• Add AWS dynamic cluster management service
• Add support for various event query endpoints in APIClient
• Improve the configuration wizard

PyPI v0.12.2

Features and Bug Fixes

• Improve configuration wizard
• Improve docker and k8s support
• Add docker build command
• Add initial Poetry support
• Fix docker CUDA support

PyPI v0.12.1

Features and Bug Fixes

• Windows support! YES, you can now have Windows machines as part of your cluster (notice --docker is not supported on Windows)
• Add initial Conda package manager support (still in beta)
• Add --gpus and --cpu-only for easier GPU control when running multiple trains-agent instances on the same machine
• Add python_binary can now be specified (the default is the same python binary executing the trains-agent)
• Fix Issue #2

PyPI v0.12.0

TRAINS Agent - Initial Release

• TRAINS Agent currently supports Linux & Mac
• Python packaging system supported: pip
• Containers supported: Docker v19.03 and above
• Version Control supporedt: git
• Support for standalone scripts & Jupyter Notebooks
• TRAINS-server v0.12 and above
• TRAINS v0.11.3 and above
• Sample configuration file available here