diff --git a/.github/workflows/python-publish.yml b/.github/workflows/python-publish.yml new file mode 100644 index 000000000..3bfabfc12 --- /dev/null +++ b/.github/workflows/python-publish.yml @@ -0,0 +1,36 @@ +# This workflow will upload a Python Package using Twine when a release is created +# For more information see: https://help.github.com/en/actions/language-and-framework-guides/using-python-with-github-actions#publishing-to-package-registries + +# This workflow uses actions that are not certified by GitHub. +# They are provided by a third-party and are governed by +# separate terms of service, privacy policy, and support +# documentation. + +name: Upload Python Package + +on: + release: + types: [published] + +jobs: + deploy: + + runs-on: ubuntu-latest + + steps: + - uses: actions/checkout@v2 + - name: Set up Python + uses: actions/setup-python@v2 + with: + python-version: '3.x' + - name: Install dependencies + run: | + python -m pip install --upgrade pip + pip install build + - name: Build package + run: python -m build + - name: Publish package + uses: pypa/gh-action-pypi-publish@27b31702a0e7fc50959f5ad993c78deac1bdfc29 + with: + user: __token__ + password: ${{ secrets.PYPI_API_TOKEN }} diff --git a/.wci.yml b/.wci.yml new file mode 100644 index 000000000..0ced2c93b --- /dev/null +++ b/.wci.yml @@ -0,0 +1,12 @@ +name: Merlin + +icon: https://raw.githubusercontent.com/LLNL/merlin/main/docs/images/merlin_icon.png + +headline: Enabling Machine Learning HPC Workflows + +description: The Merlin workflow framework targets large-scale scientific machine learning (ML) workflows in High Performance Computing (HPC) environments. Merlin is a producer-consumer workflow model that enables multi-machine, cross-batch job, dynamically allocated yet persistent workflows capable of utilizing surge-compute resources. Key features are a flexible and intuitive HPC-centric interface, low per-task overhead, multi-tiered fault recovery, and a hierarchical sampling algorithm that allows for highly scalable task execution and queuing to ensembles of millions of tasks. + +documentation: + general: https://merlin.readthedocs.io/ + installation: https://merlin.readthedocs.io/en/latest/modules/installation/installation.html + tutorial: https://merlin.readthedocs.io/en/latest/tutorial.html diff --git a/CHANGELOG.md b/CHANGELOG.md index 010dbd254..b44333f52 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,7 +4,13 @@ All notable changes to Merlin will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). -## [Unreleased] +## [1.8.4] +### Added +- Auto-release of pypi packages +- Workflows Community Initiative metadata file + +### Fixed +- Old references to stale branches ## [1.8.3] ### Added diff --git a/LICENSE b/LICENSE index 494069b61..746aac58b 100644 --- a/LICENSE +++ b/LICENSE @@ -1,6 +1,6 @@ MIT License -Copyright (c) 2019 Lawrence Livermore National Laboratory +Copyright (c) 2022 Lawrence Livermore National Laboratory Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal diff --git a/Makefile b/Makefile index 2f5086ce9..d1e441a87 100644 --- a/Makefile +++ b/Makefile @@ -1,12 +1,12 @@ ############################################################################### -# Copyright (c) 2019, Lawrence Livermore National Security, LLC. +# Copyright (c) 2022, Lawrence Livermore National Security, LLC. # Produced at the Lawrence Livermore National Laboratory # Written by the Merlin dev team, listed in the CONTRIBUTORS file. # # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.8.3. +# This file is part of Merlin, Version: 1.8.4. # # For details, see https://github.com/LLNL/merlin. # @@ -191,7 +191,7 @@ reqlist: release: - $(PYTHON) setup.py sdist bdist_wheel + $(PYTHON) -m build . clean-release: diff --git a/README.md b/README.md index dbbd57707..fad1de93f 100644 --- a/README.md +++ b/README.md @@ -4,7 +4,7 @@ [![Issues](https://img.shields.io/github/issues/LLNL/merlin)](https://github.com/LLNL/merlin/issues) [![Pull requests](https://img.shields.io/github/issues-pr/LLNL/merlin)](https://github.com/LLNL/merlin/pulls) -![Merlin](https://raw.githubusercontent.com/LLNL/merlin/master/docs/images/merlin.png) +![Merlin](https://raw.githubusercontent.com/LLNL/merlin/main/docs/images/merlin.png) ## A brief introduction to Merlin Merlin is a tool for running machine learning based workflows. The goal of @@ -133,6 +133,6 @@ the Merlin community, you agree to abide by its rules. ## License -Merlin is distributed under the terms of the [MIT LICENSE](https://github.com/LLNL/merlin/blob/master/LICENSE). +Merlin is distributed under the terms of the [MIT LICENSE](https://github.com/LLNL/merlin/blob/main/LICENSE). LLNL-CODE-797170 diff --git a/docs/images/merlin_icon.png b/docs/images/merlin_icon.png new file mode 100644 index 000000000..664ed4655 Binary files /dev/null and b/docs/images/merlin_icon.png differ diff --git a/docs/source/modules/contribute.rst b/docs/source/modules/contribute.rst index 3180bc13e..8e8cde7af 100644 --- a/docs/source/modules/contribute.rst +++ b/docs/source/modules/contribute.rst @@ -41,6 +41,6 @@ Merlin is an open source project, so contributions are welcome. Contributions ca Contributing to Merlin is easy! Just `send us a pull request `_ from your fork. Before you send it, summarize your change in the ``[Unreleased]`` section of ``CHANGELOG.md`` and make sure develop is the destination branch. We also appreciate `squash commits `_ before pull requests are merged. -Merlin uses a rough approximation of the Git Flow branching model. The develop branch contains the latest contributions, and master is always tagged and points to the latest stable release. +Merlin uses a rough approximation of the Git Flow branching model. The develop branch contains the latest contributions, and main is always tagged and points to the latest stable release. If you're a contributor, try to test and run on develop. That's where all the magic is happening (and where we hope bugs stop). diff --git a/merlin/__init__.py b/merlin/__init__.py index 75513329f..b99002f88 100644 --- a/merlin/__init__.py +++ b/merlin/__init__.py @@ -1,12 +1,12 @@ ############################################################################### -# Copyright (c) 2019, Lawrence Livermore National Security, LLC. +# Copyright (c) 2022, Lawrence Livermore National Security, LLC. # Produced at the Lawrence Livermore National Laboratory # Written by the Merlin dev team, listed in the CONTRIBUTORS file. # # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.8.3. +# This file is part of Merlin, Version: 1.8.4. # # For details, see https://github.com/LLNL/merlin. # @@ -38,7 +38,7 @@ import sys -__version__ = "1.8.3" +__version__ = "1.8.4" VERSION = __version__ PATH_TO_PROJ = os.path.join(os.path.dirname(__file__), "") diff --git a/merlin/ascii_art.py b/merlin/ascii_art.py index 8acd698b4..2b3366693 100644 --- a/merlin/ascii_art.py +++ b/merlin/ascii_art.py @@ -1,12 +1,12 @@ ############################################################################### -# Copyright (c) 2019, Lawrence Livermore National Security, LLC. +# Copyright (c) 2022, Lawrence Livermore National Security, LLC. # Produced at the Lawrence Livermore National Laboratory # Written by the Merlin dev team, listed in the CONTRIBUTORS file. # # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.8.3. +# This file is part of Merlin, Version: 1.8.4. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/celery.py b/merlin/celery.py index c2e98f3a3..a2e1de2bd 100644 --- a/merlin/celery.py +++ b/merlin/celery.py @@ -1,12 +1,12 @@ ############################################################################### -# Copyright (c) 2019, Lawrence Livermore National Security, LLC. +# Copyright (c) 2022, Lawrence Livermore National Security, LLC. # Produced at the Lawrence Livermore National Laboratory # Written by the Merlin dev team, listed in the CONTRIBUTORS file. # # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.8.3. +# This file is part of Merlin, Version: 1.8.4. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/common/__init__.py b/merlin/common/__init__.py index f78f9c91a..da4ad9f74 100644 --- a/merlin/common/__init__.py +++ b/merlin/common/__init__.py @@ -1,12 +1,12 @@ ############################################################################### -# Copyright (c) 2019, Lawrence Livermore National Security, LLC. +# Copyright (c) 2022, Lawrence Livermore National Security, LLC. # Produced at the Lawrence Livermore National Laboratory # Written by the Merlin dev team, listed in the CONTRIBUTORS file. # # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.8.3. +# This file is part of Merlin, Version: 1.8.4. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/common/abstracts/__init__.py b/merlin/common/abstracts/__init__.py index f78f9c91a..da4ad9f74 100644 --- a/merlin/common/abstracts/__init__.py +++ b/merlin/common/abstracts/__init__.py @@ -1,12 +1,12 @@ ############################################################################### -# Copyright (c) 2019, Lawrence Livermore National Security, LLC. +# Copyright (c) 2022, Lawrence Livermore National Security, LLC. # Produced at the Lawrence Livermore National Laboratory # Written by the Merlin dev team, listed in the CONTRIBUTORS file. # # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.8.3. +# This file is part of Merlin, Version: 1.8.4. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/common/abstracts/enums/__init__.py b/merlin/common/abstracts/enums/__init__.py index e16c45498..c6357ef4b 100644 --- a/merlin/common/abstracts/enums/__init__.py +++ b/merlin/common/abstracts/enums/__init__.py @@ -1,12 +1,12 @@ ############################################################################### -# Copyright (c) 2019, Lawrence Livermore National Security, LLC. +# Copyright (c) 2022, Lawrence Livermore National Security, LLC. # Produced at the Lawrence Livermore National Laboratory # Written by the Merlin dev team, listed in the CONTRIBUTORS file. # # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.8.3. +# This file is part of Merlin, Version: 1.8.4. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/common/openfilelist.py b/merlin/common/openfilelist.py index 4c87bb978..beb8a69aa 100644 --- a/merlin/common/openfilelist.py +++ b/merlin/common/openfilelist.py @@ -1,14 +1,14 @@ #!/usr/bin/env python ############################################################################### -# Copyright (c) 2019, Lawrence Livermore National Security, LLC. +# Copyright (c) 2022, Lawrence Livermore National Security, LLC. # Produced at the Lawrence Livermore National Laboratory # Written by the Merlin dev team, listed in the CONTRIBUTORS file. # # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.8.3. +# This file is part of Merlin, Version: 1.8.4. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/common/opennpylib.py b/merlin/common/opennpylib.py index bcb298035..793a869b6 100644 --- a/merlin/common/opennpylib.py +++ b/merlin/common/opennpylib.py @@ -1,14 +1,14 @@ #!/usr/bin/env python ############################################################################### -# Copyright (c) 2019, Lawrence Livermore National Security, LLC. +# Copyright (c) 2022, Lawrence Livermore National Security, LLC. # Produced at the Lawrence Livermore National Laboratory # Written by the Merlin dev team, listed in the CONTRIBUTORS file. # # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.8.3. +# This file is part of Merlin, Version: 1.8.4. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/common/sample_index.py b/merlin/common/sample_index.py index 6d4511307..8f6d4f9bf 100644 --- a/merlin/common/sample_index.py +++ b/merlin/common/sample_index.py @@ -1,12 +1,12 @@ ############################################################################### -# Copyright (c) 2019, Lawrence Livermore National Security, LLC. +# Copyright (c) 2022, Lawrence Livermore National Security, LLC. # Produced at the Lawrence Livermore National Laboratory # Written by the Merlin dev team, listed in the CONTRIBUTORS file. # # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.8.3. +# This file is part of Merlin, Version: 1.8.4. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/common/sample_index_factory.py b/merlin/common/sample_index_factory.py index e23fb9081..5161821b2 100644 --- a/merlin/common/sample_index_factory.py +++ b/merlin/common/sample_index_factory.py @@ -1,12 +1,12 @@ ############################################################################### -# Copyright (c) 2019, Lawrence Livermore National Security, LLC. +# Copyright (c) 2022, Lawrence Livermore National Security, LLC. # Produced at the Lawrence Livermore National Laboratory # Written by the Merlin dev team, listed in the CONTRIBUTORS file. # # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.8.3. +# This file is part of Merlin, Version: 1.8.4. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/common/security/__init__.py b/merlin/common/security/__init__.py index f78f9c91a..da4ad9f74 100644 --- a/merlin/common/security/__init__.py +++ b/merlin/common/security/__init__.py @@ -1,12 +1,12 @@ ############################################################################### -# Copyright (c) 2019, Lawrence Livermore National Security, LLC. +# Copyright (c) 2022, Lawrence Livermore National Security, LLC. # Produced at the Lawrence Livermore National Laboratory # Written by the Merlin dev team, listed in the CONTRIBUTORS file. # # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.8.3. +# This file is part of Merlin, Version: 1.8.4. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/common/security/encrypt.py b/merlin/common/security/encrypt.py index 7c12a70d4..c76eb0688 100644 --- a/merlin/common/security/encrypt.py +++ b/merlin/common/security/encrypt.py @@ -1,12 +1,12 @@ ############################################################################### -# Copyright (c) 2019, Lawrence Livermore National Security, LLC. +# Copyright (c) 2022, Lawrence Livermore National Security, LLC. # Produced at the Lawrence Livermore National Laboratory # Written by the Merlin dev team, listed in the CONTRIBUTORS file. # # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.8.3. +# This file is part of Merlin, Version: 1.8.4. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/common/security/encrypt_backend_traffic.py b/merlin/common/security/encrypt_backend_traffic.py index 1c566a377..b5f9953bc 100644 --- a/merlin/common/security/encrypt_backend_traffic.py +++ b/merlin/common/security/encrypt_backend_traffic.py @@ -1,12 +1,12 @@ ############################################################################### -# Copyright (c) 2019, Lawrence Livermore National Security, LLC. +# Copyright (c) 2022, Lawrence Livermore National Security, LLC. # Produced at the Lawrence Livermore National Laboratory # Written by the Merlin dev team, listed in the CONTRIBUTORS file. # # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.8.3. +# This file is part of Merlin, Version: 1.8.4. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/common/tasks.py b/merlin/common/tasks.py index 0737bd887..a3b63e0c3 100644 --- a/merlin/common/tasks.py +++ b/merlin/common/tasks.py @@ -1,12 +1,12 @@ ############################################################################### -# Copyright (c) 2019, Lawrence Livermore National Security, LLC. +# Copyright (c) 2022, Lawrence Livermore National Security, LLC. # Produced at the Lawrence Livermore National Laboratory # Written by the Merlin dev team, listed in the CONTRIBUTORS file. # # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.8.3. +# This file is part of Merlin, Version: 1.8.4. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/common/util_sampling.py b/merlin/common/util_sampling.py index 2c71cee2d..e11721cae 100644 --- a/merlin/common/util_sampling.py +++ b/merlin/common/util_sampling.py @@ -1,12 +1,12 @@ ############################################################################### -# Copyright (c) 2019, Lawrence Livermore National Security, LLC. +# Copyright (c) 2022, Lawrence Livermore National Security, LLC. # Produced at the Lawrence Livermore National Laboratory # Written by the Merlin dev team, listed in the CONTRIBUTORS file. # # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.8.3. +# This file is part of Merlin, Version: 1.8.4. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/config/__init__.py b/merlin/config/__init__.py index 47945b01b..f7c96c35e 100644 --- a/merlin/config/__init__.py +++ b/merlin/config/__init__.py @@ -1,12 +1,12 @@ ############################################################################### -# Copyright (c) 2019, Lawrence Livermore National Security, LLC. +# Copyright (c) 2022, Lawrence Livermore National Security, LLC. # Produced at the Lawrence Livermore National Laboratory # Written by the Merlin dev team, listed in the CONTRIBUTORS file. # # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.8.3. +# This file is part of Merlin, Version: 1.8.4. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/config/broker.py b/merlin/config/broker.py index 4084d13c2..5a0430dc4 100644 --- a/merlin/config/broker.py +++ b/merlin/config/broker.py @@ -1,12 +1,12 @@ ############################################################################### -# Copyright (c) 2019, Lawrence Livermore National Security, LLC. +# Copyright (c) 2022, Lawrence Livermore National Security, LLC. # Produced at the Lawrence Livermore National Laboratory # Written by the Merlin dev team, listed in the CONTRIBUTORS file. # # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.8.3. +# This file is part of Merlin, Version: 1.8.4. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/config/configfile.py b/merlin/config/configfile.py index 2d5141ae3..e237d741a 100644 --- a/merlin/config/configfile.py +++ b/merlin/config/configfile.py @@ -1,12 +1,12 @@ ############################################################################### -# Copyright (c) 2019, Lawrence Livermore National Security, LLC. +# Copyright (c) 2022, Lawrence Livermore National Security, LLC. # Produced at the Lawrence Livermore National Laboratory # Written by the Merlin dev team, listed in the CONTRIBUTORS file. # # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.8.3. +# This file is part of Merlin, Version: 1.8.4. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/config/results_backend.py b/merlin/config/results_backend.py index 01c210a7f..ad1504754 100644 --- a/merlin/config/results_backend.py +++ b/merlin/config/results_backend.py @@ -1,12 +1,12 @@ ############################################################################### -# Copyright (c) 2019, Lawrence Livermore National Security, LLC. +# Copyright (c) 2022, Lawrence Livermore National Security, LLC. # Produced at the Lawrence Livermore National Laboratory # Written by the Merlin dev team, listed in the CONTRIBUTORS file. # # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.8.3. +# This file is part of Merlin, Version: 1.8.4. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/data/celery/__init__.py b/merlin/data/celery/__init__.py index f78f9c91a..da4ad9f74 100644 --- a/merlin/data/celery/__init__.py +++ b/merlin/data/celery/__init__.py @@ -1,12 +1,12 @@ ############################################################################### -# Copyright (c) 2019, Lawrence Livermore National Security, LLC. +# Copyright (c) 2022, Lawrence Livermore National Security, LLC. # Produced at the Lawrence Livermore National Laboratory # Written by the Merlin dev team, listed in the CONTRIBUTORS file. # # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.8.3. +# This file is part of Merlin, Version: 1.8.4. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/display.py b/merlin/display.py index 2ad038034..9d58acbd0 100644 --- a/merlin/display.py +++ b/merlin/display.py @@ -1,12 +1,12 @@ ############################################################################### -# Copyright (c) 2019, Lawrence Livermore National Security, LLC. +# Copyright (c) 2022, Lawrence Livermore National Security, LLC. # Produced at the Lawrence Livermore National Laboratory # Written by the Merlin dev team, listed in the CONTRIBUTORS file. # # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.8.3. +# This file is part of Merlin, Version: 1.8.4. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/examples/__init__.py b/merlin/examples/__init__.py index f78f9c91a..da4ad9f74 100644 --- a/merlin/examples/__init__.py +++ b/merlin/examples/__init__.py @@ -1,12 +1,12 @@ ############################################################################### -# Copyright (c) 2019, Lawrence Livermore National Security, LLC. +# Copyright (c) 2022, Lawrence Livermore National Security, LLC. # Produced at the Lawrence Livermore National Laboratory # Written by the Merlin dev team, listed in the CONTRIBUTORS file. # # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.8.3. +# This file is part of Merlin, Version: 1.8.4. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/examples/examples.py b/merlin/examples/examples.py index bbbad64d9..059e0ae9c 100644 --- a/merlin/examples/examples.py +++ b/merlin/examples/examples.py @@ -1,12 +1,12 @@ ############################################################################### -# Copyright (c) 2019, Lawrence Livermore National Security, LLC. +# Copyright (c) 2022, Lawrence Livermore National Security, LLC. # Produced at the Lawrence Livermore National Laboratory # Written by the Merlin dev team, listed in the CONTRIBUTORS file. # # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.8.3. +# This file is part of Merlin, Version: 1.8.4. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/examples/generator.py b/merlin/examples/generator.py index d1ce08b4e..31ab74c5d 100644 --- a/merlin/examples/generator.py +++ b/merlin/examples/generator.py @@ -1,12 +1,12 @@ ############################################################################### -# Copyright (c) 2019, Lawrence Livermore National Security, LLC. +# Copyright (c) 2022, Lawrence Livermore National Security, LLC. # Produced at the Lawrence Livermore National Laboratory # Written by the Merlin dev team, listed in the CONTRIBUTORS file. # # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.8.3. +# This file is part of Merlin, Version: 1.8.4. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/examples/workflows/feature_demo/scripts/pgen.py b/merlin/examples/workflows/feature_demo/scripts/pgen.py index 99b9b105d..6ffb3ed37 100644 --- a/merlin/examples/workflows/feature_demo/scripts/pgen.py +++ b/merlin/examples/workflows/feature_demo/scripts/pgen.py @@ -5,7 +5,7 @@ def get_custom_generator(env, **kwargs): p_gen = ParameterGenerator() params = { "X2": {"values": [1 / i for i in range(3, 6)], "label": "X2.%%"}, - "N_NEW": {"values": [2 ** i for i in range(1, 4)], "label": "N_NEW.%%"}, + "N_NEW": {"values": [2**i for i in range(1, 4)], "label": "N_NEW.%%"}, } for key, value in params.items(): diff --git a/merlin/examples/workflows/null_spec/scripts/launch_jobs.py b/merlin/examples/workflows/null_spec/scripts/launch_jobs.py index bb5ba8a4d..99c3c3d6a 100644 --- a/merlin/examples/workflows/null_spec/scripts/launch_jobs.py +++ b/merlin/examples/workflows/null_spec/scripts/launch_jobs.py @@ -21,8 +21,8 @@ # launch n_samples * n_conc merlin workflow jobs submit_path: str = os.path.abspath(os.path.dirname(os.path.abspath(__file__))) -concurrencies: List[int] = [2 ** 0, 2 ** 1, 2 ** 2, 2 ** 3, 2 ** 4, 2 ** 5, 2 ** 6] -samples: List[int] = [10 ** 1, 10 ** 2, 10 ** 3, 10 ** 4, 10 ** 5] +concurrencies: List[int] = [2**0, 2**1, 2**2, 2**3, 2**4, 2**5, 2**6] +samples: List[int] = [10**1, 10**2, 10**3, 10**4, 10**5] nodes: List = [] c: int for c in concurrencies: diff --git a/merlin/examples/workflows/openfoam_wf/scripts/learn.py b/merlin/examples/workflows/openfoam_wf/scripts/learn.py index 1a4ba7c00..b7c40f6d3 100644 --- a/merlin/examples/workflows/openfoam_wf/scripts/learn.py +++ b/merlin/examples/workflows/openfoam_wf/scripts/learn.py @@ -26,7 +26,7 @@ U = outputs["arr_0"] enstrophy = outputs["arr_1"] -energy_byhand = np.sum(np.sum(U ** 2, axis=3), axis=2) / U.shape[2] / 2 +energy_byhand = np.sum(np.sum(U**2, axis=3), axis=2) / U.shape[2] / 2 enstrophy_all = np.sum(enstrophy, axis=2) X = np.load(inputs_dir + "/samples.npy") diff --git a/merlin/examples/workflows/openfoam_wf_no_docker/scripts/learn.py b/merlin/examples/workflows/openfoam_wf_no_docker/scripts/learn.py index 1a4ba7c00..b7c40f6d3 100644 --- a/merlin/examples/workflows/openfoam_wf_no_docker/scripts/learn.py +++ b/merlin/examples/workflows/openfoam_wf_no_docker/scripts/learn.py @@ -26,7 +26,7 @@ U = outputs["arr_0"] enstrophy = outputs["arr_1"] -energy_byhand = np.sum(np.sum(U ** 2, axis=3), axis=2) / U.shape[2] / 2 +energy_byhand = np.sum(np.sum(U**2, axis=3), axis=2) / U.shape[2] / 2 enstrophy_all = np.sum(enstrophy, axis=2) X = np.load(inputs_dir + "/samples.npy") diff --git a/merlin/examples/workflows/optimization/scripts/test_functions.py b/merlin/examples/workflows/optimization/scripts/test_functions.py index d6f391a0a..ab1a3ae4a 100644 --- a/merlin/examples/workflows/optimization/scripts/test_functions.py +++ b/merlin/examples/workflows/optimization/scripts/test_functions.py @@ -17,14 +17,14 @@ def rosenbrock(X): def rastrigin(X, A=10): first_term = A * len(inputs) - return first_term + sum([(x ** 2 - A * np.cos(2 * math.pi * x)) for x in X]) + return first_term + sum([(x**2 - A * np.cos(2 * math.pi * x)) for x in X]) def ackley(X): firstSum = 0.0 secondSum = 0.0 for x in X: - firstSum += x ** 2.0 + firstSum += x**2.0 secondSum += np.cos(2.0 * np.pi * x) n = float(len(X)) @@ -32,7 +32,7 @@ def ackley(X): def griewank(X): - term_1 = (1.0 / 4000.0) * sum(X ** 2) + term_1 = (1.0 / 4000.0) * sum(X**2) term_2 = 1.0 for i, x in enumerate(X): term_2 *= np.cos(x) / np.sqrt(i + 1) diff --git a/merlin/examples/workflows/remote_feature_demo/scripts/pgen.py b/merlin/examples/workflows/remote_feature_demo/scripts/pgen.py index ad13cc50b..5332d5b93 100644 --- a/merlin/examples/workflows/remote_feature_demo/scripts/pgen.py +++ b/merlin/examples/workflows/remote_feature_demo/scripts/pgen.py @@ -10,7 +10,7 @@ def get_custom_generator(env, **kwargs) -> ParameterGenerator: p_gen: ParameterGenerator = ParameterGenerator() params: Dict[str, Union[List[float], str]] = { "X2": {"values": [1 / i for i in range(3, 6)], "label": "X2.%%"}, - "N_NEW": {"values": [2 ** i for i in range(1, 4)], "label": "N_NEW.%%"}, + "N_NEW": {"values": [2**i for i in range(1, 4)], "label": "N_NEW.%%"}, } key: str value: Union[List[float], str] diff --git a/merlin/exceptions/__init__.py b/merlin/exceptions/__init__.py index 8fac45395..20a32755f 100644 --- a/merlin/exceptions/__init__.py +++ b/merlin/exceptions/__init__.py @@ -1,12 +1,12 @@ ############################################################################### -# Copyright (c) 2019, Lawrence Livermore National Security, LLC. +# Copyright (c) 2022, Lawrence Livermore National Security, LLC. # Produced at the Lawrence Livermore National Laboratory # Written by the Merlin dev team, listed in the CONTRIBUTORS file. # # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.8.3. +# This file is part of Merlin, Version: 1.8.4. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/log_formatter.py b/merlin/log_formatter.py index cbb285618..4e3969c2c 100644 --- a/merlin/log_formatter.py +++ b/merlin/log_formatter.py @@ -1,14 +1,14 @@ """This module handles setting up the extensive logging system in Merlin.""" ############################################################################### -# Copyright (c) 2019, Lawrence Livermore National Security, LLC. +# Copyright (c) 2022, Lawrence Livermore National Security, LLC. # Produced at the Lawrence Livermore National Laboratory # Written by the Merlin dev team, listed in the CONTRIBUTORS file. # # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.8.3. +# This file is part of Merlin, Version: 1.8.4. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/main.py b/merlin/main.py index 8a7b5d302..9f6c90a73 100644 --- a/merlin/main.py +++ b/merlin/main.py @@ -1,14 +1,14 @@ """The top level main function for invoking Merlin.""" ############################################################################### -# Copyright (c) 2019, Lawrence Livermore National Security, LLC. +# Copyright (c) 2022, Lawrence Livermore National Security, LLC. # Produced at the Lawrence Livermore National Laboratory # Written by the Merlin dev team, listed in the CONTRIBUTORS file. # # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.8.3. +# This file is part of Merlin, Version: 1.8.4. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/merlin_templates.py b/merlin/merlin_templates.py index f9885b242..94c57445e 100644 --- a/merlin/merlin_templates.py +++ b/merlin/merlin_templates.py @@ -1,12 +1,12 @@ ############################################################################### -# Copyright (c) 2019, Lawrence Livermore National Security, LLC. +# Copyright (c) 2022, Lawrence Livermore National Security, LLC. # Produced at the Lawrence Livermore National Laboratory # Written by the Merlin dev team, listed in the CONTRIBUTORS file. # # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.8.3. +# This file is part of Merlin, Version: 1.8.4. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/router.py b/merlin/router.py index 85e298510..c559883fa 100644 --- a/merlin/router.py +++ b/merlin/router.py @@ -1,12 +1,12 @@ ############################################################################### -# Copyright (c) 2019, Lawrence Livermore National Security, LLC. +# Copyright (c) 2022, Lawrence Livermore National Security, LLC. # Produced at the Lawrence Livermore National Laboratory # Written by the Merlin dev team, listed in the CONTRIBUTORS file. # # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.8.3. +# This file is part of Merlin, Version: 1.8.4. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/spec/__init__.py b/merlin/spec/__init__.py index f78f9c91a..da4ad9f74 100644 --- a/merlin/spec/__init__.py +++ b/merlin/spec/__init__.py @@ -1,12 +1,12 @@ ############################################################################### -# Copyright (c) 2019, Lawrence Livermore National Security, LLC. +# Copyright (c) 2022, Lawrence Livermore National Security, LLC. # Produced at the Lawrence Livermore National Laboratory # Written by the Merlin dev team, listed in the CONTRIBUTORS file. # # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.8.3. +# This file is part of Merlin, Version: 1.8.4. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/spec/all_keys.py b/merlin/spec/all_keys.py index cd4bf238c..3a344a98e 100644 --- a/merlin/spec/all_keys.py +++ b/merlin/spec/all_keys.py @@ -1,12 +1,12 @@ ############################################################################### -# Copyright (c) 2019, Lawrence Livermore National Security, LLC. +# Copyright (c) 2022, Lawrence Livermore National Security, LLC. # Produced at the Lawrence Livermore National Laboratory # Written by the Merlin dev team, listed in the CONTRIBUTORS file. # # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.8.3. +# This file is part of Merlin, Version: 1.8.4. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/spec/defaults.py b/merlin/spec/defaults.py index ef3f6da3e..74dc0be46 100644 --- a/merlin/spec/defaults.py +++ b/merlin/spec/defaults.py @@ -1,12 +1,12 @@ ############################################################################### -# Copyright (c) 2019, Lawrence Livermore National Security, LLC. +# Copyright (c) 2022, Lawrence Livermore National Security, LLC. # Produced at the Lawrence Livermore National Laboratory # Written by the Merlin dev team, listed in the CONTRIBUTORS file. # # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.8.3. +# This file is part of Merlin, Version: 1.8.4. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/spec/expansion.py b/merlin/spec/expansion.py index 6d4ec0329..30a71b25c 100644 --- a/merlin/spec/expansion.py +++ b/merlin/spec/expansion.py @@ -1,12 +1,12 @@ ############################################################################### -# Copyright (c) 2019, Lawrence Livermore National Security, LLC. +# Copyright (c) 2022, Lawrence Livermore National Security, LLC. # Produced at the Lawrence Livermore National Laboratory # Written by the Merlin dev team, listed in the CONTRIBUTORS file. # # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.8.3. +# This file is part of Merlin, Version: 1.8.4. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/spec/specification.py b/merlin/spec/specification.py index eb77c3157..122b8f002 100644 --- a/merlin/spec/specification.py +++ b/merlin/spec/specification.py @@ -1,12 +1,12 @@ ############################################################################### -# Copyright (c) 2019, Lawrence Livermore National Security, LLC. +# Copyright (c) 2022, Lawrence Livermore National Security, LLC. # Produced at the Lawrence Livermore National Laboratory # Written by the Merlin dev team, listed in the CONTRIBUTORS file. # # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.8.3. +# This file is part of Merlin, Version: 1.8.4. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/study/__init__.py b/merlin/study/__init__.py index f78f9c91a..da4ad9f74 100644 --- a/merlin/study/__init__.py +++ b/merlin/study/__init__.py @@ -1,12 +1,12 @@ ############################################################################### -# Copyright (c) 2019, Lawrence Livermore National Security, LLC. +# Copyright (c) 2022, Lawrence Livermore National Security, LLC. # Produced at the Lawrence Livermore National Laboratory # Written by the Merlin dev team, listed in the CONTRIBUTORS file. # # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.8.3. +# This file is part of Merlin, Version: 1.8.4. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/study/batch.py b/merlin/study/batch.py index dec4cfac4..0be2e8739 100644 --- a/merlin/study/batch.py +++ b/merlin/study/batch.py @@ -1,12 +1,12 @@ ############################################################################### -# Copyright (c) 2019, Lawrence Livermore National Security, LLC. +# Copyright (c) 2022, Lawrence Livermore National Security, LLC. # Produced at the Lawrence Livermore National Laboratory # Written by the Merlin dev team, listed in the CONTRIBUTORS file. # # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.8.3. +# This file is part of Merlin, Version: 1.8.4. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/study/celeryadapter.py b/merlin/study/celeryadapter.py index eef29aa63..b7045bc46 100644 --- a/merlin/study/celeryadapter.py +++ b/merlin/study/celeryadapter.py @@ -1,12 +1,12 @@ ############################################################################### -# Copyright (c) 2019, Lawrence Livermore National Security, LLC. +# Copyright (c) 2022, Lawrence Livermore National Security, LLC. # Produced at the Lawrence Livermore National Laboratory # Written by the Merlin dev team, listed in the CONTRIBUTORS file. # # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.8.3. +# This file is part of Merlin, Version: 1.8.4. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/study/dag.py b/merlin/study/dag.py index 31ef0f08b..6821d486e 100644 --- a/merlin/study/dag.py +++ b/merlin/study/dag.py @@ -1,12 +1,12 @@ ############################################################################### -# Copyright (c) 2019, Lawrence Livermore National Security, LLC. +# Copyright (c) 2022, Lawrence Livermore National Security, LLC. # Produced at the Lawrence Livermore National Laboratory # Written by the Merlin dev team, listed in the CONTRIBUTORS file. # # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.8.3. +# This file is part of Merlin, Version: 1.8.4. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/study/script_adapter.py b/merlin/study/script_adapter.py index 07c4622b9..63a11d0c7 100644 --- a/merlin/study/script_adapter.py +++ b/merlin/study/script_adapter.py @@ -1,12 +1,12 @@ ############################################################################### -# Copyright (c) 2019, Lawrence Livermore National Security, LLC. +# Copyright (c) 2022, Lawrence Livermore National Security, LLC. # Produced at the Lawrence Livermore National Laboratory # Written by the Merlin dev team, listed in the CONTRIBUTORS file. # # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.8.3. +# This file is part of Merlin, Version: 1.8.4. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/study/step.py b/merlin/study/step.py index af331c3cf..649610ab6 100644 --- a/merlin/study/step.py +++ b/merlin/study/step.py @@ -1,12 +1,12 @@ ############################################################################### -# Copyright (c) 2019, Lawrence Livermore National Security, LLC. +# Copyright (c) 2022, Lawrence Livermore National Security, LLC. # Produced at the Lawrence Livermore National Laboratory # Written by the Merlin dev team, listed in the CONTRIBUTORS file. # # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.8.3. +# This file is part of Merlin, Version: 1.8.4. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/study/study.py b/merlin/study/study.py index 1cf1e2adc..d98a31419 100644 --- a/merlin/study/study.py +++ b/merlin/study/study.py @@ -1,12 +1,12 @@ ############################################################################### -# Copyright (c) 2019, Lawrence Livermore National Security, LLC. +# Copyright (c) 2022, Lawrence Livermore National Security, LLC. # Produced at the Lawrence Livermore National Laboratory # Written by the Merlin dev team, listed in the CONTRIBUTORS file. # # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.8.3. +# This file is part of Merlin, Version: 1.8.4. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/utils.py b/merlin/utils.py index b35ef2b77..657349fed 100644 --- a/merlin/utils.py +++ b/merlin/utils.py @@ -1,12 +1,12 @@ ############################################################################### -# Copyright (c) 2019, Lawrence Livermore National Security, LLC. +# Copyright (c) 2022, Lawrence Livermore National Security, LLC. # Produced at the Lawrence Livermore National Laboratory # Written by the Merlin dev team, listed in the CONTRIBUTORS file. # # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.8.3. +# This file is part of Merlin, Version: 1.8.4. # # For details, see https://github.com/LLNL/merlin. # diff --git a/requirements/dev.txt b/requirements/dev.txt index c66854a61..9321694f8 100644 --- a/requirements/dev.txt +++ b/requirements/dev.txt @@ -1,4 +1,5 @@ # Development dependencies. +build black dep-license flake8 diff --git a/setup.py b/setup.py index e9ed37693..68e4b8ed7 100644 --- a/setup.py +++ b/setup.py @@ -1,12 +1,12 @@ ############################################################################### -# Copyright (c) 2019, Lawrence Livermore National Security, LLC. +# Copyright (c) 2022, Lawrence Livermore National Security, LLC. # Produced at the Lawrence Livermore National Laboratory # Written by the Merlin dev team, listed in the CONTRIBUTORS file. # # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.8.3. +# This file is part of Merlin, Version: 1.8.4. # # For details, see https://github.com/LLNL/merlin. # @@ -100,6 +100,7 @@ def extras_require(): "Programming Language :: Python :: 3.7", "Programming Language :: Python :: 3.8", "Programming Language :: Python :: 3.9", + "Programming Language :: Python :: 3.10", ], keywords="machine learning workflow", url="https://github.com/LLNL/merlin", diff --git a/tests/integration/run_tests.py b/tests/integration/run_tests.py index f92f20354..c7d3745b1 100644 --- a/tests/integration/run_tests.py +++ b/tests/integration/run_tests.py @@ -1,12 +1,12 @@ ############################################################################### -# Copyright (c) 2019, Lawrence Livermore National Security, LLC. +# Copyright (c) 2022, Lawrence Livermore National Security, LLC. # Produced at the Lawrence Livermore National Laboratory # Written by the Merlin dev team, listed in the CONTRIBUTORS file. # # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.8.3. +# This file is part of Merlin, Version: 1.8.4. # # For details, see https://github.com/LLNL/merlin. #