Merge branch 'main' into devel

.github/workflows/testing.yml

@@ -0,0 +1,46 @@
+name: Testing
+
+on: [push, pull_request]
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+
+    steps:
+      - uses: actions/checkout@v4
+      - name: Set up micromamba
+        uses: mamba-org/[email protected]
+        with:
+          # the create command looks like this:
+          # `micromamba create -n test-env python=<the corresponding version> kim-api=2.3.0`
+          environment-name: test-env
+          cache-environment: true
+          create-args: >-
+            -f tests/environment.yml
+
+      - name: Install AFLOW
+        shell: bash -el {0}
+        run: |
+          wget https://materials.duke.edu/AFLOW/aflow.3.2.14.tar.xz
+          tar xf aflow.3.2.14.tar.xz
+          cd aflow.3.2.14
+          make -j4
+          cd ..
+
+      - name: Install KIM model
+        shell: bash -el {0}
+        run: |
+          kim-api-collections-management install user LJ_Shifted_Bernardes_1958LowCutoff_Ar__MO_720819638419_004
+
+      - name: Install
+        shell: bash -el {0}
+        run: |
+          python -m pip install .
+
+      - name: Run tests
+        shell: bash -el {0}
+        run: |
+          export PATH=$PATH:$PWD/aflow.3.2.14
+          cd tests
+          export KIM_PROPERTY_PATH=$PWD/mock-test-drivers-dir/*/local-props:$PWD/mock-test-drivers-dir/*/local_props          
+          pytest

.gitignore

@@ -7,8 +7,9 @@ dist/
 build/
 dist-old/
 
-# for when we add tests
+# test stuff
 kim.log
+lammps.log
 /tests/kim.log
 /tests/__pycache__/
 /tests/*/__pycache__/

bin/add_or_update_property

@@ -1,14 +1,9 @@
 #!/usr/bin/python
+from kim_tools.test_driver.core import PROP_SEARCH_PATHS_INFO
 
-from kim_tools import add_or_update_property
-from sys import argv
-from os.path import isfile
-
-if len(argv) != 2:
-    raise RuntimeError('This script must be passed exactly one command-line argument.')
-if not isfile(argv[1]):
-    raise RuntimeError('The argument passed is not a valid file path.')
-try:
-    add_or_update_property(argv[1])
-except Exception:
-    raise RuntimeError('Failed to add or update property from definition file located at:\n%s'%argv[1])
+print('\n!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!\n'+\
+      'THIS UTILITY IS DEPRECATED\n'+\
+      'kim-tools should now be able to find your custom properties automatically.\n'+\
+      'It recursively looks in the following places:\n'+\
+      PROP_SEARCH_PATHS_INFO+\
+      '\n!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!\n')

docs/source/index.rst

@@ -78,7 +78,7 @@ Once the above prerequisites are satisfiied, you can install using ``pip``:
 
    pip install --user git+https://github.com/openkim/kim-tools.git
 
-If you are installing on an HPC cluster, you may need to use different options to achieve the desired result. Note that if you wish to be able to use the included command-line utility ``add_or_update_property`` (you will if you are developing a new Crystal Genome Test Driver), the prerequisite package ``kim-property`` must be installed in an editable location. The ``--user`` option included in the example command does so.
+If you are installing on an HPC cluster, you may need to use different options to achieve the desired result.
 
 API Documentation, Indices and tables
 =====================================

docs/source/tutorial_debug.rst

@@ -12,12 +12,8 @@ A practical guide for testing your Driver on a variety of crystal structures is
 Everything you need to run the example script is containerized in the :ref:`doc.KDP`,
 or alternatively will be installed if you follow the :ref:`doc.standalone_installation`. 
 
-If you have not yet done so, you must also use the ``add_or_update_property`` command-line tool packaged with ``kim-tools``
-to add any custom properties:
-
-.. code-block:: bash
-
-    add_or_update_property ~/test-drivers/CrystalGenomeASEExample__TD_000000654321_000/local-props/energy-vs-volume-isotropic-crystal.edn
+``kim-tools`` will automatically look for property definitions in the ``local-props`` and ``local_props`` subdirectories of the current working directory. If you wish to put them somewhere else,
+you can point the environment variable ``KIM_PROPERTY_PATH`` to their location. ``kim-tools`` will expand any globs, including recursive ``**``.
 
 Example Script for Running a Crystal Genome Test Driver
 =======================================================

docs/source/tutorial_pipeline.rst

@@ -60,7 +60,6 @@ Assuming you have left the first few lines of ``test_generator.json`` intact, yo
 .. code-block:: bash
 
     kimitems install -D  EAM_Dynamo_AcklandTichyVitek_1987_Ag__MO_212700056563_005
-    add_or_update_property ~/test-drivers/CrystalGenomeASEExample__TD_000000654321_000/local-props/energy-vs-volume-isotropic-crystal.edn
     pipeline-run-pair CrystalGenomeASEExample_A_cF4_225_a_Ag_0_1__TE_* EAM_Dynamo_AcklandTichyVitek_1987_Ag__MO_212700056563_005 -v
 
 You should see the output of the test, and there should be a new directory in ``~/test-results/`` with the ``results.edn`` file containing the resulting KIM Property Instance, 

docs/source/tutorial_property.rst

@@ -16,11 +16,8 @@ Here are some examples of existing Crystal Genome properties:
 
 It is possible that your Test Driver will write multiple material properties. In this case, create a separate property definition file for each. For example, the bulk modulus and elastic constants shown above are separate properties.
 
-Once you have created your property definition file, you need to run the ``add_or_update_property`` command-line utility included with this package. Pass the path to the property definition file as a command line argument. For example, to add the property provided with ``CrystalGenomeASEExample__TD_000000654321_000``, assuming you have placed the driver into ``/home/openkim/test-drivers/``, run 
-
-.. code-block:: bash
-
-    add_or_update_property /home/openkim/test-drivers/CrystalGenomeASEExample__TD_000000654321_000/local-props/energy-vs-volume-isotropic-crystal.edn
+``kim-tools`` will automatically look for property definitions in the ``local-props`` and ``local_props`` subdirectories of the current working directory. If you wish to put them somewhere else,
+you can point the environment variable ``KIM_PROPERTY_PATH`` to their location. ``kim-tools`` will expand any globs, including recursive ``**``.
 
 Once your property is finalized, you will need to request a member of the OpenKIM team to permanently add it to the collection of `KIM Property Definitions <https://openkim.org/properties>`_. Your property should only be added after you have gone through the rest of the steps in this tutorial to make sure you do not encounter any unanticipated problems while developing your Test Driver. However, you are encouraged to contact us early in the development to begin iterating on your property definiton.
 

kim_tools/test_driver/core.py

@@ -47,8 +47,7 @@
 from ase.optimize.optimize import Optimizer
 from ase.constraints import ExpCellFilter, UnitCellFilter
 from abc import ABC, abstractmethod
-import kim_property
-from kim_property import kim_property_create, kim_property_modify, kim_property_dump
+from kim_property import kim_property_create, kim_property_modify, kim_property_dump, get_properties, get_property_id_path
 from kim_property.modify import STANDARD_KEYS_SCLAR_OR_WITH_EXTENT
 import kim_edn
 from .. import aflow_util
@@ -57,9 +56,8 @@
 import os
 from warnings import warn
 from io import StringIO
-from packaging import version
-from pprint import PrettyPrinter
 import shutil
+from pathlib import Path
 
 __author__ = ["ilia Nikiforov", "Eric Fuemmeler"]
 __all__ = [
@@ -70,25 +68,16 @@
     "CrystalGenomeTestDriver",
     "query_crystal_genome_structures",
     "minimize_wrapper",
-    "add_or_update_property",
-]
-
-def add_or_update_property(property_path:str):
-    assert os.access(kim_property.__path__[0],os.W_OK), 'kim_property must be installed in an editable location in order to add properties'
-    properties=kim_property.get_properties()
-    property=kim_edn.load(property_path)
-    property_id=property['property-id']
-    properties[property_id]=property
-    if version.parse(kim_property.__version__) < version.parse('2.6.0'):
-        kim_property.pickle.pickle_kim_properties(properties)
-    else:
-        kim_property.ednify.ednify_kim_properties(properties)
-    PrettyPrinter().pprint(kim_property.get_properties()[property_id])
-    print('\n\nSuccessfully pickled or ednified properties! Scroll up to check the property you just added.')
+]    
 
 FMAX_INITIAL = 1e-5 # Force tolerance for the optional initial relaxation of the provided cell
 MAXSTEPS_INITIAL = 10000 # Maximum steps for the optional initial relaxation of the provided cell
 
+PROP_SEARCH_PATHS_INFO=(\
+'- $KIM_PROPERTY_PATH (expanding globs including recursive **)\n'
+'- $PWD/local-props/**/\n'
+'- $PWD/local_props/**/')
+
 def minimize_wrapper(supercell:Atoms, fmax:float=1e-5, steps:int=10000, \
                          variable_cell:bool=True, logfile:Optional[Union[str,IO]]='-',
                          algorithm: Optimizer = LBFGSLineSearch, 
@@ -292,6 +281,63 @@ def _add_property_instance(self, property_name: str, disclaimer: Optional[str]=N
             if property_instance["instance-id"] == new_instance_index:
                 raise KIMTestDriverError("instance-id that matches the length of self.property_instances already exists.\n"
                                   "Was self.property_instances edited directly instead of using this package?")
+        existing_properties = get_properties()
+        property_in_existing_properties = False
+        for existing_property in existing_properties:
+            if existing_property == property_name or get_property_id_path(existing_property)[3] == property_name:
+                property_in_existing_properties = True
+
+        if not property_in_existing_properties:
+            print('\nThe property name or id\n%s\nwas not found in kim-properties.\n'%property_name)
+            print('I will now look for an .edn file containing its definition in the following locations:\n%s\n'%PROP_SEARCH_PATHS_INFO)
+
+            property_search_paths = []
+
+            # environment varible
+            if 'KIM_PROPERTY_PATH' in os.environ:
+                property_search_paths += os.environ['KIM_PROPERTY_PATH'].split(':')
+
+            # CWD
+            property_search_paths.append(os.path.join(Path.cwd(),'local_props','**'))
+            property_search_paths.append(os.path.join(Path.cwd(),'local-props','**'))
+
+            # recursively search for .edn files in the paths, check if they are a property definition
+            # with the correct name
+
+            found_custom_property = False
+
+            for search_path in property_search_paths:
+                if found_custom_property:
+                    break
+                else:
+                    # hack to expand globs in both absolute and relative paths
+                    if search_path[0] == '/':
+                        base_path = Path('/')
+                        search_glob = os.path.join(search_path[1:],'*.edn')
+                    else:
+                        base_path = Path()
+                        search_glob = os.path.join(search_path,'*.edn')
+
+                    for path in base_path.glob(search_glob):
+                        if not os.path.isfile(path): # in case there's a directory named *.edn
+                            continue 
+                        try:
+                            path_str = str(path)
+                            dict_from_edn = kim_edn.load(path_str)
+                            if ('property-id') in dict_from_edn:
+                                property_id = dict_from_edn['property-id']
+                                if property_id == property_name or get_property_id_path(property_id)[3] == property_name:
+                                    property_name = path_str
+                                    found_custom_property = True
+                                    break
+                        except Exception as e:
+                            pass
+
+            if not found_custom_property:
+                raise KIMTestDriverError(
+                    '\nThe property name or id\n%s\nwas not found in kim-properties.\n'%property_name + \
+                    'I failed to find an .edn file containing a matching "property-id" key in the following locations:\n' + PROP_SEARCH_PATHS_INFO)
+
         self._property_instances = kim_property_create(new_instance_index, property_name, self._property_instances, disclaimer)
 
     def _add_key_to_current_property_instance(self,

tests/environment.yml

@@ -0,0 +1,10 @@
+name: kim-tools-env
+channels:
+  - conda-forge
+dependencies:
+  - kimpy
+  - ase
+  - spglib
+  - kim-property
+  - kim-query
+  - kim-edn

tests/local-props/atomic-mass0.edn

@@ -0,0 +1,19 @@
+{
+  "property-id"           "tag:[email protected],2016-05-11:property/atomic-mass0"
+  "property-title"        "Atomic mass"
+  "property-description"  "The atomic mass of the element"
+  "species" {
+    "type"         "string"
+    "has-unit"     false
+    "extent"       []
+    "required"     true
+    "description"  "Element symbol of the species"
+  }
+  "mass" {
+    "type"         "float"
+    "has-unit"     true
+    "extent"       []
+    "required"     true
+    "description"  "Mass of a single atom of the species"
+  }
+}

tests/local-props/atomic-mass1.edn

@@ -0,0 +1,19 @@
+{
+  "property-id"           "tag:[email protected],2016-05-11:property/atomic-mass1"
+  "property-title"        "Atomic mass"
+  "property-description"  "The atomic mass of the element"
+  "species" {
+    "type"         "string"
+    "has-unit"     false
+    "extent"       []
+    "required"     true
+    "description"  "Element symbol of the species"
+  }
+  "mass" {
+    "type"         "float"
+    "has-unit"     true
+    "extent"       []
+    "required"     true
+    "description"  "Mass of a single atom of the species"
+  }
+}

tests/local-props/atomic-mass2/atomic-mass2.edn

@@ -0,0 +1,19 @@
+{
+  "property-id"           "tag:[email protected],2016-05-11:property/atomic-mass2"
+  "property-title"        "Atomic mass"
+  "property-description"  "The atomic mass of the element"
+  "species" {
+    "type"         "string"
+    "has-unit"     false
+    "extent"       []
+    "required"     true
+    "description"  "Element symbol of the species"
+  }
+  "mass" {
+    "type"         "float"
+    "has-unit"     true
+    "extent"       []
+    "required"     true
+    "description"  "Mass of a single atom of the species"
+  }
+}

tests/mock-test-drivers-dir/mock-td/local_props/atomic-mass3.edn

@@ -0,0 +1,19 @@
+{
+  "property-id"           "tag:[email protected],2016-05-11:property/atomic-mass3"
+  "property-title"        "Atomic mass"
+  "property-description"  "The atomic mass of the element"
+  "species" {
+    "type"         "string"
+    "has-unit"     false
+    "extent"       []
+    "required"     true
+    "description"  "Element symbol of the species"
+  }
+  "mass" {
+    "type"         "float"
+    "has-unit"     true
+    "extent"       []
+    "required"     true
+    "description"  "Mass of a single atom of the species"
+  }
+}

tests/test_CrystalGenomeTest.py

@@ -1,26 +1,19 @@
 #!/usr/bin/python
 
 from kim_tools.test_driver import CrystalGenomeTestDriver, query_crystal_genome_structures
-from kim_python_utils.ase import get_isolated_energy_per_atom
 from kim_tools.aflow_util import get_stoich_reduced_list_from_prototype
 
 class TestTestDriver(CrystalGenomeTestDriver):
     def _calculate(self,**kwargs):
         """
         example calculate method. Just writes the binding-energy and crystal-structure-npt properties assuming the provided
         structure is already at equilibrium
-
-        Args:
-            structure_index:
-                KIM tests can loop over multiple structures (i.e. crystals, molecules, etc.). This indicates which is being used for the current calculation.        
         """
         # calculate potential energy and do the required stuff to figure out per-formula and per-atom, and subtract isolated energy
         potential_energy = self.atoms.get_potential_energy()
         potential_energy_per_atom = potential_energy/len(self.atoms)
         reduced_stoichiometry = get_stoich_reduced_list_from_prototype(self.prototype_label) # i.e. "AB3\_...." -> [1,3]        
         binding_energy_per_formula = potential_energy_per_atom * sum(reduced_stoichiometry)
-        for num_in_formula,species in zip(reduced_stoichiometry,self.stoichiometric_species):
-            binding_energy_per_formula -= num_in_formula*get_isolated_energy_per_atom(self.kim_model_name,species)
         binding_energy_per_atom = binding_energy_per_formula/sum(reduced_stoichiometry)
 
         # add property instance and common fields
@@ -35,11 +28,10 @@ def _calculate(self,**kwargs):
             f.write("dummy\nfile")
         self._add_file_to_current_property_instance("restart-file","restart.dump")
 
-
-test = TestTestDriver("MEAM_LAMMPS_KoJimLee_2012_FeP__MO_179420363944_002")
-list_of_crystal_descriptions = query_crystal_genome_structures("MEAM_LAMMPS_KoJimLee_2012_FeP__MO_179420363944_002",["Fe","P"],"AB_oP8_62_c_c")
-print(list_of_crystal_descriptions)
-test(**list_of_crystal_descriptions[0])
-test(**list_of_crystal_descriptions[1])
-print(test.property_instances)
-test.write_property_instances_to_file()
+
+def test_cg():        
+    test = TestTestDriver("LJ_Shifted_Bernardes_1958LowCutoff_Ar__MO_720819638419_004")
+    list_of_crystal_descriptions = query_crystal_genome_structures("LJ_Shifted_Bernardes_1958LowCutoff_Ar__MO_720819638419_004",["Ar"],"A_hP2_194_c")
+    test(**list_of_crystal_descriptions[0])    
+    assert len(test.property_instances) == 2
+    test.write_property_instances_to_file()