Merge pull request #608 from ARGOeu/devel

Prepare to Release [3.4.11] - 2024-06-07

CHANGELOG.md

@@ -1,5 +1,22 @@
 # Changelog
 
+## [3.4.11] - 2024-06-07
+
+### Added
+
+* ARGO-4382 Placeholders instead of loading icon
+
+### Fixed
+
+* ARGO-4530 Minor code fixes and major tests fixes wrt react-router-dom bump
+
+### Changed
+
+* ARGO-4383 Bump react-router-dom-v6
+* ARGO-4384 Upgrade React to version 18
+* ARGO-4385 Upgrade Bootstrap to version 5.3.2
+* ARGO-4543 Bump outdated python packages
+
 ## [3.4.10] - 2023-09-11
 
 ### Added

doc/v1/docs/superadmin_packages.md

@@ -23,8 +23,9 @@ Mandatory fields:
 * **YUM repo** - **at least one** YUM repo must be defined; YUM repo which provides the given package:
     * **CentOS 6 repo** - repo tagged with CentOS 6 tag,
     * **CentOS 7 repo** - repo tagged with CentOS 7 tag.
+    * **Rocky 9 repo** - repo tagged with Rocky 9 tag.
 
-If **Use version which is present in repo** is checked, package version is going to be tagged as present, and the version which exists in the repo is going to be installed on the monitoring box.
+If the **Use version which is present in repo** is checked, package version is going to be tagged as present, and the version which exists in the repo is going to be installed on the monitoring box.
 
 
 ### List of probes provided by the package

doc/v1/docs/superadmin_repos.md

@@ -19,6 +19,7 @@ All the fields are mandatory:
 * **Name** - name of YUM repo (**without** `.repo` file extension);
 * **Tag** - tag which marks for which OS the repo is defined:
     * CentOS 6,
-    * CentOS 7;
+    * CentOS 7,
+    * Rocky 9;
 * **File content** - content of the YUM repo file;
 * **Description** - short free text description, shown in the YUM repo list.

doc/v1/docs/tenant_servicetypes.md

@@ -2,7 +2,7 @@
 
 Service types page is accessible from the menu on the left side. The data is presented in a table, with information on the name of service type, service description, its source (either topology or POEM), and a checkbox column used for deletion of service types.
 
-Service type can be defined as part of the topology (`topology` source tag), or it can be defined in this page (`POEM` source tag). Note that the description for the service types defined in the topology cannot be modified, while the same field for service types defined through POEM can be modified. If modified, it is necessary to click the `Save` button in order to save the changes to ARGO Web-API.
+Service type can be defined as part of the topology (`topology` source tag), or it can be defined in this page (`poem` source tag). Note that the description for the service types defined in the topology cannot be modified, while the same field for service types defined through POEM can be modified. If modified, it is necessary to click the `Save` button in order to save the changes to ARGO Web-API.
 
 The page layout is shown in the figure below. Service types can be searched by name, or by description.
 
@@ -12,17 +12,24 @@ Some tenants may have also service type titles defined. In that case, they are s
 
 ![Tenant Service Types with Titles](figures/tenant_servicetypes_with_titles.png)
 
+## Importing/exporting service types
+
+Service types can be exported to a `.csv` file, and also imported from such a file. This is done by clicking on the `CSV` button and selecting `Export` for exporting of `.csv` file, and `Import` for importing data from a `.csv` file.
+
+In case of export, the file name is of the form `<TENANT_NAME>-service-types-devel.csv` in case of devel POEM, and `<TENANT_NAME>-service-types.csv` in case of production POEM.
+
+When importing service types, all the existing service types are overridden, and the new (imported) ones all have `poem` tag. Keep in mind that if there are service types defined in the topology, they will be recreated next time the connector syncs the data from the topology (the ones added through POEM will remain).
+
 ## Adding service types
 
-Service types can be bulk added by clicking on the `Add` button. It opens a new page shown in the figure below. Users add service name and description in the top fields, and by clicking the `Add new` button, add them to the table below. Descriptions can be conveniently edited in the table as well. The changes are sent to the ARGO Web API by clicking the button `Save`. Service types added this way are always stored with `POEM` source tag.
+Service types can be bulk added by clicking on the `Add` button. It opens a new page shown in the figure below. Users add service name and description in the top fields, and by clicking the `Add new` button, add them to the table below. Descriptions can be conveniently edited in the table as well. The changes are sent to the ARGO Web API by clicking the button `Save`. Service types added this way are always stored with `poem` source tag.
 
 ![Tenant Service Types Bulk Add](figures/tenant_servicetypes_bulk_add.png)
 
 In case the tenant is using service types' titles as well, the form has one additional field for the title, as shown in figure below. The saving of service type is the same as in case without the titles.
+
 ![Tenant Service Types Bulk Add with Titles](figures/tenant_servicetypes_bulk_add_with_titles.png)
 
 ## Deleting service types
 
-Service types may be deleted by clicking in the `Checked` column of the row(s) of the service type(s) you wish to delete, and finally clicking on the `Delete selected` button. 
-
-Keep in mind that the service types which are defined in the topology will be restored when ARGO Web-API is synced with the topology source. Service types defined in the topology should be removed at the source, if they need to be removed.
+Service types may be deleted by clicking in the `Select` column of the row(s) of the service type(s) you wish to delete, or, alternatively, you can use the checkbox just below the `Select` column name, to select all of them. Finally, you delete them by clicking on the `Delete selected` button. 

docker/Dockerfile

@@ -1,6 +1,6 @@
 FROM centos:7
 MAINTAINER [email protected]
-ADD https://repository.egi.eu/sw/production/cas/1/current/repo-files/EGI-trustanchors.repo /etc/yum.repos.d/
+ADD https://repository.egi.eu/sw/production/cas/1/current/repo-files/egi-trustanchors.repo /etc/yum.repos.d/
 RUN groupadd user -g 1000 && useradd -u 1000 -g 1000 user -m -d /home/user
 RUN passwd -d root
 RUN sed -i -r 's/bash/zsh/' /etc/passwd; sed -i -r 's/^tsflags/# tsflags/' /etc/yum.conf

poem/Poem/Makefile

@@ -10,7 +10,7 @@ devel-watch:
 	node_modules/.bin/webpack --config webpack.config.js --progress --mode development --watch
 
 devel-django-server:
-	docker exec -ti $(container_name) ./run-django-server.sh
+	docker exec -ti $(container_name) /home/user/run-django-server.sh
 
 devel-webpack-server:
 	node_modules/.bin/webpack-dev-server --config webpack.devserver.config.js --progress

poem/Poem/api/internal_views/metrictemplates.py

@@ -24,30 +24,54 @@
 class ListMetricTemplates(APIView):
     authentication_classes = (SessionAuthentication,)
 
+    def _get_metrics_per_tenant(self, schema_name):
+        with schema_context(schema_name):
+            metrics = [m.name for m in Metric.objects.all()]
+
+        return metrics
+
     def get(self, request, name=None):
         public_tenant = None
-        importable = None
+        tenants = dict()
         if name:
             metrictemplates = admin_models.MetricTemplate.objects.filter(
                 name=name
             )
             if metrictemplates.count() == 0:
                 raise NotFound(status=404, detail='Metric template not found')
+
         else:
             public_tenant = Tenant.objects.get(
                 schema_name=get_public_schema_name()
             )
             metrictemplates = admin_models.MetricTemplate.objects.all()
-            if request.tenant != public_tenant:
-                avail_metrics = Metric.objects.all().values_list(
-                    'name', flat=True
-                )
-                importable = dict()
+
+            if request.tenant == public_tenant:
+                tenants_metrics = dict()
+                for tenant in Tenant.objects.all():
+                    if tenant != public_tenant:
+                        tenants_metrics.update({
+                            tenant.name: self._get_metrics_per_tenant(
+                                tenant.schema_name
+                            )
+                        })
                 for metrictemplate in metrictemplates:
-                    if metrictemplate.name in avail_metrics:
-                        importable[metrictemplate.name] = False
-                    else:
-                        importable[metrictemplate.name] = True
+                    if request.tenant == public_tenant:
+                        mt_versions = [
+                            m.name for m in
+                            admin_models.MetricTemplateHistory.objects.filter(
+                                object_id=metrictemplate
+                            )
+                        ]
+
+                        tenants.update({
+                            metrictemplate.name: sorted([
+                                t for t, m in tenants_metrics.items() if
+                                len(
+                                    set(mt_versions).intersection(set(m))
+                                ) > 0
+                            ])
+                        })
 
         results = []
         for metrictemplate in metrictemplates:
@@ -75,45 +99,34 @@ def get(self, request, name=None):
             else:
                 probeversion = ''
 
-            if not name and request.tenant != public_tenant:
-                results.append(dict(
-                    id=metrictemplate.id,
-                    name=metrictemplate.name,
-                    importable=importable[metrictemplate.name],
-                    mtype=metrictemplate.mtype.name,
-                    ostag=ostag,
-                    tags=sorted(tags),
-                    probeversion=probeversion,
-                    description=metrictemplate.description,
-                    parent=parent,
-                    probeexecutable=probeexecutable,
-                    config=config,
-                    attribute=attribute,
-                    dependency=dependency,
-                    flags=flags,
-                    files=files,
-                    parameter=parameter,
-                    fileparameter=fileparameter
-                ))
-            else:
-                results.append(dict(
-                    id=metrictemplate.id,
-                    name=metrictemplate.name,
-                    mtype=metrictemplate.mtype.name,
-                    ostag=ostag,
-                    tags=sorted(tags),
-                    probeversion=probeversion,
-                    description=metrictemplate.description,
-                    parent=parent,
-                    probeexecutable=probeexecutable,
-                    config=config,
-                    attribute=attribute,
-                    dependency=dependency,
-                    flags=flags,
-                    files=files,
-                    parameter=parameter,
-                    fileparameter=fileparameter
-                ))
+            result_dict = {
+                "id": metrictemplate.id,
+                "name": metrictemplate.name,
+                "mtype": metrictemplate.mtype.name,
+                "ostag": ostag,
+                "tags": sorted(tags),
+                "probeversion": probeversion,
+                "description": metrictemplate.description,
+                "parent": parent,
+                "probeexecutable": probeexecutable,
+                "config": config,
+                "attribute": attribute,
+                "dependency": dependency,
+                "flags": flags,
+                "files": files,
+                "parameter": parameter,
+                "fileparameter": fileparameter
+            }
+
+            if not name and request.tenant == public_tenant:
+                if metrictemplate.name in tenants:
+                    tl = tenants[metrictemplate.name]
+                else:
+                    tl = list()
+
+                result_dict.update({"tenants": tl})
+
+            results.append(result_dict)
 
         results = sorted(results, key=lambda k: k['name'])
 

poem/Poem/api/internal_views/yumrepos.py

@@ -1,11 +1,11 @@
 from Poem.api.views import NotFound
 from Poem.poem_super_admin import models as admin_models
 from django.db import IntegrityError
+from django_tenants.utils import get_public_schema_name
 from rest_framework import status
 from rest_framework.authentication import SessionAuthentication
 from rest_framework.response import Response
 from rest_framework.views import APIView
-from django_tenants.utils import get_public_schema_name
 
 from .utils import error_response
 
@@ -16,10 +16,10 @@ class ListYumRepos(APIView):
     def get(self, request, name=None, tag=None):
         if name and tag:
             try:
-                if tag == 'centos6':
-                    ostag = admin_models.OSTag.objects.get(name='CentOS 6')
-                else:
-                    ostag = admin_models.OSTag.objects.get(name='CentOS 7')
+                ostag = [
+                    t for t in admin_models.OSTag.objects.all() if
+                    t.name.lower().replace(" ", "") == tag
+                ][0]
 
                 repo = admin_models.YumRepo.objects.get(name=name, tag=ostag)
                 result = {
@@ -31,7 +31,7 @@ def get(self, request, name=None, tag=None):
                 }
                 return Response(result)
 
-            except admin_models.YumRepo.DoesNotExist:
+            except (IndexError, admin_models.YumRepo.DoesNotExist):
                 return Response(status=status.HTTP_404_NOT_FOUND)
 
         elif not name and not tag:
@@ -143,13 +143,13 @@ def delete(self, request, name=None, tag=None):
         if request.tenant.schema_name == get_public_schema_name() and \
                 request.user.is_superuser:
             if name and tag:
-                if tag == 'centos6':
-                    ostag = admin_models.OSTag.objects.get(name='CentOS 6')
-
-                elif tag == 'centos7':
-                    ostag = admin_models.OSTag.objects.get(name='CentOS 7')
+                try:
+                    ostag = [
+                        t for t in admin_models.OSTag.objects.all() if
+                        t.name.lower().replace(" ", "") == tag
+                    ][0]
 
-                else:
+                except IndexError:
                     raise NotFound(status=404, detail='OS tag does not exist.')
 
                 try: