Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Broken tests: cloud_level_testing suite seems to be broken #141

Open
scoopex opened this issue Dec 5, 2024 · 1 comment · Fixed by #142
Open

Broken tests: cloud_level_testing suite seems to be broken #141

scoopex opened this issue Dec 5, 2024 · 1 comment · Fixed by #142

Comments

@scoopex
Copy link
Contributor

scoopex commented Dec 5, 2024 •
edited
Loading

When executing the cloud_level_testing tests, various tests are failing with a up2date OSISM installation.

$  behave cloud_level_testing/features/

2024-12-05 17:53:06,628 - root - INFO - Starting logger in level INFO
@vm @benchmark
Feature: Test VM under load # cloud_level_testing/features/openstack_benchmark_4000pi.feature:3

  Scenario Outline: Test VM response time while under load (calculating 4000 digits of pi) -- @1.1   # cloud_level_testing/features/openstack_benchmark_4000pi.feature:15
    Given I can get the shared context from previous feature                                         # cloud_level_testing/features/steps/definitions.py:899 0.000s
    Given I have deployed JHs                                                                        # cloud_level_testing/features/steps/definitions.py:786 0.000s
      Assertion Failed: redirs is no dictionary, but is <class 'NoneType'>

2024-12-05 17:53:06,657 - root - INFO - Feature 'Test VM under load' failed: performing cleanup or additional actions
2024-12-05 17:53:06,657 - root - INFO - Feature completed: performing additional actions
    And I have a private key at infra-keypair for ubuntu                                             # None
    Then I should be able to retrieve the first floating ip and portnumber of the network            # None
    Then I should be able to SSH into the VM                                                         # None
    Then I start calculating 4000 digits of pi on VM and check the ping response                     # None

@ssh @test
Feature: Test JH connectivity with ping # cloud_level_testing/features/openstack_benchmark_JH_test_connection.feature:3

  Scenario: Test jh connectivity                                  # cloud_level_testing/features/openstack_benchmark_JH_test_connection.feature:5
    Given I can get the shared context from previous feature      # cloud_level_testing/features/steps/definitions.py:899 0.000s
    Given I have deployed jumphosts with floating ips             # cloud_level_testing/features/steps/definitions.py:906 1.050s
      Assertion Failed: No jumphosts with attached floating ip address

2024-12-05 17:53:07,708 - root - INFO - Feature 'Test JH connectivity with ping' failed: performing cleanup or additional actions
2024-12-05 17:53:07,709 - root - INFO - Feature completed: performing additional actions
    Then I should be able to ping the jumphosts with floating ips # None

@benchmark @create @delete
Feature: Build benchmark infrastructure # cloud_level_testing/features/openstack_benchmark_build_infra.feature:4

  Scenario Outline: Build the benchmark infrastructure used for benchmark tests -- @1.1 Build benchmark infrastructure                                                   # cloud_level_testing/features/openstack_benchmark_build_infra.feature:27
    Given I have a private key at infra-keypair for ubuntu                                                                                                               # cloud_level_testing/features/steps/definitions.py:681 0.000s
    Given I want to build the benchmark infrastructure by using resources having the infix infra                                                                         # cloud_level_testing/features/steps/benchmark_infra.py:29 0.000s
    Then I should be able to create a router connected to the external network named ext01                                                                               # cloud_level_testing/features/steps/benchmark_infra.py:61 0.192s
      Assertion Failed: Expecting to find exactly one external network.

2024-12-05 17:53:07,903 - root - INFO - Feature 'Build benchmark infrastructure' failed: performing cleanup or additional actions
2024-12-05 17:53:07,903 - root - INFO - Feature completed: performing additional actions
    Then I should be able to fetch availability zones                                                                                                                    # None
    Then I should be able to create networks for both the jump hosts and for each availability zone                                                                      # None
    Then I should be able to create subnets for both the jump hosts and vms                                                                                              # None
    Then I should be able to connect the router to the jump host subnet                                                                                                  # None
    Then I should be able to connect the router to the vm subnets                                                                                                        # None
    Then I should be able to create default security groups                                                                                                              # None
    Then I should be able to create a security group for the hosts allowing inbound tcp connections for the port range 222 to 229                                        # None
    Then I should be able to create 2 VMs with a key pair named infra-keypair and strip them over the VM networks                                                        # None
    Then I should be able to query the ip addresses of the created 2 VMs                                                                                                 # None
    Then I should be able to calculate the port forwardings for the jump hosts by associating the VM ip addresses with the jump hosts by az in the port range 222 to 229 # None
    Then I should be able to create a jump host for each az using a key pair named infra-keypair                                                                         # None
    Then I should be able to attach floating ips to the jump hosts                                                                                                       # None
    Then I can pass the context to another feature                                                                                                                       # None

@iperf3 @benchmark
Feature: Benchmark Iperf3 VMs # cloud_level_testing/features/openstack_benchmark_iperf3.feature:3

  Scenario Outline: Build the benchmark infrastructure used for benchmark tests and perform iperf3 -- @1.1 Build benchmark infrastructure  # cloud_level_testing/features/openstack_benchmark_iperf3.feature:13
    Given I can get the shared context from previous feature                                                                               # cloud_level_testing/features/steps/definitions.py:899 0.000s
    Given I have a private key at infra-keypair for ubuntu                                                                                 # cloud_level_testing/features/steps/definitions.py:681 0.000s
    Given I have deployed JHs                                                                                                              # cloud_level_testing/features/steps/definitions.py:786 0.000s
      Assertion Failed: redirs is no dictionary, but is <class 'NoneType'>

2024-12-05 17:53:07,905 - root - INFO - Feature 'Benchmark Iperf3 VMs' failed: performing cleanup or additional actions
2024-12-05 17:53:07,905 - root - INFO - Feature completed: performing additional actions
    Then I should be able to SSH into VMs and perform iperf3 test                                                                          # None

@loadbalancer @benchmark
Feature: Benchmark loadbalancer # cloud_level_testing/features/openstack_benchmark_loadbalancer.feature:3
2024-12-05 17:53:07,905 - root - INFO - Feature 'Benchmark loadbalancer' passed
2024-12-05 17:53:07,905 - root - INFO - Feature completed: performing additional actions

  Scenario Outline: Create load balancers and check whether VMs can be reached through their endpoint -- @1.1 Build benchmark loadbalancer  # cloud_level_testing/features/openstack_benchmark_loadbalancer.feature:19

@vm @benchmark
Feature: Benchmark ping VMs # cloud_level_testing/features/openstack_benchmark_pingVM.feature:3

  Scenario Outline: Collecting IPs of VMs can be reached through Openstack, ping them from remote accesses and track retries and failures -- @1.1 Build benchmark infrastructure  # cloud_level_testing/features/openstack_benchmark_pingVM.feature:12
    Given I can get the shared context from previous feature                                                                                                                      # cloud_level_testing/features/steps/definitions.py:899 0.000s
    Given I have deployed JHs                                                                                                                                                     # cloud_level_testing/features/steps/definitions.py:786 0.000s
      Assertion Failed: redirs is no dictionary, but is <class 'NoneType'>

2024-12-05 17:53:07,906 - root - INFO - Feature 'Benchmark ping VMs' failed: performing cleanup or additional actions
2024-12-05 17:53:07,907 - root - INFO - Feature completed: performing additional actions
    And I have a private key at infra-keypair for ubuntu                                                                                                                          # None
    Then I should be able to SSH into JHs and test their ping connectivity                                                                                                        # None

Feature: OpenStack Availability zones creation # cloud_level_testing/features/openstack_create_availability_zone.feature:1

  Scenario Outline: Connect to OpenStack and create an availability zone -- @1.1 Test availability zone  # cloud_level_testing/features/openstack_create_availability_zone.feature:9
    Given I connect to OpenStack                                                                         # cloud_level_testing/features/steps/definitions.py:18 0.003s
    Then I should be able to create an availabilityzone01                                                # None

  Scenario Outline: Connect to OpenStack and create an availability zone -- @1.2 Test availability zone  # cloud_level_testing/features/openstack_create_availability_zone.feature:10
    Given I connect to OpenStack                                                                         # cloud_level_testing/features/steps/definitions.py:18 0.003s
    Then I should be able to create an availabilityzone02                                                # None
2024-12-05 17:53:07,914 - root - INFO - Feature 'OpenStack Availability zones creation' failed: performing cleanup or additional actions
2024-12-05 17:53:07,914 - root - INFO - Feature completed: performing additional actions

Feature: Manage multiple disks in OpenStack # cloud_level_testing/features/openstack_create_disk.feature:1

  Scenario Outline: Create multiple volumes -- @1.1   # cloud_level_testing/features/openstack_create_disk.feature:9
    Given I connect to OpenStack                      # cloud_level_testing/features/steps/definitions.py:18 0.003s
    When I create 1 volumes                           # None
2024-12-05 17:53:07,918 - root - INFO - Feature 'Manage multiple disks in OpenStack' failed: performing cleanup or additional actions
2024-12-05 17:53:07,918 - root - INFO - Feature completed: performing additional actions

@jumphost @create
Feature: OpenStack Jumphost Creation # cloud_level_testing/features/openstack_create_jumphost.feature:3

  Scenario Outline: Connect to OpenStack and create a jumphost -- @1.1                                   # cloud_level_testing/features/openstack_create_jumphost.feature:12
    Given I connect to OpenStack                                                                         # cloud_level_testing/features/steps/definitions.py:18 0.003s
    Then I create a jumphost with name default-jh on network scs-hm-jh-default with keypair test-keypair # cloud_level_testing/features/steps/definitions.py:614 0.001s
      Traceback (most recent call last):
        File "/home/marc/src/github/osba/scs/scs-health-monitor/venv/lib/python3.12/site-packages/behave/model.py", line 1329, in run
          match.run(runner.context)
        File "/home/marc/src/github/osba/scs/scs-health-monitor/venv/lib/python3.12/site-packages/behave/matchers.py", line 98, in run
          self.func(context, *args, **kwargs)
        File "cloud_level_testing/features/steps/definitions.py", line 632, in create_a_jumphost
          keypair_filename = f"{context.keypair_name}-private"
                                ^^^^^^^^^^^^^^^^^^^^
        File "/home/marc/src/github/osba/scs/scs-health-monitor/venv/lib/python3.12/site-packages/behave/runner.py", line 321, in __getattr__
          raise AttributeError(msg)
      AttributeError: 'Context' object has no attribute 'keypair_name'

    Then I attach a floating ip to server default-jh                                                     # None

  Scenario Outline: Connect to OpenStack and create a jumphost -- @1.2                                        # cloud_level_testing/features/openstack_create_jumphost.feature:13
    Given I connect to OpenStack                                                                              # cloud_level_testing/features/steps/definitions.py:18 0.003s
    Then I create a jumphost with name scs-hm-jh-4 on network scs-hm-jh-network-1 with keypair create_keypair # cloud_level_testing/features/steps/definitions.py:614 0.001s
      Traceback (most recent call last):
        File "/home/marc/src/github/osba/scs/scs-health-monitor/venv/lib/python3.12/site-packages/behave/model.py", line 1329, in run
          match.run(runner.context)
        File "/home/marc/src/github/osba/scs/scs-health-monitor/venv/lib/python3.12/site-packages/behave/matchers.py", line 98, in run
          self.func(context, *args, **kwargs)
        File "cloud_level_testing/features/steps/definitions.py", line 632, in create_a_jumphost
          keypair_filename = f"{context.keypair_name}-private"
                                ^^^^^^^^^^^^^^^^^^^^
        File "/home/marc/src/github/osba/scs/scs-health-monitor/venv/lib/python3.12/site-packages/behave/runner.py", line 321, in __getattr__
          raise AttributeError(msg)
      AttributeError: 'Context' object has no attribute 'keypair_name'

2024-12-05 17:53:07,928 - root - INFO - Feature 'OpenStack Jumphost Creation' failed: performing cleanup or additional actions
2024-12-05 17:53:07,928 - root - INFO - Feature completed: performing additional actions
    Then I attach a floating ip to server scs-hm-jh-4                                                         # None

@loadbalancer @create
Feature: OpenStack Network Creation # cloud_level_testing/features/openstack_create_loadbalancer.feature:3

  Scenario Outline: Connect to OpenStack and create a loadbalancer -- @1.1 Test loadbalancers       # cloud_level_testing/features/openstack_create_loadbalancer.feature:12
    Given I connect to OpenStack                                                                    # cloud_level_testing/features/steps/definitions.py:18 0.003s
    When A subnet with name scs-hm-subnet-2 exists in network scs-hm-network-2                      # cloud_level_testing/features/steps/definitions.py:47 1.060s
      Assertion Failed: Network with name scs-hm-network-2 does not exist

2024-12-05 17:53:08,991 - root - INFO - Feature 'OpenStack Network Creation' failed: performing cleanup or additional actions
2024-12-05 17:53:08,991 - root - INFO - Feature completed: performing additional actions
    Then I should be able to create 2 loadbalancers for scs-hm-subnet-2 in network scs-hm-network-2 # None

@network @create
Feature: OpenStack Network Creation # cloud_level_testing/features/openstack_create_network.feature:3

  Scenario Outline: Connect to OpenStack and create a network -- @1.1 Test networks  # cloud_level_testing/features/openstack_create_network.feature:12
    Given I connect to OpenStack                                                     # cloud_level_testing/features/steps/definitions.py:18 0.003s
    Then I should be able to create 2 networks                                       # cloud_level_testing/features/steps/definitions.py:153 3.136s
    Then I should be able to list networks                                           # cloud_level_testing/features/steps/definitions.py:147
2024-12-05 17:53:12,380 - root - INFO - ['062ebc72-41bd-4779-b85b-080b707af1d4', '3d132940-0603-4993-9771-e606bb06cb7e', '3fc950e8-a46b-4c12-9364-06121efb2b98', '6aa4b44c-a259-4955-b1ae-bee8b865976e', '6da101a1-4bae-4955-abf1-68dd77e39462', 'a0485817-7fba-4082-a2ed-3978b3d3c965', 'c597fc59-a8c7-4997-acbc-d8b449c1ba13', 'da22be2c-3e56-    Then I should be able to list networks                                           # cloud_level_testing/features/steps/definitions.py:147 0.248s
2024-12-05 17:53:12,380 - root - INFO - Feature 'OpenStack Network Creation' passed
2024-12-05 17:53:12,380 - root - INFO - Feature completed: performing additional actions

@network @create
Feature: OpenStack create ports for subnets # cloud_level_testing/features/openstack_create_ports_for_networks.feature:3

  Scenario: Connect to OpenStack and create a ports for every subnet in test  # cloud_level_testing/features/openstack_create_ports_for_networks.feature:4
    Given I connect to OpenStack                                              # cloud_level_testing/features/steps/definitions.py:18 0.003s
    Then I should be able to create port for networks                         # cloud_level_testing/features/steps/definitions.py:75 7.703s
2024-12-05 17:53:20,087 - root - INFO - Feature 'OpenStack create ports for subnets' passed
2024-12-05 17:53:20,087 - root - INFO - Feature completed: performing additional actions

@router @create
Feature: OpenStack Router Creation # cloud_level_testing/features/openstack_create_router.feature:3

  Scenario Outline: Connect to OpenStack and create a routers -- @1.1   # cloud_level_testing/features/openstack_create_router.feature:10
    Given I connect to OpenStack                                        # cloud_level_testing/features/steps/definitions.py:18 0.003s
    Then I should be able to create 3 routers                           # cloud_level_testing/features/steps/definitions.py:63 2.108s
2024-12-05 17:53:22,200 - root - INFO - Feature 'OpenStack Router Creation' passed
2024-12-05 17:53:22,200 - root - INFO - Feature completed: performing additional actions

@security_group @create
Feature: OpenStack Security Group Creation # cloud_level_testing/features/openstack_create_security_group.feature:3

  Scenario Outline: Connect to OpenStack and create a security group -- @1.1 Test security groups  # cloud_level_testing/features/openstack_create_security_group.feature:11
    Given I connect to OpenStack                                                                   # cloud_level_testing/features/steps/definitions.py:18 0.003s
    Then I should be able to create 2 security groups                                              # cloud_level_testing/features/steps/definitions.py:335 1.617s
2024-12-05 17:53:23,820 - root - INFO - Feature 'OpenStack Security Group Creation' passed
2024-12-05 17:53:23,821 - root - INFO - Feature completed: performing additional actions

@security_group_rule @create
Feature: OpenStack Security Group Rule Creation # cloud_level_testing/features/openstack_create_security_group_rule.feature:3

  Scenario Outline: Connect to OpenStack and create a security group rule -- @1.1 Test security groups rule           # cloud_level_testing/features/openstack_create_security_group_rule.feature:11
    Given I connect to OpenStack                                                                                      # cloud_level_testing/features/steps/definitions.py:18 0.003s
    Then I should be able to create a security group rule with direction ingress protocol tcp and port range 80 to 80 # None

  Scenario Outline: Connect to OpenStack and create a security group rule -- @1.2 Test security groups rule            # cloud_level_testing/features/openstack_create_security_group_rule.feature:12
    Given I connect to OpenStack                                                                                       # cloud_level_testing/features/steps/definitions.py:18 0.003s
    Then I should be able to create a security group rule with direction egress protocol tcp and port range 430 to 430 # None
2024-12-05 17:53:23,828 - root - INFO - Feature 'OpenStack Security Group Rule Creation' failed: performing cleanup or additional actions
2024-12-05 17:53:23,828 - root - INFO - Feature 'OpenStack Security Group Rule Creation' is a deletion or creation feature: performing cleanup
2024-12-05 17:53:23,830 - root - INFO - Ports to delete: ['7ec3d3cc-861a-45b3-b173-3135c6bf14bd', 'e4ffc928-31ba-4a0d-a2d0-3785fc06a653', '48c6b163-033e-43f1-9a46-848fa44c7bf8', 'ae088cef-ed9f-4f62-9626-c7b81177b37b', 'ac633ff1-7a57-403f-906c-a0917a67e3f7', '627cebd3-e9ae-47b2-9545-7e1dbc3076f2', '621b6046-2cda-4904-9ce8-695d6398e498', '3a304510-ce3e-4be5-ba40-a01082492677']
2024-12-05 17:53:25,440 - root - INFO - Port 7ec3d3cc-861a-45b3-b173-3135c6bf14bd deleted
2024-12-05 17:53:26,186 - root - INFO - Port e4ffc928-31ba-4a0d-a2d0-3785fc06a653 deleted
2024-12-05 17:53:26,959 - root - INFO - Port 48c6b163-033e-43f1-9a46-848fa44c7bf8 deleted
2024-12-05 17:53:27,616 - root - INFO - Port ae088cef-ed9f-4f62-9626-c7b81177b37b deleted
2024-12-05 17:53:28,325 - root - INFO - Port ac633ff1-7a57-403f-906c-a0917a67e3f7 deleted
2024-12-05 17:53:29,069 - root - INFO - Port 627cebd3-e9ae-47b2-9545-7e1dbc3076f2 deleted
2024-12-05 17:53:29,710 - root - INFO - Port 621b6046-2cda-4904-9ce8-695d6398e498 deleted
2024-12-05 17:53:30,459 - root - INFO - Port 3a304510-ce3e-4be5-ba40-a01082492677 deleted
2024-12-05 17:53:30,459 - root - INFO - Networks to delete: ['6da101a1-4bae-4955-abf1-68dd77e39462', 'de5bd717-1aba-429a-9c84-d9b485db194a']
2024-12-05 17:53:31,395 - root - INFO - Network 6da101a1-4bae-4955-abf1-68dd77e39462 deleted
2024-12-05 17:53:32,462 - root - INFO - Network de5bd717-1aba-429a-9c84-d9b485db194a deleted
2024-12-05 17:53:32,462 - root - INFO - Routers to delete: ['6ccd293a-7920-4c7c-9916-dbb3d1525508', '6ea4a842-2d3d-465e-9295-be857adc0541', '9aa09083-2d6e-4218-855c-c09395539050']
2024-12-05 17:53:33,225 - root - INFO - Router 6ccd293a-7920-4c7c-9916-dbb3d1525508 deleted
2024-12-05 17:53:33,937 - root - INFO - Router 6ea4a842-2d3d-465e-9295-be857adc0541 deleted
2024-12-05 17:53:34,606 - root - INFO - Router 9aa09083-2d6e-4218-855c-c09395539050 deleted
2024-12-05 17:53:35,240 - root - INFO - Feature completed: performing additional actions

@subnet @create
Feature: OpenStack Subnet Creation # cloud_level_testing/features/openstack_create_subnet.feature:3

  Scenario Outline: Connect to OpenStack and create a subnet -- @1.1 Test subnets  # cloud_level_testing/features/openstack_create_subnet.feature:11
    Given I connect to OpenStack                                                   # cloud_level_testing/features/steps/definitions.py:18 0.003s
    Then I should be able to create 2 subnets                                      # cloud_level_testing/features/steps/definitions.py:281
2024-12-05 17:53:36,309 - root - INFO - Creating subnets for network: openstack.network.v2.network.Network(id=062ebc72-41bd-4779-b85b-080b707af1d4, name=None-network-1, admin_state_up=True, mtu=8942, status=ACTIVE, subnets=[], shared=False, availability_zone_hints=[], availability_zones=[], ipv4_address_scope=None, ipv6_address_scope=None, router:external=False, description=, qos_policy_id=None, port_security_enabled=True, dns_domain=, tags=[], created_at=2024-12-05T13:07:06Z, updated_at=2024-12-05T16:52:20Z, revision_number=13, project_id=029430400c294e84891240a82a7b2d84, tenant_id=029430400c294e84891240a82a7b2d84, location=Munch({'cloud': 'scs-health-monitor-scs-health-monitor-test-project', 'region_name': '', 'zone': None, 'project': Munch({'id': '029430400c294e84891240a82a7b2d84', 'name': 'scs-health-monitor-test-project', 'domain_id': None, 'domain_name': 'scs-health-monitor'})}))
2024-12-05 17:53:37,985 - root - INFO - Created subnet None-subnet-1
2024-12-05 17:53:40,055 - root - INFO - Created subnet None-subnet-2
2024-12-05 17:53:40,056 - root - INFO - Creating subnets for network: openstack.network.v2.network.Network(id=3d132940-0603-4993-9771-e606bb06cb7e, name=None-network-1, admin_state_up=True, mtu=8942, status=ACTIVE, subnets=[], shared=False, availability_zone_hints=[], availability_zones=[], ipv4_address_scope=None, ipv6_address_scope=None, router:external=False, description=, qos_policy_id=None, port_security_enabled=True, dns_domain=, tags=[], created_at=2024-12-05T13:04:05Z, updated_at=2024-12-05T16:52:21Z, revision_number=13, project_id=029430400c294e84891240a82a7b2d84, tenant_id=029430400c294e84891240a82a7b2d84, location=Munch({'cloud': 'scs-health-monitor-scs-health-monitor-test-project', 'region_name': '', 'zone': None, 'project': Munch({'id': '029430400c294e84891240a82a7b2d84', 'name': 'scs-health-monitor-test-project', 'domain_id': None, 'domain_name': 'scs-health-monitor'})}))
2024-12-05 17:53:41,935 - root - INFO - Created subnet None-subnet-1
2024-12-05 17:53:43,958 - root - INFO - Created subnet None-subnet-2
2024-12-05 17:53:43,959 - root - INFO - Creating subnets for network: openstack.network.v2.network.Network(id=3fc950e8-a46b-4c12-9364-06121efb2b98, name=None-network-1, admin_state_up=True, mtu=8942, status=ACTIVE, subnets=[], shared=False, availability_zone_hints=[], availability_zones=[], ipv4_address_scope=None, ipv6_address_scope=None, router:external=False, description=, qos_policy_id=None, port_security_enabled=True, dns_domain=, tags=[], created_at=2024-12-05T13:05:07Z, updated_at=2024-12-05T16:52:22Z, revision_number=13, project_id=029430400c294e84891240a82a7b2d84, tenant_id=029430400c294e84891240a82a7b2d84, location=Munch({'cloud': 'scs-health-monitor-scs-health-monitor-test-project', 'region_name': '', 'zone': None, 'project': Munch({'id': '029430400c294e84891240a82a7b2d84', 'name': 'scs-health-monitor-test-project', 'domain_id': None, 'domain_name': 'scs-health-monitor'})}))
2024-12-05 17:53:45,773 - root - INFO - Created subnet None-subnet-1
2024-12-05 17:53:48,283 - root - INFO - Created subnet None-subnet-2
2024-12-05 17:53:48,285 - root - INFO - Creating subnets for network: openstack.network.v2.network.Network(id=6aa4b44c-a259-4955-b1ae-bee8b865976e, name=None-network-2, admin_state_up=True, mtu=8942, status=ACTIVE, subnets=[], shared=False, availability_zone_hints=[], availability_zones=[], ipv4_address_scope=None, ipv6_address_scope=None, router:external=False, description=, qos_policy_id=None, port_security_enabled=True, dns_domain=, tags=[], created_at=2024-12-05T13:07:07Z, updated_at=2024-12-05T16:52:22Z, revision_number=13, project_id=029430400c294e84891240a82a7b2d84, tenant_id=029430400c294e84891240a82a7b2d84, location=Munch({'cloud': 'scs-health-monitor-scs-health-monitor-test-project', 'region_name': '', 'zone': None, 'project': Munch({'id': '029430400c294e84891240a82a7b2d84', 'name': 'scs-health-monitor-test-project', 'domain_id': None, 'domain_name': 'scs-health-monitor'})}))
2024-12-05 17:53:50,094 - root - INFO - Created subnet None-subnet-1
2024-12-05 17:53:52,314 - root - INFO - Created subnet None-subnet-2
2024-12-05 17:53:52,316 - root - INFO - Creating subnets for network: openstack.network.v2.network.Network(id=c597fc59-a8c7-4997-acbc-d8b449c1ba13, name=None-network-2, admin_state_up=True, mtu=8942, status=ACTIVE, subnets=[], shared=False, availability_zone_hints=[], availability_zones=[], ipv4_address_scope=None, ipv6_address_scope=None, router:external=False, description=, qos_policy_id=None, port_security_enabled=True, dns_domain=, tags=[], created_at=2024-12-05T13:05:08Z, updated_at=2024-12-05T16:52:23Z, revision_number=13, project_id=029430400c294e84891240a82a7b2d84, tenant_id=029430400c294e84891240a82a7b2d84, location=Munch({'cloud': 'scs-health-monitor-scs-health-monitor-test-project', 'region_name': '', 'zone': None, 'project': Munch({'id': '029430400c294e84891240a82a7b2d84', 'name': 'scs-health-monitor-test-project', 'domain_id': None, 'domain_name': 'scs-health-monitor'})}))
2024-12-05 17:53:54,608 - root - INFO - Created subnet None-subnet-1
2024-12-05 17:53:56,777 - root - INFO - Created subnet None-subnet-2
2024-12-05 17:53:56,779 - root - INFO - Creating subnets for network: openstack.network.v2.network.Network(id=e994d8dd-c255-4221-ba1a-8b8014ee9e37, name=None-network-2, admin_state_up=True, mtu=8942, status=ACTIVE, subnets=[], shared=False, availability_zone_hints=[], availability_zones=[], ipv4_address_scope=None, ipv6_address_scope=None, router:external=False, description=, qos_policy_id=None, port_security_enabled=True, dns_domain=, tags=[], created_at=2024-12-05T13:04:07Z, updated_at=2024-12-05T16:52:24Z, revision_number=13, project_id=029430400c294e84891240a82a7b2d84, tenant_id=029430400c294e84891240a82a7b2d84, location=Munch({'cloud': 'scs-health-monitor-scs-health-monitor-test-project', 'region_name': '', 'zone': None, 'project': Munch({'id': '029430400c294e84891240a82a7b2d84', 'name': 'scs-health-monitor-test-project', 'domain_id': None, 'domain_name': 'scs-health-monitor'})}))
2024-12-05 17:53:58,632 - root - INFO - Created subnet None-subnet-1
2024-12-05 17:54:00,818 - root - INFO - Created subnet None-subnet-2
2024-12-05 17:54:00,819 - root - INFO - Subnets in collector: ['a50ffe4f-f5aa-4324-bb36-ce482d488d7d', '23c4324f-dd65-4098-8bfe-910dbe0bed37', '3a0243eb-d932-4b7f-8290-437657b33096', '7d58ba96-ad45-453c-a9eb-a57d369cf6ad', '074c3a71-c9d9-458d-8506-676dd73674c9', '374f33f8-f5b3-4162-b234-8f86141e7cef', 'dbd1eb7e-081e-4348-8497-73851cf2378e', '03c589d7-5606-4558-9bf4-c9fb62ebbee9', '93e38e02-8606-48ea-914a-547b5e712da9', 'f2388408-cdb2-4cab-881c-f8fda88df606', '6bbb596b-18f9-4d64-a6df-f2eaf9a0fb32', '    Then I should be able to create 2 subnets                                      # cloud_level_testing/features/steps/definitions.py:281 25.574s
2024-12-05 17:54:00,819 - root - INFO - Feature 'OpenStack Subnet Creation' passed
2024-12-05 17:54:00,819 - root - INFO - Feature completed: performing additional actions

@vm @create
Feature: OpenStack Virtual Machine Creation # cloud_level_testing/features/openstack_create_vm.feature:3

  Scenario Outline: Connect to OpenStack and create a virtual machine -- @1.1   # cloud_level_testing/features/openstack_create_vm.feature:12
    Given I connect to OpenStack                                                # cloud_level_testing/features/steps/definitions.py:18 0.003s
    When A security group with name ping-sg exists                              # cloud_level_testing/features/steps/definitions.py:83 0.973s
      Assertion Failed: Security group with name ping-sg doesn't exist

2024-12-05 17:54:01,796 - root - INFO - Feature 'OpenStack Virtual Machine Creation' failed: performing cleanup or additional actions
2024-12-05 17:54:01,797 - root - INFO - Feature 'OpenStack Virtual Machine Creation' is a deletion or creation feature: performing cleanup
2024-12-05 17:54:01,800 - root - INFO - Subnets to delete: ['a50ffe4f-f5aa-4324-bb36-ce482d488d7d', '23c4324f-dd65-4098-8bfe-910dbe0bed37', '3a0243eb-d932-4b7f-8290-437657b33096', '7d58ba96-ad45-453c-a9eb-a57d369cf6ad', '074c3a71-c9d9-458d-8506-676dd73674c9', '374f33f8-f5b3-4162-b234-8f86141e7cef', 'dbd1eb7e-081e-4348-8497-73851cf2378e', '03c589d7-5606-4558-9bf4-c9fb62ebbee9', '93e38e02-8606-48ea-914a-547b5e712da9', 'f2388408-cdb2-4cab-881c-f8fda88df606', '6bbb596b-18f9-4d64-a6df-f2eaf9a0fb32', '5042a84e-6f33-4837-aff0-2f8e2ed55b6c']
2024-12-05 17:54:03,229 - root - INFO - Subnet a50ffe4f-f5aa-4324-bb36-ce482d488d7d deleted
2024-12-05 17:54:03,679 - root - INFO - Subnet 23c4324f-dd65-4098-8bfe-910dbe0bed37 deleted
2024-12-05 17:54:04,253 - root - INFO - Subnet 3a0243eb-d932-4b7f-8290-437657b33096 deleted
2024-12-05 17:54:04,804 - root - INFO - Subnet 7d58ba96-ad45-453c-a9eb-a57d369cf6ad deleted
2024-12-05 17:54:05,236 - root - INFO - Subnet 074c3a71-c9d9-458d-8506-676dd73674c9 deleted
2024-12-05 17:54:05,762 - root - INFO - Subnet 374f33f8-f5b3-4162-b234-8f86141e7cef deleted
2024-12-05 17:54:06,295 - root - INFO - Subnet dbd1eb7e-081e-4348-8497-73851cf2378e deleted
2024-12-05 17:54:06,849 - root - INFO - Subnet 03c589d7-5606-4558-9bf4-c9fb62ebbee9 deleted
2024-12-05 17:54:07,284 - root - INFO - Subnet 93e38e02-8606-48ea-914a-547b5e712da9 deleted
2024-12-05 17:54:07,838 - root - INFO - Subnet f2388408-cdb2-4cab-881c-f8fda88df606 deleted
2024-12-05 17:54:08,340 - root - INFO - Subnet 6bbb596b-18f9-4d64-a6df-f2eaf9a0fb32 deleted
2024-12-05 17:54:08,879 - root - INFO - Subnet 5042a84e-6f33-4837-aff0-2f8e2ed55b6c deleted
2024-12-05 17:54:09,211 - root - INFO - Feature completed: performing additional actions
    Then I should be able to create 1 VMs                                       # None

Feature: Create disk from image in OpenStack # cloud_level_testing/features/openstack_create_volume_from_image.feature:1

  Scenario Outline: Create a disk from an image -- @1.1   # cloud_level_testing/features/openstack_create_volume_from_image.feature:9
    Given I connect to OpenStack                          # cloud_level_testing/features/steps/definitions.py:18 0.004s
    Then I create 2 volumes                               # cloud_level_testing/features/steps/definitions.py:572 1.350s
      Traceback (most recent call last):
        File "/home/marc/src/github/osba/scs/scs-health-monitor/venv/lib/python3.12/site-packages/behave/model.py", line 1329, in run
          match.run(runner.context)
        File "/home/marc/src/github/osba/scs/scs-health-monitor/venv/lib/python3.12/site-packages/behave/matchers.py", line 98, in run
          self.func(context, *args, **kwargs)
        File "cloud_level_testing/features/steps/definitions.py", line 580, in create_multiple_volumes
          tools.ensure_volume_exist(
        File "/home/marc/src/github/osba/scs/scs-health-monitor/cloud_level_testing/features/steps/tools.py", line 499, in ensure_volume_exist
          if check_volumes_created(client=client, test_name=test_name) == "available":
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/marc/src/github/osba/scs/scs-health-monitor/cloud_level_testing/features/steps/tools.py", line 579, in check_volumes_created
          if test_name in volume.name:
             ^^^^^^^^^^^^^^^^^^^^^^^^
      TypeError: 'in <string>' requires string as left operand, not NoneType

2024-12-05 17:54:10,566 - root - INFO - Feature 'Create disk from image in OpenStack' failed: performing cleanup or additional actions
2024-12-05 17:54:10,566 - root - INFO - Feature completed: performing additional actions

Feature: OpenStack Availability zones creation # cloud_level_testing/features/openstack_delete_availability_zone.feature:1

  Scenario Outline: Connect to OpenStack and delete availability zone -- @1.1 Test availability zone  # cloud_level_testing/features/openstack_delete_availability_zone.feature:9
    Given I connect to OpenStack                                                                      # cloud_level_testing/features/steps/definitions.py:18 0.003s
    Then I should be able to delete an availabilityzone01                                             # None

  Scenario Outline: Connect to OpenStack and delete availability zone -- @1.2 Test availability zone  # cloud_level_testing/features/openstack_delete_availability_zone.feature:10
    Given I connect to OpenStack                                                                      # cloud_level_testing/features/steps/definitions.py:18 0.003s
    Then I should be able to delete an availabilityzone02                                             # None
2024-12-05 17:54:10,573 - root - INFO - Feature 'OpenStack Availability zones creation' failed: performing cleanup or additional actions
2024-12-05 17:54:10,574 - root - INFO - Feature completed: performing additional actions

Feature: Manage multiple disks in OpenStack # cloud_level_testing/features/openstack_delete_disk.feature:1

  Scenario: Delete all "volume_test" volumes         # cloud_level_testing/features/openstack_delete_disk.feature:3
    Given I am connected to OpenStack                # None
2024-12-05 17:54:10,574 - root - INFO - Feature 'Manage multiple disks in OpenStack' failed: performing cleanup or additional actions
2024-12-05 17:54:10,574 - root - INFO - Feature completed: performing additional actions
    And volumes named "volume_test" exist            # None
    When I delete all volumes named "volume_test"    # None
    Then no volumes named "volume_test" should exist # None

@network @create
Feature: OpenStack floating ip deletion # cloud_level_testing/features/openstack_delete_floating_ip.feature:3
Exception TypeError: 'NoneType' object is not iterable
Traceback (most recent call last):
  File "/home/marc/src/github/osba/scs/scs-health-monitor/venv/bin/behave", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/marc/src/github/osba/scs/scs-health-monitor/venv/lib/python3.12/site-packages/behave/__main__.py", line 183, in main
    return run_behave(config)
           ^^^^^^^^^^^^^^^^^^
  File "/home/marc/src/github/osba/scs/scs-health-monitor/venv/lib/python3.12/site-packages/behave/__main__.py", line 127, in run_behave
    failed = runner.run()
             ^^^^^^^^^^^^
  File "/home/marc/src/github/osba/scs/scs-health-monitor/venv/lib/python3.12/site-packages/behave/runner.py", line 804, in run
    return self.run_with_paths()
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/marc/src/github/osba/scs/scs-health-monitor/venv/lib/python3.12/site-packages/behave/runner.py", line 824, in run_with_paths
    return self.run_model()
           ^^^^^^^^^^^^^^^^
  File "/home/marc/src/github/osba/scs/scs-health-monitor/venv/lib/python3.12/site-packages/behave/runner.py", line 626, in run_model
    failed = feature.run(self)
             ^^^^^^^^^^^^^^^^^
  File "/home/marc/src/github/osba/scs/scs-health-monitor/venv/lib/python3.12/site-packages/behave/model.py", line 321, in run
    failed = scenario.run(runner)
             ^^^^^^^^^^^^^^^^^^^^
  File "/home/marc/src/github/osba/scs/scs-health-monitor/venv/lib/python3.12/site-packages/behave/model.py", line 1112, in run
    for scenario in self.scenarios:     # -- REQUIRE: BUILD-SCENARIOS
                    ^^^^^^^^^^^^^^
  File "/home/marc/src/github/osba/scs/scs-health-monitor/venv/lib/python3.12/site-packages/behave/model.py", line 1017, in scenarios
    self._scenarios = builder.build_scenarios(self)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/marc/src/github/osba/scs/scs-health-monitor/venv/lib/python3.12/site-packages/behave/model.py", line 889, in build_scenarios
    for row_index, row in enumerate(example.table):
                          ^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: 'NoneType' object is not iterable
@scoopex scoopex changed the title Broken tests: cloud_level_testing Broken tests: cloud_level_testing suite seems to be broken Dec 5, 2024
@scoopex scoopex mentioned this issue Dec 6, 2024
Merged
@scoopex
Copy link
Contributor Author

scoopex commented Dec 6, 2024

Fixed some tests with #142, test runs are still broken.

  • cloud_level_testing/features/openstack_benchmark_4000pi.feature:15 Test VM response time while under load (calculating 4000 digits of pi) -- @1.1
  • cloud_level_testing/features/openstack_benchmark_JH_test_connection.feature:5 Test jh connectivity
  • cloud_level_testing/features/openstack_benchmark_iperf3.feature:13 Build the benchmark infrastructure used for benchmark tests and perform iperf3 -- @1.1 Build benchmark infrastructure
  • cloud_level_testing/features/openstack_benchmark_pingVM.feature:12 Collecting IPs of VMs can be reached through Openstack, ping them from remote accesses and track retries and failures -- @1.1 Build benchmark infrastructure

@scoopex scoopex reopened this Dec 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Bugfix tests SovereignCloudStack/scs-health-monitor
1 participant
@scoopex