Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Have nicer result failure messages #587

Open
carl-baillargeon opened this issue Mar 18, 2024 · 5 comments · Fixed by #1027, #1029 or #1031
Open

Have nicer result failure messages #587

carl-baillargeon opened this issue Mar 18, 2024 · 5 comments · Fixed by #1027, #1029 or #1031
Assignees

Comments

@carl-baillargeon
Copy link
Contributor

Would be nice to have a more human readable format for the test result failures messages:
image

BGP tests are also not very user friendly:
image

Could be related to #427

@carl-baillargeon
Copy link
Contributor Author

Adding VerifySnmpContact to the list:
image

@carl-baillargeon
Copy link
Contributor Author

@vitthalmagadum Will tackle the interfaces, LLDP, hardware tests.
@carl-baillargeon Will tackle the BGP tests.

@carl-baillargeon
Copy link
Contributor Author

Some tests failure messages will be improved as part of #388

@carl-baillargeon
Copy link
Contributor Author

@vitthalmagadum and @geetanjalimanegslab Please investigate remaining tests so we know how big of an effort it would be to remove the [ ] and { } in the failure messages.

Thanks

@geetanjalimanegslab
Copy link
Collaborator

geetanjalimanegslab commented Feb 4, 2025

Hi @carl-baillargeon
Here is the list of test files and testscases :

  1. test_generic.py - PR refactor(anta.tests): Nicer result failure messages generic routing module #1027
    1. VerifyRoutingTableSize
    2. VerifyRoutingTableEntry

  2. test_isis.py - fixed in PR fix(anta.tests): Cleaning up ISIS tests module #963

  3. . test_ospf.py
    1. VerifyOSPFNeighborState
    2. VerifyOSPFNeighborCount
    3. VerifyOSPFMaxLSA

  4. test_aaa.py PR refactor(anta.tests): Nicer result failure messages for AAA test module #1031
    1. VerifyTacacsServers
    2. VerifyTacacsServerGroups
    3. VerifyAuthenMethods
    4. VerifyAuthzMethods
    5. VerifyAcctDefaultMethods
    6. VerifyAcctConsoleMethods

  5. test_avt.py
    1. VerifyAVTSpecificPath
    2. VerifyBFDPeersHealth

  6. test_bfd.py PR - refactor(anta.tests): Nicer result failure messages BFD & Connectivity test module #1029
    1. VerifyBFDPeersRegProtocols

  7. test_connectivity.py PR - refactor(anta.tests): Nicer result failure messages BFD & Connectivity test module #1029
    1. VerifyReachability
    2. VerifyLLDPNeighbors

  8. test_field_notices.py
    1. VerifyFieldNotice44Resolution

  9. test_hardware.py
    1. VerifyTransceiversManufacturers
    2. VerifyTransceiversTemperature
    3. VerifyEnvironmentPower

  10. test_interfaces.py
    1. VerifyInterfaceUtilization
    2. VerifyInterfaceErrors
    3. VerifyInterfaceDiscards
    4. VerifyInterfaceErrDisabled
    5. VerifyStormControlDrops
    6. VerifyPortChannels
    7. VerifyIllegalLACP
    8. VerifyLoopbackCount
    9. VerifyL3MTU
    10. VerifyL2MTU
    11. VerifyIPProxyARP
    12. VerifyInterfacesSpeed

  11. test_logging.py PR refactor(anta.tests): Nicer result failure messages for Logging and security test module #1028
    1. VerifyLoggingHosts

  12. test_mlag.py
    1. VerifyMlagStatus
    2. VerifyMlagInterfaces
    3. VerifyMlagConfigSanity
    4. VerifyMlagReloadDelay
    5. VerifyMlagDualPrimary

  13. test_ptp.py
    1. VerifyPtpOffset
    2. VerifyPtpPortModeStatus

  14. test_security.py PR refactor(anta.tests): Nicer result failure messages for Logging and security test module #1028
    1. VerifySSHIPv4Acl
    2. VerifySSHIPv6Acl
    3. VerifyAPIHttpsSSL
    4. VerifyAPIIPv4Acl
    5. VerifyAPIIPv6Acl

  15. test_services.py
    1. VerifyDNSServers

  16. test_snmp.py
    1. VerifySnmpIPv4Acl
    2. VerifySnmpIPv6Acl
    3. VerifySnmpPDUCounters
    4. VerifySnmpErrorCounters

  17. test_software.py
    1. VerifyEOSVersion
    2. VerifyTerminAttrVersion
    3. VerifyEOSExtensions

  18. test_stp.py
    1. VerifySTPMode
    2. VerifySTPBlockedPorts
    3. VerifySTPCounters
    4. VerifySTPForwardingPorts
    5. VerifySTPRootPriority
    6. VerifyStpTopologyChanges

  19. test_system.py
    1. VerifyCoredump
    2. VerifyNTPAssociations

  20. test_vxlan.py
    1. VerifyVxlanConfigSanity
    2. VerifyVxlanVniBinding
    3. VerifyVxlanVtep

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment