Skip to content

Commit 5c15ba1

Browse files
siqbal1986siqbal1986
committed
Squashed commit of the following:
commit 0b6042544e8dcccdcd79a25c7748fd11b9bc27ad Author: siqbal1486 <[email protected]> Date: Wed Aug 10 15:26:43 2022 -0700 changed suggested in review. cleanup commit 0cc1d72b7e0c5da97815fc0a69d12d2a0c2171a9 Merge: f6f02f03 6850440d Author: siqbal1486 <[email protected]> Date: Wed Aug 10 14:31:28 2022 -0700 Merge branch 'bfd_test_multihop' of https://github.com/siqbal1986/sonic-mgmt into bfd_test_multihop commit 6850440d5f90a1e2ae0d78c2f2f42f9fc39b3c95 Merge: 5924f75c 93323578 Author: siqbal1986 <[email protected]> Date: Wed Aug 10 14:30:19 2022 -0700 Merge branch 'master' into bfd_test_multihop commit f6f02f036b767dc5012b55bee92f4f3944470083 Merge: f78da62f 5924f75c Author: siqbal1486 <[email protected]> Date: Wed Aug 10 13:06:42 2022 -0700 Merge branch 'bfd_test_multihop' of https://github.com/siqbal1986/sonic-mgmt into bfd_test_multihop commit 9332357850282dd61dba5bbfbe68463dd088e91d Author: Jibin Bao <[email protected]> Date: Thu Aug 11 00:09:18 2022 +0800 Add test plan for syslog source ip feature (#5943) commit fb51ba2b092ea48d9233e9f3efcc1811afef2668 Author: Nana@Nvidia <[email protected]> Date: Thu Aug 11 00:06:43 2022 +0800 [Qos]TestQosSai should not be skipped on ptf32, ptf64 topo (#6112) - What is the motivation for this PR? For mellanox asic, TestQosSai support to run on ptf32,ptf64 topos, it should be skip on these topos - How did you do it? Add support for ptf32,ptf64 in tests/common/plugins/conditional_mark/tests_mark_conditions.yaml - How did you verify/test it? Run the TestQosSai on ptf topo, and it is not skipped. Change-Id: I6d37aca287e8e797ae43de903920fb61c2e1ae9c commit 22ac478a87f1c81643e6733cb3090e8d5f696d9e Author: Ashwin Srinivasan <[email protected]> Date: Wed Aug 10 08:56:03 2022 -0700 Removed the superfluous pdb trace command from the get_healthy_psu_num function in test_platform_info (#6135) commit 22fb68f8ade261eabaf323ba85ec63028d324d75 Author: Cong Hou <[email protected]> Date: Wed Aug 10 23:16:45 2022 +0800 [sub-interface] use OrderedDict instead of built-in dict for ptf and dut ports in get_port() function of sub-interface test (#6125) The function get_port() in tests/sub_port_interfaces/sub_ports_helpers.py is using built-in dictionary to store the dut ports and ptf ports selected for the subinterface test. However, because there's no order in the built-in dict, sometimes the dut port could be paired with a wrong ptf port, which will cause the test to fail. In the function get_ports() the dut ports is returned in dict and the ptf ports is returned in list of the dict values, and they are zipped in the caller to do iteration. It is not guaranteed that when zipping, the dut port is paired with the correct ptf port. For example in tests/sub_port_interfaces/conftest.py So need to use OrderedDict instead of built-in dictitonary to store the selected dut ports and ptf port in get_ports(). commit f3748cfef4bca1604037ab116586c7b33a2c8b81 Author: Yutong Zhang <[email protected]> Date: Wed Aug 10 20:49:59 2022 +0800 [bugfix] skip vlan/test_vlan_ping.py (#6137) Description of PR In pr #5708 , we skip the test cases in tests/common/plugins/conditional_mark/tests_mark_conditions.yaml. There is a merge conflict and forget to skip vlan/test_vlan_ping.py when the asic_type is broadcom. In this pr, skip this module. What is the motivation for this PR? In pr #5708 , we skip the test cases in tests/common/plugins/conditional_mark/tests_mark_conditions.yaml. There is a merge conflict and forget to skip vlan/test_vlan_ping.py when the asic_type is broadcom. In this pr, skip this module. How did you do it? Add the condition to skip vlan/test_vlan_ping.py. Signed-off-by: Yutong Zhang <[email protected]> commit b1f80d1cc63b092f280e6ed3d6e80da251b1fbed Author: Kostiantyn Yarovyi <[email protected]> Date: Wed Aug 10 13:03:29 2022 +0200 add sleep after remove vrf (#6133) What is the motivation for this PR? vrf does not have enough time to remove before a creation. Therefore a test TestVrfDeletion::test_vrf1_neigh_after_restore failed How did you do it? add sleep How did you verify/test it? run vrf/test_vrf.py::TestVrfDeletion::test_vrf1_neigh_after_restore commit 5b9a30c112c08b611c9212efdc30262c31ce7cd1 Author: Yutong Zhang <[email protected]> Date: Wed Aug 10 16:33:16 2022 +0800 Restore tacacs_server after the module tacacs/test_accounting.py running. (#6117) Description of PR In module tacacs/test_accounting.py, the fixture check_tacacs use the function setup_tacacs_client to delete the default tacacs server, and set the ptf mgmt ip as tacacs sever ip. But it doesn't restore this config when the module finish running. We want to keep the config in consistent before and after the testcase running, so fix it. What is the motivation for this PR? In module tacacs/test_accounting.py, the fixture check_tacacs use the function setup_tacacs_client to delete the default tacacs server, and set the ptf mgmt ip as tacacs sever ip. But it doesn't restore this config when the module finish running. We want to keep the config in consistent before and after the testcase running, so fix it. How did you do it? Get the default tacacs server and put them into a list, when the module finish running, delete the ptf mgmt ipand restore the default tacacs server ip. How did you verify/test it? Running the test cases in this module and compare the tacacs server ip before and after running. Signed-off-by: Yutong Zhang <[email protected]> commit a60e5a6e1d32ea7fa104046d36aae8d1ba707dd9 Author: Zhaohui Sun <[email protected]> Date: Wed Aug 10 15:02:15 2022 +0800 Add StartTimestamp column in TestReportPipeline table (#6132) What is the motivation for this PR? Add StartTimestamp column in TestReportPipeline table How did you do it? Use another API to get the start time of pipeline and upload it to Kusto. How did you verify/test it? python3 collect_azp_results.py 8888 python3 report_uploader.py -c "test_result" -e "vms-t0-kvm.201911#132728" -t "vms-t0-kvm" -i "http://****/sonic-broadcom.bin" results SonicTestData Signed-off-by: Zhaohui Sun <[email protected]> commit 5514acb1a07a61284ce481d788274ad5d2a9ac18 Author: Ihor Chekh <[email protected]> Date: Wed Aug 10 00:42:20 2022 +0300 BFD test fixes and improvements (#6082) *Single hop BFD test fixes and improvements commit f78da62f1afc481fd5b38dc716fce18163a88625 Merge: 3ea15a68 c309ff26 Author: siqbal1986 <[email protected]> Date: Tue Aug 9 13:01:16 2022 -0700 Merge branch 'master' into bfd_test_multihop commit 5924f75c2db3dbad70c6373f989fdb20b74345d4 Merge: 3ea15a68 c309ff26 Author: siqbal1986 <[email protected]> Date: Tue Aug 9 13:01:16 2022 -0700 Merge branch 'master' into bfd_test_multihop commit c309ff26b1a4dd1783782f68aeffe464ca68f463 Author: Ye Jianquan <[email protected]> Date: Tue Aug 9 17:25:36 2022 +0800 [TestbedV2]Convert t1-lag pr test to TestbedV2 (#6127) Convert t1-lag pr test to TestbedV2 Approach What is the motivation for this PR? Convert the t1-lag pr test to TestbedV2, to reduce test time by distributing test cases on multi-instances. Currently, the preparation of the testbed(add-topo, deploy-mg) is operated implicitly, before the testbed is ready, the progress of the test plan keeps 0. We will refine the progress indicator in a future release. The conversion can be dynamically reverted by modifying an AZP library variable: Testbed-Tools/RUN_TEST_BY_SCHEDULER : YES/NO How did you do it? Modify the pipeline yaml file. After converting to TestbedV2, the AZP only create the test plan and poll the result of the test plan. How did you verify/test it? The pass result of this pr is the test result of this pr. Signed-off-by: Jianquan Ye<[email protected]> commit c3f124f34a37e1fca93aab2e912b418cbb084841 Author: ppikh <[email protected]> Date: Tue Aug 9 06:20:48 2022 +0300 Fixed dut_basic_facts ansible module to have support SONiC images which does not have attribute "is_supervisor" (#6118) Description of PR Fixed dut_basic_facts ansible module to have support SONiC images which does not have attribute "is_supervisor" Previoulsy when we call dut_basic_facts ansible module on SONiC image which does not have attribute "is_supervisor"(for example: 202012) we received error: AttributeError("'module' object has no attribute 'is_supervisor'" Now issue fixed - script will work on all SONiC branches Issue introduced in PR: #5708 Summary: Fixed dut_basic_facts ansible module to have support SONiC images which does not have attribute "is_supervisor" What is the motivation for this PR? Fix AttributeError("'module' object has no attribute 'is_supervisor'" How did you do it? See code How did you verify/test it? Executed ansible module: dut_basic_facts Signed-off-by: Petro Pikh <[email protected]> commit 5ee2f0cd3237114bd4bc0b0cc910dac2788f8123 Author: Ze Gan <[email protected]> Date: Tue Aug 9 10:11:39 2022 +0800 Revert "[kvmtest.sh]: Ignore test_t0_sonic temporarily (#6104)" (#6119) This reverts commit d3bc674964cf4244994bb204b37e0c19e140ca10. commit f46f36171819265a3514c48509e8e8b685593ae2 Author: Yutong Zhang <[email protected]> Date: Tue Aug 9 07:57:36 2022 +0800 [bugfix] Fix an error in tests_mark_conditions.yaml (#6113) Description of PR There is a condition error in tests_mark_conditions.yaml, fix it. What is the motivation for this PR? There is a condition error in tests_mark_conditions.yaml, fix it. Signed-off-by: Yutong Zhang <[email protected]> commit bf02aeae1d6c8e7d4523e21ec5fdf8e04688b689 Author: Lawrence Lee <[email protected]> Date: Mon Aug 8 14:51:06 2022 -0700 [dualtor]: Resolve neighbor after neighbor removal (#6071) - After restarting arp_responder during the test, also restart arp_update process on the DUT to resolve failed neighbor entries - Improve test case cleanup Signed-off-by: Lawrence Lee <[email protected]> commit 41d7b15524017f9a267ff010846d5a1f9681b307 Author: Nana@Nvidia <[email protected]> Date: Mon Aug 8 20:34:45 2022 +0800 Add rif loopback action test plan (#5956) Add test plan for the RIF interface loopback action feature. The HLD for the RIF interface loopback action: https://github.com/sonic-net/SONiC/blob/master/doc/ip-interface/loopback-action/ip-interface-loopback-action-design.md commit 24dad8f4b7036376b873c4a0a71a5b7d8a649be8 Author: Yutong Zhang <[email protected]> Date: Mon Aug 8 11:01:03 2022 +0800 Remove "BGP_BBR" in config after test case test_bbr_disabled_dut_asn_in_aspath running. (#6102) Description of PR During the test case test_bbr_disabled_dut_asn_in_aspath running, it changes the status of "BGP_BBR" in config db. But before running, there is no "BGP_BBR" in config db. This cause the inconsistent in config db before and after the test case running. In this pr, we delete the key "BGP_BBR" in config db after the test case running. What is the motivation for this PR? During the test case test_bbr_disabled_dut_asn_in_aspath running, it changes the status of "BGP_BBR" in config db. But before running, there is no "BGP_BBR" in config db. This cause the inconsistent in config db before and after the test case running. In this pr, we delete the key "BGP_BBR" in config db after the test case running. How did you do it? Use configlet to delete the config after test case running. How did you verify/test it? Check the config db before and after the test case running. Signed-off-by: Yutong Zhang <[email protected]> commit e011ed0ac2ea7e062bf4f6d55177ecfd4e907569 Author: Jing Zhang <[email protected]> Date: Sun Aug 7 18:42:01 2022 -0700 Enable `test_normal_op` test cases on active-active dualtor interfaces (#5984) Approach What is the motivation for this PR? To enable dualtor io tests on active-active dualtor ports. How did you do it? 1. Added control utilities in nic_simulator_control, for toggling active-active interfaces to standby/active states on any or both duthosts. Toggles is triggered through cmd line, which is different to active-standby ports. 2. Added active-active type in cable_type fixture. 3. Update test_normal_op cases to adapt active-active interfaces. For some cases, disruption is not expected any more. 4. Adjust test names, commets, to better suit today's usage. How did you verify/test it? Run test cases on mixed topology. commit d3bc674964cf4244994bb204b37e0c19e140ca10 Author: Ze Gan <[email protected]> Date: Sun Aug 7 18:41:38 2022 +0800 [kvmtest.sh]: Ignore test_t0_sonic temporarily (#6104) What is the motivation for this PR? There is a bug in vsonic as neighbor devices, ignore t0_sonic temporarily and added it back if the bug is fixed. How did you do it? add || ture in ./run_tests.sh to ignore the test result. Signed-off-by: Ze Gan <[email protected]> commit 35a1f1e5b0fc788648dab9405ec2a478732ed99e Author: Xin Wang <[email protected]> Date: Sat Aug 6 15:40:18 2022 +0800 Fix cEOS duplicated mac address issue on Ubuntu 22.04 (#6090) What is the motivation for this PR? If deploy a topology using cEOS, one of the steps is to create veth interfaces for the cEOS docker containers. For example, the current steps to create backplane interfaces: 1.1 Create veth pair in host for container VM0100 ip link add VM0100-back type veth peer name eth5 1.2 Add the eth5 interface to network namespace of container VM0100 2.1 Create veth pair in host for another container VM0101 ip link add VM0101-back type veth peer name eth5 2.2 Add the eth5 interface to network namespace of container VM0101 As we can see that after step 1.2, eth5 is no longer in the host namespace. Then in step 2.1 we can add another interface with same name eth5. The problem is that on Ubuntu 22.04, mac address of eth5 created in step 2.1 will be the same as the eth5 interface created in step 1.1. Possibly Ubuntu 22.04 is using a different algorithm for assigning mac address to new veth interfaces. If interface name is same, then mac address will be same too. Because all the VMxxxx-back interfaces will be attached to a same ovs bridge, their peer interfaces should not use same mac address. How did you do it? The fix is to create veth interfaces with unique name in host for all cEOS containers in the beginning. Then all the interfaces in different cEOS have unique mac address. How did you verify/test it? Tested using 'testbed-cli.sh remove-topo' and 'testbed-cli.sh add-topo' Signed-off-by: Xin Wang <[email protected]> commit a3268ac644162c6feff44163f023a0df41ad337a Author: jingwenxie <[email protected]> Date: Sat Aug 6 05:26:55 2022 +0800 [tests/configlet] Remove ignore path in addrack test (#6088) Summary: Remove the ignore path that were blocked by YANG before. ### Approach #### What is the motivation for this PR? The ignore_path should be removed in apply-patch operation. #### How did you do it? Remove ignore_path. commit 180641d9fbcbe21af61b6c75364e7d799454649e Author: Vaibhav Hemant Dixit <[email protected]> Date: Fri Aug 5 11:31:08 2022 -0700 [decap] Bug fix: add missing import for util function in test_decap (#6072) Bug fix: add missing import for util function in test_decap test_decap is failing on master branch due to a bug introduced by #5834 The changes were tested on 202012, but not on master where the json import did not exist. commit 8ac482562757308f2bc24608d423e3fddc477c06 Author: ShiyanWangMS <[email protected]> Date: Thu Aug 4 21:58:59 2022 -0700 Improve debug capability for testcase [test_ecn_during_decap_on_active] (#6091) What is the motivation for this PR? The testcase(test_ecn_during_decap_on_active) results are not stable. Sometime it will fail due to not receiving expected packets. And there is no useful debug information in log file. How did you do it? Add "portstat -c" before sending packets and add "portstat -j" after sending packets. Add "show arp" to quickly identify which is the RX/TX port. How did you verify/test it? Manually run the testcase without Python error. commit 4cac24854714eb8c52a682311bddbb454f3874ee Author: Ashwin Srinivasan <[email protected]> Date: Thu Aug 4 09:59:14 2022 -0700 Adds a function to get the number of healthy PSUs in a device (#6060) commit 98752da2c0410dd31696d400651c516facce62c2 Author: Yutong Zhang <[email protected]> Date: Thu Aug 4 14:21:59 2022 +0800 Replace `pytest.skip` in test scripts with conditional marks. (#5708) Description of PR When we use pytest.skip in test scripts, it will first execute some fixtures in test cases, which will waste some time. When using conditional marks to skip test cases, it will skip the case in the collect period, which will not execute fixtures in test cases and save some execute time. In this pr, we replace pytest.skip in test scripts with conditional marks to skip test case in advance and save execute time. What is the motivation for this PR? When we use pytest.skip in test scripts, it will first execute some fixtures in test cases, which will waste some time. When using conditional marks to skip test cases, it will skip the case in the collect period, which will not execute fixtures in test cases and save some running time. In this pr, we replace pytest.skip in test scripts with conditional marks to skip test case in advance and save running time. How did you do it? Replace pytest.skip in test scripts with conditional marks in tests_mark_conditions.yaml. How did you verify/test it? By running whole test cases and observe the running time. Signed-off-by: Yutong Zhang <[email protected]> commit 5eb6d94ad0ce3cb25e82b4f47f21044ddc83ce41 Author: Stephen Sun <[email protected]> Date: Thu Aug 4 10:51:01 2022 +0800 Fix issue: there should be one DSCP mapped to queue 2/6 in non dual-ToR scenarios (#6089) Signed-off-by: Stephen Sun <[email protected]> commit 709d503a8f73ec6d31d9173d7c7701ae0f520ed9 Author: Sudharsan Dhamal Gopalarathnam <[email protected]> Date: Wed Aug 3 09:57:34 2022 -0700 [kvm]Avoid running ebtables test in KVM (#6073) *Avoid running ebtables test in KVM sonic-net/sonic-buildimage#11585 ebtables shouldn't be installed in KVM which blocks L2 forwarding. So removed the logic to install ebtables rules in SONiC. Hence removing the ebtables tests to be executed on KVM. commit c8ff3f0d69f558fee6b48c0d6510b5e7693ec8c2 Author: rraghav-cisco <[email protected]> Date: Wed Aug 3 09:07:51 2022 -0700 Adding cisco-8000 to the list of platforms for forward action. (#6068) For cisco-8000 platforms, set forward action on Rx in presence of pfc-wd Change is made after: #5665 commit 96a460309c8c98038d075941b6f050cfb708c025 Author: Neetha John <[email protected]> Date: Wed Aug 3 09:06:56 2022 -0700 [swap_syncd] Avoid bgp idle check since bgp docker is already down (#6083) Signed-off-by: Neetha John <[email protected]> What is the motivation for this PR? With the latest image, qos sai tests are failing during test setup with the following error. This is due to the changes introduced in sonic-net/sonic-buildimage#11000. Since swss docker is already stopped prior to this check, bgp docker is stopped and hence the show commands will no longer work 02/08/2022 13:14:58 utilities.wait_until L0113 ERROR | Exception caught while checking ready_for_swap:Traceback (most recent call last): File "/azp/agent/_work/26/s/sonic-mgmt-int/tests/common/utilities.py", line 107, in wait_until check_result = condition(*args, **kwargs) File "/azp/agent/_work/26/s/sonic-mgmt-int/tests/common/system_utils/docker.py", line 186, in ready_for_swap not duthost.is_bgp_state_idle() File "/azp/agent/_work/26/s/sonic-mgmt-int/tests/common/devices/multi_asic.py", line 304, in is_bgp_state_idle return self.sonichost.is_bgp_state_idle() File "/azp/agent/_work/26/s/sonic-mgmt-int/tests/common/devices/sonic.py", line 1657, in is_bgp_state_idle bgp_summary = self.command("show ip bgp summary")["stdout_lines"] File "/azp/agent/_work/26/s/sonic-mgmt-int/tests/common/devices/base.py", line 89, in _run raise RunAnsibleModuleFail("run module {} failed".format(self.module_name), res) RunAnsibleModuleFail: run module command failed, Ansible Results => { "changed": true, "cmd": [ "show", "ip", "bgp", "summary" ], "delta": "0:00:00.881363", "end": "2022-08-02 13:14:58.004283", "failed": true, "invocation": { "module_args": { "_raw_params": "show ip bgp summary", "_uses_shell": false, "argv": null, "chdir": null, "creates": null, "executable": null, "removes": null, "stdin": null, "stdin_add_newline": true, "strip_empty_ends": true, "warn": true } }, "msg": "non-zero return code", "rc": 2, "start": "2022-08-02 13:14:57.122920", "stderr": "Usage: show ip [OPTIONS] COMMAND [ARGS]...\nTry "show ip -h" for help.\n\nError: No such command "bgp".", "stderr_lines": [ "Usage: show ip [OPTIONS] COMMAND [ARGS]...", "Try "show ip -h" for help.", "", "Error: No such command "bgp"." ], "stdout": "", "stdout_lines": [] } How did you do it? Remove the bgp idle state check How did you verify/test it? Ran the qos sai testcase with the changes and it passed commit d6e332386641a3e5344220e7b06316c7acc40722 Author: jingwenxie <[email protected]> Date: Wed Aug 3 15:42:36 2022 +0800 [tests/override_config_table] Add empty table removal test (#6087) Summary: Add E2E test for empty table removal in Golden Config What is the motivation for this PR? We should have an agreement on how Golden Config removes initial table config. This test is to verify the empty table removal in the E2E test. How did you do it? Add E2E test for empty table removal in Golden Config. How did you verify/test it? kvm test. commit a3e2a8e280da76b33486b3bc4d7c89400bbdc373 Author: jingwenxie <[email protected]> Date: Wed Aug 3 15:41:50 2022 +0800 [GCU] Change to identical DUT for cacl test (#6086) Summary: Resolve dualTor cacl test failure. What is the motivation for this PR? Change to identical DUT for cacl test2 How did you do it? Change fixture from duthost to rand_selected_dut commit 185d46f90e2b829e9304a709b4ba6caa294040dd Author: bingwang-ms <[email protected]> Date: Wed Aug 3 15:13:59 2022 +0800 Support 4 lossless queues in `test_buffer_deployment` (#5921) * Support 4 lossless queues in test_buffer_deployment commit 7d1fb7a2b7d3fcd24a150d59aef4f02133f237c5 Author: Ze Gan <[email protected]> Date: Wed Aug 3 15:08:40 2022 +0800 [loganalyzer]: Add ignore log for wpa_supplicant (#6075) What is the motivation for this PR? wpa_supplicant may get the following log ERR macsec#wpa_supplicant[15]: KaY: Life time has not elapsed since prior SAK distributed when the rekey action. But the mka session was finally be recovered otherwise the testcases will fail instead of log checker. How did you do it? Add r, ".* ERR macsec#wpa_supplicant.*KaY: Life time has not elapsed since prior SAK distributed.*" in loganalyzer_common_ignore.txt Signed-off-by: Ze Gan <[email protected]> commit 6658606edff2694f4832fe06c69ad8ac5e1452d8 Author: Zhaohui Sun <[email protected]> Date: Mon Aug 1 16:42:13 2022 +0800 Add test_lag_db_status and test_lag_db_status_with_po_update back (#6074) What is the motivation for this PR? Add test_lag_db_status and test_lag_db_status_with_po_update back which were reverted in #6058. How did you do it? Enhance these two cases to support 202012 kvm testbed, use wait_until instead of checking interface status immediately after shutdown or no shutdown. How did you verify/test it? Verified test_lag_db_status and test_lag_db_status_with_po_update on these testbeds: * kvm testbed with master image * kvm testbed with 202012 image * T0 * T1-lag * Dualtor Signed-off-by: Zhaohui Sun <[email protected]> commit 3ea15a685ed58cc93b78e8d9098ff687cd41c8fc Merge: 17720c67 fa259b22 Author: siqbal1986 <[email protected]> Date: Sun Jul 31 15:37:26 2022 -0700 Merge branch 'master' into bfd_test_multihop commit 17720c671dd125b6ef3237b28a893648b582afe1 Author: Shahzad Iqbal (SHAHZADIQBAL) <[email protected]> Date: Sun Jul 31 15:21:47 2022 -0700 minor changes suggested in review. commit fa259b22248a957c20ff38abe5c1c82718389fe9 Author: Zhaohui Sun <[email protected]> Date: Fri Jul 29 04:05:16 2022 +0800 Revert 2 test cases in test_lag_2 and reduce t1-lag running time (#6058) What is the motivation for this PR? test_lag_db_status_with_po_update can ran success on master image. But failed for image 202012. Because for 202012, there is no netdev_oper_status key in PORT_TABLE ofSTATE_DB, the test case chooses oper_status in APPL_DB , but sync time seems to be different, after shutdown, it checks the status of oper_status immediately, it should wait for a while until the status is correct. Revert these 2 cases firstly, will submit a new PR to add them after enough verification on 202012 and master image. How did you do it? Two changes in this PR: Revert test_lag_db_status and test_lag_db_status_with_po_update. Use --completeness_level=confident to reduce running time, it will pick up 4 ports, not all of them for t1-lag. How did you verify/test it? Run pc/test_lag_2.py Signed-off-by: Zhaohui Sun <[email protected]> commit 5153efb5b98cad7e21f61bcd01cc7b4cfcc84582 Author: Xin Wang <[email protected]> Date: Thu Jul 28 16:11:29 2022 +0800 Disable CI testing keep only PR testing (#6061) What is the motivation for this PR? Currently the pipeline has both PR and CI testing enabled. After a PR is merged, the pipeline is triggered again. This is a waste of resource. How did you do it? This change disabled CI testing. The PR testing will be still triggered as usual. Signed-off-by: Xin Wang <[email protected]> commit 15aa60774a529862c2c348f8af5435a3afc61d0a Author: bingwang-ms <[email protected]> Date: Thu Jul 28 14:52:01 2022 +0800 Remove 'active_tor_mac' from fixture dualtor_info (#6053) PR #5923 updated fixture dualtor_info to add active_tor_mac. However, it caused issue for some other test cases because the return dict is directly passed to check_tunnel_balance, and the function complained that ``` File "/azp/agent/_work/5/s/tests/dualtor/test_orchagent_standby_tor_downstream.py", line 204, in test_standby_tor_downstream_loopback_route_readded check_tunnel_balance(**params) TypeError: check_tunnel_balance() got an unexpected keyword argument 'active_tor_mac' ``` This PR addressed the issue by removing active_tor_mac from the fixture. The variable is added for test case test_encap_dscp_rewrite and test_bounced_back_traffic_in_expected_queue What is the motivation for this PR? This PR is to fix issue caused by updating fixture dualtor_info. How did you do it? Remove the newly added variable active_tor_mac from fixture dualtor_info. How did you verify/test it? The change is verified by running test cases test_standby_tor_downstream_loopback_route_readded and test_tunnel_qos_remap.py. commit e5daa2aa0c11c949290b2a5e576ad66798cbb81d Author: Liu Shilong <[email protected]> Date: Thu Jul 28 14:37:42 2022 +0800 [ci] Transfer organization from Azure to sonic-net (#6059) Description of PR Summary: Transfer organization from Azure to sonic-net commit 60886753bd4e4ddb582438c9a0f09711699e792d Author: Andrii-Yosafat Lozovyi <[email protected]> Date: Thu Jul 28 08:22:53 2022 +0300 Fix vrf "KeyError: 'target_dest_mac'" (#5785) Vrf tests cases fail with KeyError 'target_dest_mac', this issue started to appear after PR - 5456 Changes made in this PR should fix KeyError issue in vrf TC "fib_test.FibTest ... ERROR", "", "======================================================================", "ERROR: fib_test.FibTest", "----------------------------------------------------------------------", "Traceback (most recent call last):", " File \"ptftests/fib_test.py\", line 458, in runTest", " self.check_ip_ranges()", " File \"ptftests/fib_test.py\", line 163, in check_ip_ranges", " self.check_ip_range(ip_range, dut_index, ipv4)", " File \"ptftests/fib_test.py\", line 204, in check_ip_range", " self.check_ip_route(src_port, dst_ip, exp_ports, ipv4)", " File \"ptftests/fib_test.py\", line 237, in check_ip_route", " res = self.check_ipv4_route(src_port, dst_ip_addr, dst_port_list)", " File \"ptftests/fib_test.py\", line 267, in check_ipv4_route", " router_mac = self.ptf_test_port_map[str(src_port)]['target_dest_mac']", "KeyError: 'target_dest_mac'", Signed-off-by: Andrii-Yosafat Lozovyi <[email protected]> commit 7c7d5a25ce7cf4a29820a6de86ea9227a62b64de Author: Soumya Velamala <[email protected]> Date: Wed Jul 27 22:20:07 2022 -0700 Update ip_in_ip_tunnel_test.py (#5853) In test_orchagent_standby_tor_downstream.py, DF bit is set from Cisco-8000 silicon one ASIC since fragmentation on the encapsulated packet is not supported and the expected packet doesn't have it set. This causes the tests to fail despite receiving the complete expected packets. In reference to https://datatracker.ietf.org/doc/html/rfc2003, the outer packet can have the DF bit set when the inner packet does not. Identification, Flags, Fragment Offset These three fields are set as specified in [10]. However, if the "Don't Fragment" bit is set in the inner IP header, it MUST be set in the outer IP header; if the "Don't Fragment" bit is not set in the inner IP header, it MAY be set in the outer IP header, as described in Section 5.1. commit a5ce7f33d6800005b5146ea7a53743711c0d89de Author: slutati1536 <[email protected]> Date: Thu Jul 28 08:18:31 2022 +0300 Remove ONIE downgrade from fwutil tests (#5920) What is the motivation for this PR? We had a bug where the switch doesn't reboot back to sonic after ONIE update. Upon further investigation this bug reproduces on a downgrade from ONIE 5.3.007 to a prior version of ONIE. The cause for this bug is that the current ONIE release (ONIE 5.3.007) has a new E2FS package that is not compatible with the old E2FS package version that existed in previous ONIE versions. As we do not expect this flow to occur in production, only in testing, it was decided to remove the ONIE downgrade from fwutil tests. How did you do it? By removing the ONIE from the random components used in the tests. How did you verify/test it? Run the fwutil tests with the change commit 327a2850029347ecfb8b2f3aab316de60127acbc Author: lipxu <[email protected]> Date: Thu Jul 28 13:14:55 2022 +0800 Fix test_buffer_deployment case NoneType issue (#6050) What is the motivation for this PR? Case test_buffer_deployment failed on the Broadcom devices. How did you do it? Init lossless headroom data for non-mellanox device How did you verify/test it? Re-run the failure case commit ff373cc764a6214e4401c3e9bdf497567e815be8 Author: Anton Ptashnik <[email protected]> Date: Thu Jul 28 07:35:47 2022 +0300 Skipped test_pfc_asym_off_rx_pause_frames for Barefoot platform (#6047) commit fcfe9ebe91151bd54da5bc186d64b7a1d2290ece Author: Oleksandr Kozodoi <[email protected]> Date: Thu Jul 28 07:35:15 2022 +0300 Added ignoring expected errors in test_add_rack TC (#6038) What is the motivation for this PR? There are scenarios, where TC applies patch by using config_updater, but updating is still considered as failed: ``` Failed to apply patch Usage: config apply-patch [OPTIONS] PATCH_FILE_PATH Try "config apply-patch -h" for help. Error: After applying patch to config, there are still some parts not updated ``` Steps from generic_patch.py module cover this case, but TC still failed due to errors of sonic_yang in syslog. So was added fixture which provides an approach for ignoring those errors. How did you do it? Added fixture which provides an approach for ignoring expected error messages of syslog in test_add_rack TC. How did you verify/test it? Run test cases. Tests passed. configlet/test_add_rack.py::test_add_rack PASSED [100%] Signed-off-by: Oleksandr Kozodoi <[email protected]> commit 7e7bb7415f6fb8985ad34e2e1df6b442bd0f7750 Author: andywongarista <[email protected]> Date: Wed Jul 27 21:34:01 2022 -0700 [platform_tests/api] Fix reading voltage and max_supp_power issue of the psu test (#6013) What is the motivation for this PR? Fix issues in psu test that are causing failures on Arista platforms How did you do it? Fix test_power to check if reading voltage is supported for psu Fix test_power to correctly check against max_supp_power How did you verify/test it? Tested on Arista platform commit 56b57a2f9a1026937ee2f27a654fe3d0b93f98d6 Author: ppikh <[email protected]> Date: Thu Jul 28 07:32:13 2022 +0300 [VXLAN] Stabilized VNET VXLAN test in case of scale (#6009) Changed logic to do sleep for longer time after applying VNET VXLAN configs, because current logic does not work in case of scale for example with 330000 vnet routes. Now in case of scale we will wait some time until all configuration applied What is the motivation for this PR? When run test tests/vxlan/test_vnet_vxlan.py with scale num routes 33000 test will fail, because not all routes applied and Wr_ARP test also will fail because warm-reboot does not happen (configuration still in progress). Now this issue fixed. How did you do it? Added sleep in case of scale How did you verify/test it? Executed test tests/vxlan/test_vnet_vxlan.py with and without scale Signed-off-by: Petro Pikh <[email protected]> commit b60dcddded76780365c89a82abd98907f8943b9a Author: bingwang-ms <[email protected]> Date: Thu Jul 28 09:23:25 2022 +0800 [testplan] Add testplan for tunnel QoS remapping (#5508) * Add testplan for tunnel QoS remapping Signed-off-by: bingwang <[email protected]> commit debe6495bacc7022dd099571f8709e7113029616 Author: wenyiz2021 <[email protected]> Date: Wed Jul 27 18:04:30 2022 -0700 [MASIC] [PR checks] [mgmt] Add sonic-mgmt PR check (#6043) * Add 'image' option to get sonic-vs-4asic.img * Add multi-asic job * run multi-asic-t1-lag-pr for now which only runs test_bgp_fact.py * Fix space * update dutname to vlab-08 * Update azure-pipelines.yml for Azure Pipelines * Revert back deleted jobs * Add back empty line * Update azure-pipelines.yml * Update displayname * Align format * Make the PR check optional for now * Remove spaces * Remove spaces commit 2da3aad03cd74828f635e7b1d9e7999ea5ea91f3 Author: Stephen Sun <[email protected]> Date: Thu Jul 28 08:44:05 2022 +0800 [QoS] Verify the additional lossless queues and PGs in QoS test in dual ToR scenario (#5947) * Provide a WA to verify the additional lossless queues and PGs in QoS test in dual ToR scenario 1. Add a CLI option to specify whether it is to verify ports with or without additional lossless PGs and queues. It will collect all the dual ToR ports at the beginning of the test. If the option is on, the additional lossless PGs/queues will be verified in the corresponding tests 2. Update the logic to fetch buffer profiles from BUFFER_QUEUE and BUFFER_PG table according to the CLI 2. Two additional profiles are introduced for verifying the additional lossless PGs/queues in XON and XOFF test if the CLI option is on 3. For headroom pool test, all 4 DSCPs are passed to PTF script as well as the CLI option. The additional DSCPs will be skipped by the PTF docker according to whether the ports are with additional lossless PGs/queues. 4. For DSCP2queue mapping test, the CLI option is passed to PTF script so that the latter is able to check the mapping according to the CLI option. Signed-off-by: Stephen Sun <[email protected]> commit 6af5158a3c7df54c939f1273ab03a1f306246ef7 Author: siqbal1986 <[email protected]> Date: Wed Jul 27 10:41:46 2022 -0700 BFD test update (#5836) Updated BFD test commit c7d19480a23cad967ffc4dee9f799bbccde6e56c Author: mannytaheri <[email protected]> Date: Wed Jul 27 04:46:59 2022 -0400 Added support for returning failure reason if config_system_checks_passed fails (#5975) What is the motivation for this PR? One of the tasks performed by config_system_checks_passed definition is to check if the dut is running. This is done by executing the command "systemctl is-system-running". It returns "False" If the dut is not running but it does not check why the dut is not running. We need to add support for checking the failure reason How did you do it? Execute the command "systemctl is-system-running". Pass if dut is running. Execute the command "systemctl list-units --state=failed" if dut is not running. This will provide the failure reason. Example of failure reason: In this case tacacs-config.timer loaded failed base.py:82 /data/tests/common/devices/multi_asic.py::_run_on_asics#100: [ixre-cpm-chassis10] AnsibleModule::shell Result => {"stderr_lines": [], "cmd": "systemctl list-units --state=failed", "end": "2022-07-09 10:12:18.720054", "_ansible_no_log": false, "stdout": "UNIT LOAD ACTIVE SUB DESCRIPTION\n\u25cf tacacs-config.timer loaded failed failed Delays tacacs apply until SONiC has started\n\n LOAD = Reflects whether the unit definition was properly loaded.\nACTIVE = The high-level unit activation state, i.e. generalization of SUB.\n SUB = The low-level unit activation state, values depend on unit type.\n1 loaded units listed." How did you verify/test it? Tested the code on a dut when it was in the "running" state Tested the code on a dut when it was in the "degraded" state commit d4edaa3262c5bb83be7990de0e0fb9598b842886 Author: Ze Gan <[email protected]> Date: Wed Jul 27 16:42:41 2022 +0800 [loganalyzer]Add ignore log for wpa_supplicant (#5887) #### What is the motivation for this PR? We use SIGINT to stop the wpa_supplication in macsecmgmr. but may get the following log: `wpa_supplicant[388]: eloop: could not process SIGINT or SIGTERM in two seconds. Looks like there#012is a bug that ends up in a busy loop that prevents clean shutdown.#012Killing program forcefully.` #### How did you do it? Add this log to ignore list for log analyzer. Signed-off-by: Ze Gan <[email protected]> commit 9f45f195209b707891b21534d666109dd58f7be0 Author: ppikh <[email protected]> Date: Wed Jul 27 11:40:02 2022 +0300 [sensors] Added new mlnx platforms which have different sensors (#6010) What is the motivation for this PR? Mlnx platforms: 3700, 3700c, 4600c could have different sensors, added logic which allow to test 3700, 3700c ,4600c with different sensors How did you do it? Added new sensors data and improved test to support different sensors(platforms) How did you verify/test it? Executed test_sensors.py Any platform specific information? msn3700, msn3700c, msn4600c Signed-off-by: Petro Pikh <[email protected]> commit cf8fa2549d5440c62abe46c2a902cf41e01cfd61 Author: Nick Wang <[email protected]> Date: Wed Jul 27 16:36:51 2022 +0800 Move test_unknown_mac.py to ARP directory because it it not related to (#6018) For test_unknown_mac.py, it is to test unknown MAC (exists in L2 FDB table but not in ARP table), which is not related to PFC feature. Therefore, the test file should be move to proper directory (ARP). commit bfb4a5965bbc766f826a8272b08456501e047751 Author: cgangx <[email protected]> Date: Wed Jul 27 16:24:37 2022 +0800 Allow LDP traffic on default LDP port (#6019) Add LDP default port to iptables rule set in case LDP is enabled. What is the motivation for this PR? LDP is enabled by default in our case and should be add to iptables rule set. How did you do it? Add iptables rules allowing traffic on LDP default port. How did you verify/test it? Run test on virtual testbed. Co-authored-by: Gang Chen <[email protected]> commit a6fde46867abd986ef58bd221397ee2e694b22fd Author: Shahzad Iqbal (SHAHZADIQBAL) <[email protected]> Date: Tue Jul 26 13:55:56 2022 -0700 Updated BFD multihop test to run between non-subnet ip addresses. commit 6319acec81e2cca81bfacec92bb61ebc58b858e5 Author: SuvarnaMeenakshi <[email protected]> Date: Tue Jul 26 12:58:05 2022 -0700 Avoid using asic_index as list index to get the asic (#6045) What is the motivation for this PR? #5828 - Added asics_present field in inventory to provide the list of asics that are present in supervisor. After this change, asic_index cannot be used to retrive asic_instance from duthost.asics list. This fix is done to get the correct asic_instance based on asic index, without this fix test_pretest can fail on supervisor where the asics_present are not consecutive asic_index How did you do it? remove usage of asic_index as list index. How did you verify/test it? test_pretest passes on chassis after this fix. commit f8dff6d503a8b5386822fbecd0215758b46200e2 Author: roman_savchuk <[email protected]> Date: Tue Jul 26 13:20:22 2022 +0300 Added option for enable collecting DB data when TC failed (#5403) PR 5197 fixture for collection DB data when TC failed have been introduced. This is good idea, but has one major impact to test suite / regression run. It adds extra regression time to run. It will be proportionally increase execution time if number of failed cases increased. What is the motivation for this PR? Add ability to add option if engineer wants to collect DB data How did you do it? Added "--collect_db_data option which enables collecting DB data if TC failed How did you verify/test it? Run cases with --collect_db_data, if TC failed - DB data collected. Run cases without --collect_db_data, if TC failed - DB not data collected. commit 4df2f37508a292280beed1936b159fb9f2004285 Author: rskorka <[email protected]> Date: Tue Jul 26 03:08:44 2022 -0700 Added support for Cisco 8000 virtual sonic DUT (#5908) Add playbooks which allow to use T0/T1 virtual testbeds with Cisco 8000e emulator (functional emulator of Cisco 8000 Series routers). How did you do it? Add start/stop playbooks for new DUT type: 8000e. The playbook starts the emulator inside a docker container (for ease of deployment) and then vm-topology module can connect it to the topology. How did you verify/test it? ./testbed-cli.sh -m veos_vtb -n 4 -k vsonic start-vms server_1 password.txt ./testbed-cli.sh -k vsonic -t vtestbed.yaml -m veos_vtb add-topo 8000e-t0 password.txt ./testbed-cli.sh -t vtestbed.yaml -m veos_vtb deploy-mg 8000e-t0 veos_vtb password.txt cd ../tests ./run_tests.sh -n 8000e-t0 -d vlab-8k-01 -c bgp/test_bgp_fact.py -f ../ansible/vtestbed.yaml -i ../ansible/veos_vtb -e --disable_loganalyzer bgp/test_bgp_fact.py::test_bgp_facts[vlab-8k-01-None] PASSED Any platform specific information? While the 8000e emulator supports many Cisco 8000 platforms, the new 8000e playbooks support Cisco-8102-C64 platform specifically. Other 8000e platforms will be tested and enabled in the future. Co-authored-by: Rafal L Skorka <[email protected]> commit d0052cee1d4f8973683a08b05feab2111af88814 Author: MirceaDan <[email protected]> Date: Tue Jul 26 03:05:29 2022 -0700 added support for IxNetwork 9.20 Update2 (#6028) updated for spytest the container environment to add support for ixnetwork 9.20u2 What is the motivation for this PR? previous container version was having support for 9.10, but i got requests to add support for 9.20 How did you do it? just a version bump and better versioning for some python packages to avoid conflicts How did you verify/test it? ran a test via the framework Co-authored-by: MirceaDan <[email protected]> commit f1141557e47d3d58e8b300d287520075b967acc2 Author: vperumal <[email protected]> Date: Tue Jul 26 03:04:01 2022 -0700 Skipping test for absent psu's (#5679) What is the motivation for this PR? Currently the tests are trying to access PSU which are not present and failing for them. Even though there is a skip_psu_list, there is no code present to use it. Added the support for it in all the relevant cases. How did you do it? How did you verify/test it? Verified against cisco-8000 platform Co-authored-by: Perumal Venkatesh <[email protected]> commit 4033ab94a0275066b9ec7e5fbcaab03628088913 Author: oleksandrKovtunenko <[email protected]> Date: Tue Jul 26 13:02:40 2022 +0300 added skip LAG ports in case T1 TOPO Fix for issue https://github.com/Azure/sonic-mgmt/issues/5578 (#5704) Fix for issues/5578 skip LAG ports in case t1 topology usage What is the motivation for this PR? fix for #5578 exclude portchanel ports in T1 topology How did you do it? skip PortChannel interfaces on T1 topology How did you verify/test it? run test_qos_sai.py tests on t1-lag topology and check that portchanel ports are excluded Co-authored-by: alexander Kovtunenko <[email protected]> commit f5bcc52a769aee224539b73637a1b054a1b48ce2 Author: roman_savchuk <[email protected]> Date: Tue Jul 26 13:01:26 2022 +0300 [cpu_mem_usage] update TC with wait for all critical services is fully started (#5754) During regression cpu_usage TC run's after TC that does config reload at the end. Due to this syncd CPU usage is higher than TC expects as not all services come up and syncd actively works at period after system reload (reboot). What is the motivation for this PR? Make TC persistent. Avoid failures after DUT reboot (reload) How did you do it? Add wait_until method for check if all critical services come up and than run TC How did you verify/test it? Run cpu_mem_usage test case when services up (normal DUT state) and when services has been restarted. TC passed. Any platform specific information? NOTE: if TC run when all services started it takes app 180 sec to pass TC. after reboot or config reload TC run takes 180 sec + time for critical services to be started (200 sec) commit dba8c50864fed6c28fdc2b9417a2c09bf72de38f Author: Nana@Nvidia <[email protected]> Date: Tue Jul 26 09:32:06 2022 +0800 Fix the json dumps failure issue in sonic.py (#6033) What is the motivation for this PR? When the value of facts["asics_present"] is range(0,1), then the json.dumps(facts) will throw exception: TypeError: Object of type 'range' is not JSON serializable, need to change the range(0,1) to list. This PR is to fix the json.dumps(facts) failure issue How did you do it? Change facts["asics_present"] = asics_present if len(asics_present) != 0 else range(facts["num_asic"]) to facts["asics_present"] = asics_present if len(asics_present) != 0 else list(range(facts["num_asic"])) How did you verify/test it? Run the test case, and the json.dumps can pass commit 8e0d21d524c939829c73c830219b78e1ff015e16 Author: bingwang-ms <[email protected]> Date: Tue Jul 26 08:11:49 2022 +0800 [EVERFLOW][hotfix] Skip EGRESS MIRRORING test on Broadcom platform (#6026) * Skip EGRESS MIRRORING test on Broadcom platform * Skip dnx commit 2a801bc7fef29cd3e2c6fa714f6508b9cad9b0fe Author: Zhaohui Sun <[email protected]> Date: Tue Jul 26 07:55:21 2022 +0800 Fix test case issue for dualtor in test_lag_2 (#6025) What is the motivation for this PR? test_lag_db_status failed on dualtor testbed. Two issues: It added pytest.fail wrongly. dut_name, dut_lag = decode_dut_port_name(enum_dut_portchannel_with_completeness_level) It will return one dut_name in one case, but we don't which dut is chosen for dualtor. That's why we loop duthosts to find the correct one and run test. If it loops for another duthost, we should skip it not fail the test case. Add one common function get_duthost_with_name to return duthost, reduce two indents In recover phase, it uses test_lags wrongly for dualtor. Since if test case failed in step 1, test_lags is not defined. We should loop all duthosts to check if interface status is down, if so, recover it by noshutdown. Remove duthost for loop for test_lag_db_status_with_po_update, because this case if for t1-lag only. How did you do it? Remove pytest.fail in test case test_lag_db_status and test_lag_db_status_with_po_update. Enhance recover steps. How did you verify/test it? Run pc/test_lags_2.py::test_lag_db_status. Signed-off-by: Zhaohui Sun <[email protected]> commit 45b20e67f0aa9541a277c53928c55cda82177f6a Author: Nana@Nvidia <[email protected]> Date: Tue Jul 26 04:40:05 2022 +0800 [Qos]Fix the testQosSaiQSharedWatermark test failure due to SAI queue watermark is not reset (#6001) configuring counterpoll watermark enable doesn't suffice to enable the queue watermark polling if the counter polling was disabled. As as result, the queue watermarks in SAI will not be clear successfully, which will fail the queue watermark test.To avoid that, counter poll should be enabled for both queue and watermark.with this command, the SAI Queue watermark can be cleared. How did you do it? Add "counterpoll queue enable", "counterpoll queue disable" in the resetWatermark fixture How did you verify/test it? Run the testQosSaiQSharedWatermark , test will not fail due to the SAI queue watermark is not cleared commit 0fefd1318014673a351d4afb9cd152cc2f3cb7ef Author: Ye Jianquan <[email protected]> Date: Mon Jul 25 13:14:24 2022 -0700 [RDMA&SNAPPI] Skip snappi warm-reboot testcases on TD2 platform (#6030) How did you do it? Compare the skipped testcases of tgen and snappi, and skip the ones that need to be skipped in snappi. commit 63c03759f44afe2013c600ae8a7248397f2dea8c Author: Andrii-Yosafat Lozovyi <[email protected]> Date: Mon Jul 25 13:03:04 2022 +0300 [auto-ts] Fix and stabilize auto-techsupport tests (#5986) Summary: Made changes that fixes some issues with auto-techsupport and stabilizes TC Main changes that was made: 1.) Changed --since option from '300 sec': 360 to '300 sec ago': 360 This is made because TC will take actual time and add 300 seconds to it, and will collect logs and core dump only starting from that time according to - Date-input-formats 2.) Check available_tech_support_files before core dump generation is triggered in test_rate_limit_interval Test might fail because when available_tech_support_files is checked after core dump was generated, techsupport dump file is already generated, and TC fails in further steps because expects new_techdump to be generated. Signed-off-by: Andrii-Yosafat Lozovyi <[email protected]> commit 7157274f04c986501141774fb1b53dc9d42b81d2 Author: Xin Wang <[email protected]> Date: Mon Jul 25 15:00:01 2022 +0800 Improve robustness of remove-topo operation (#6020) What is the motivation for this PR? The testbed-cli.sh remove-topo operation is to remove the topology, including cEOS neighbors, ovs bridge bindings, and ovs bridges. For some reason, the ovs bridges to be removed may be gone. In this case, the testbed-cli.sh remove-topo will fail with not able to find the bridges. Indeed, this failure is unnecessary because this operation is to remove the bridge bindings and bridges. If a bridge is already gone, then we can simply ignore it. How did you do it? This change improved the vm_topology ansible module to skip a bridge if it does not exit while trying to remove the bridge bindings and bridges. With this change, the testbed-cli.sh remove-topo always can succeed and do the job. This PR also improved the error message if run a command failed. How did you verify/test it? Tested on physical and virtual testbed. Tried remove-topo, add-topo and restart-ptf with good and broken test topology. Signed-off-by: Xin Wang <[email protected]> commit 73a2e5a81045a5d2924f5fdc8c16d037ea0a704d Author: Longxiang Lyu <[email protected]> Date: Mon Jul 25 14:45:04 2022 +0800 [dualtor][active-active] Unblock fib tests (#6006) Approach What is the motivation for this PR? Enable test_fib on dualtor-mixed testbeds. Signed-off-by: Longxiang Lyu [email protected] How did you do it? 1. Add fixture mux_status_from_nic_simulator to interacts with nic_simulator to retrieve mux status for ports in active-active cable type. 2. Add fixture ptf_test_port_map_active_active to build ptf port map that supports dualtor-mixed testbed. The key difference is that, for packets ingressing ptf ports connected to DUTs' active-active cable type ports, the target DUT could be either upper ToR or lower ToR(both ToRs are active), the generated ptf port mapping will be like: ``` u'3': {u'asic_idx': 0, u'target_dest_mac': u'00:aa:bb:cc:dd:ee', u'target_dut': [0, 1], u'target_src_mac': [u'2c:dd:e9:0f:4e:50', u'2c:dd:e9:0f:3d:4c']} ``` 3. Enable ptftests hash_test and fib_test to do I/O verification for this multi-next-hop scenario. For a packet sent from ptf port that connects to an active-active port, the test will try to use the fibs from each ToR to parse its nexthops, and verify the packet forwarding on those nexthops. For ECMP behavior, the test will only check balancing over a single ToR. For example, if a packet destinated to 1.1.1.1 is sent to ptf port eth3, which is connected Ethernet12 of both ToRs, the test will determine the nexthops on both ToRs, if both ToRs forward this packet with the default route, and upper ToR will forward this packet to ptf ports [30, 32, 34, 36], and lower ToR will forward this packet to ptf ports [31, 33, 35, 37], the test will try to verify the packet forwarding on ports [30, 31, 32, 33, 34, 35, 36, 37]. And for balancing, the test will verify the traffic is balanced over a single ToR, so it will try to verify balancing on ports set [30, 32, 34, 36] or [31, 33, 35, 37] separately. How did you verify/test it? Run test_fib on dualtor, dualtor-mixed, t0, and t1 Any platform specific information? Supported testbed topology if it's a new test case? commit e027d9b449d78ac4702e8d8fbb840302eae516e3 Author: Shahzad Iqbal (SHAHZADIQBAL) <[email protected]> Date: Sat Jul 23 16:45:36 2022 -0700 Added support ofr multi-hop bfd testing. commit 857e83ef07a48af973fa458b268f6db67b0f1454 Author: Longxiang Lyu <[email protected]> Date: Sat Jul 23 11:01:54 2022 +0800 [dualtor] Leave `icmp_responder` running on `dualtor-mixed` testbeds (#5972) Approach What is the motivation for this PR? On dualtor-mixed testbeds, leave icmp_responder running to avoid introducing unnecessary toggles. Signed-off-by: Longxiang Lyu [email protected] How did you do it? Comment out the teardown to stop icmp_responder How did you verify/test it? commit ca4d0fa69063a1a2b3a119d4103304f35d7c0634 Author: Xin Wang <[email protected]> Date: Fri Jul 22 13:56:13 2022 +0800 Fix filename issue and collect important DB dumps (#6007) What is the motivation for this PR? The autoused fixture collect_db_dump has some issues: * If a test case name has some special characters, the fixture may fail. * It automatically collects dump of all databases for each failed test case. It's unnecessary to collect all the DB dumps. How did you do it? * The collected dumps are stored to a folder named with the test case name. However, test case name may have characters that can't be used in file name. This change added a utility function to remove characters illegal in filename to convert a string to a safe filename. * Not all the collected database dumps are useful for later troubleshooting. This change improved the fixture to only collect dumps of some important databases. * This change also deleted the fetch_dbs function which is not used anywhere. Signed-off-by: Xin Wang <[email protected]> commit cfc550a27ace8043dd171c113b50a8627c03ecda Author: Shahzad Iqbal <[email protected]> Date: Thu Jul 21 16:38:29 2022 -0700 LGTM Alrts fixed. removed unused function. commit e33b2284dbe6ae74234c901602d1e1ffaceaf5dd Author: ppikh <[email protected]> Date: Thu Jul 21 19:41:50 2022 +0300 [auto_techsupport] alligned auto_tech_support tests with latest CLI on SONiC master image (#5954) [auto_techsupport] alligned auto_tech_support tests with latest CLI on SONiC master image CLI has been changed - as result tests started to fail Now CLI parser fixed by implementing multi-branch support and tests should pass Signed-off-by: Petro Pikh <[email protected]> commit ffcdd14920e760bc2adeeb9ea6e031075f0c4b55 Author: StormLiangMS <[email protected]> Date: Thu Jul 21 01:54:29 2022 -0700 [vlan/test_vlan_ping] skip test on broadcom platform #6012 What is the motivation for this PR? To skip test test_vlan_ping which doesn't work for broadcom platform. How did you do it? Add test to tests/common/plugins/conditional_mark/tests_mark_conditions.yaml to skip it. How did you verify/test it? Any platform specific information? broadcom platform Supported testbed topology if it's a new test case? commit b1866a1ba00b2b338356e5999c09c549cb7c64a3 Author: Longxiang Lyu <[email protected]> Date: Thu Jul 21 11:28:16 2022 +0800 [nic_simulator] Add timeout and common options (#6005) Approach What is the motivation for this PR? mgmt client could not talks to the gRPC server. Signed-off-by: Longxiang Lyu <[email protected]> How did you do it? Bring up loopback device to enable mgmt service talks to each gRPC server interacting with SONiC. Add common gRPC options to the nic_simulator. Add timeout for the gRPC calls from mgmt service. How did you verify/test it? Verify mgmt client could talks to the mgmt service of `nic_simulator. Any platform specific information? commit 9acb53139a151a7ade913335b72a4cfa778781d5 Author: Zhaohui Sun <[email protected]> Date: Thu Jul 21 10:40:50 2022 +0800 Update require option to False for os_version in test_reporter.py to avoid uploading failure in nightly test (#6015) What is the motivation for this PR? In https://github.com/Azure/sonic-mgmt/pull/5992, wrongly add os_version parameter as required True. It should be false, because if with required True, test_reporter.py will fail to run for current nightly test. It impacts nightly test, should make it robust with current nightly test yaml file. How did you do it? Change it to required False. How did you verify/test it? Run nightly test with pipeline. Signed-off-by: Zhaohui Sun <[email protected]> commit 656624555e2fa5f99a6ceef740b05d96bd5790ff Author: Zhaohui Sun <[email protected]> Date: Thu Jul 21 08:47:30 2022 +0800 Enhance test report to include pipeline results (#5992) What is the motivation for this PR? Currently, when nightly test pipeline fails before running test, test report upload will fail too, because there is no XML file and it will throw error out. We don't know if pipeline does not run on that day, or it fails. How did you do it? Enhance test report upload scripts to record pipeline status and upload it to kusto. Add a new collect_azp_results.py to collect task status for specific pipeline. If there is no XML file, upload summary table with 0 values. Create a new table TestReportPipeline, record testbed name, os version, success tasks, failed tasks and cancelled tasks and upload the record to kusto. How did you verify/test it? Run nightly test and check kusto. Signed-off-by: Zhaohui Sun <[email protected]> commit cbab17e5b512a9f5028dfde05ff7995df9028d1f Author: ppikh <[email protected]> Date: Thu Jul 21 03:30:47 2022 +0300 [conditional_mark] Improved conditional_mark plugin to support "OR" or "AND" condition between condition in conditions list (#6008) Improved conditional_mark plugin to support "OR" or "AND" operand between condition in conditions list Previously every time we did AND operand between condition in conditions list, now we can provide "conditions_logical_operator" argument with operation which should be performed between conditions. Possible arguments (by default, if not provided - AND used): ``` conditions_logical_operator: or conditions_logical_operator: and ``` Example of usage (test will be ignored if first or second condition in list True): ``` ecmp/test_fgnhg.py: skip: reason: "Testcase ignored - check ignore condition in ignore file" conditions_logical_operator: or conditions: - "https://redmine.x.com/issues/12345 and 'msn2' in platform" - https://redmine.x.com/issues/54321 ``` Signed-off-by: Petro Pikh <[email protected]> commit f526c6f6de1de6e100bf59240155b1d50a824e9d Author: Richard.Yu <[email protected]> Date: Thu Jul 21 08:00:42 2022 +0800 add fdb test cases (#6004) * add fdb test cases Signed-off-by: richardyu-ms <[email protected]> * Update cases_brcm_t0.py commit 2c34f877cf1d537d81fdc1b40a7e8926288b332f Author: Jing Zhang <[email protected]> Date: Wed Jul 20 14:42:01 2022 -0700 [LogAnalyzer] white-list Could not get port instance error message (#5995) Summary: Fixes # (issue) This PR is to temporarily white-list the error log below: Jul 14 18:56:30.247123 svcstr-7050-acs-2 ERR pmon#ycable[30]: Error: Could not get port instance for muxcable info for Y cable port Ethernet24 There should be an image fix soon for this. sign-off: Jing Zhang [email protected] commit 85ce59619567317fbd6869c04937ca9ba33ff5e2 Author: mannytaheri <[email protected]> Date: Wed Jul 20 15:38:34 2022 -0400 enum_asic fixtures to use only asics that are present instead of all asics based on num_asics (#5828) What is the motivation for this PR? Tests that are parameterized using the enum_asic fixtures fail when the selected asic index is not present (operational) in the DUT. Example of this is on the supervisor card in a chassis, where we could have not all SFMs in the chassis. In this case, only asics corresponding to the SFMs present in the chassis are operational, but not the others. For example, on a Nokia chassis, if we have only 3 SFM's then only 6 (2 per SFM) out of the possible max 16 asics are operational. How did you do it? Added optional "asics_present" to the inventory data of the DUT in the inventory file. This represents the list of asics that are operational on the DUT. If asics_present is present in the inventory data, we would use that list for generating the list of asics to use. Else, we use range of 'num_asics' defined in the inventory data to generate the list of asics to use. Example inventory data for a supervisor card supervisor1: num_asics: 16 asics_present: [12,13] In the above case, the asic list will be only 12 and 13 and not 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15 How did you verify/test it? Tested the code on a dut that did not have all fabric-cards for the following fixtures: enum_dut_hostname enum_asic_index enum_frontend_asic_index enum_backend_asic_index enum_rand_one_asic_index enum_rand_one_frontend_asic_index Ran autorestart container test against the same chassis - which used enum_rand_one_asic_index and it picked the asic that was operational. commit 559217ba682f3bd0b9a2b42c341e64c6fffb8c29 Author: Vivek <[email protected]> Date: Wed Jul 20 09:41:48 2022 -0700 [PFCWD] [QOS] Added a Test Case which triggers a PFC Storm when the occupancy crossed into shared headroom (#5902) Signed-off-by: Vivek Reddy <[email protected]> How did you verify/test it? Tested on the device which had this problem. "start": "2022-06-17 07:11:28.679981", "stderr": "sai_qos_tests.PtfReleaseBuffer ... FAIL\n\n======================================================================\nFAIL: sai_qos_tests.PtfReleaseBuffer\n----------------------------------------------------------------------\nTraceback (most recent call last):\n File \"saitests/sai_qos_tests.py\", line 1003, in runTest\n assert(recv_counters[self.pg] == recv_counters_base[self.pg])\nAssertionError\n\n----------------------------------------------------------------------\nRan 1 test in 50.595s\n\nFAILED (failures=1)", "stderr_lines": [ "sai_qos_tests.PtfReleaseBuffer ... FAIL", "", "======================================================================", "FAIL: sai_qos_tests.PtfReleaseBuffer", "----------------------------------------------------------------------", "Traceback (most recent call last):", " File \"saitests/sai_qos_tests.py\", line 1003, in runTest", " assert(recv_counters[self.pg] == recv_counters_base[self.pg])", "AssertionError", "", "----------------------------------------------------------------------", "Ran 1 test in 50.595s", "", "FAILED (failures=1)" ], "stdout": "", "stdout_lines": [] Verified on 202012 branch as well After the storm was restored and packets were drained, the switch still sends PFC frames and the test detects it. admin@mtbc-sonic-01-2410:~$ pfcwd show stats QUEUE STATUS STORM DETECTED/RESTORED TX OK/DROP RX OK/DROP TX LAST OK/DROP RX LAST OK/DROP ----------- -------- ------------------------- ------------ ------------ ----------------- ----------------- Ethernet4:3 N/A 1/1 0/0 0/0 0/0 0/0 admin@mtbc-sonic-01-2410:~$ show pfc counters | grep -E '\<Ethernet4\>' Ethernet4 0 0 0 2,565,200 0 0 0 0 Ethernet4 0 0 0 257,943 2 0 0 0 admin@mtbc-sonic-01-2410:~$ show pfc counters | grep -E '\<Ethernet4\>' Ethernet4 0 0 0 2,565,200 0 0 0 0 Ethernet4 0 0 0 263,911 2 0 0 0 admin@mtbc-sonic-01-2410:~$ show pfc counters | grep -E '\<Ethernet4\>' Ethernet4 0 0 0 2,565,200 0 0 0 0 Ethernet4 0 0 0 375,742 2 0 0 0 Counters Recorded by PTF Docker The time diff b/w counters is 30 sec. Both of these are collected after the watchdog is restored and the packets are drained (i.e. dst port is re-enabled) There shouldn't be any any increment in SAI_PORT_STAT_PFC_3_TX_PKTS values but there is one indicating a faulty behavior 07:12:20.365 root : DEBUG : Recv Counters: [0, 0, 0, 0, 0, 83369, 2, 0, 0, 0, 5347560, 1, 0, 0], Base: [0, 0, 0, 0, 0, 38597, 2, 0, 0, 0, 2482152, 1, 0, 0] Any platform specific information? Yes, only for Mellanox platforms commit e9dceab0db97349557dcfc66480a375e876bddef Author: longhuan-cisco <[email protected]> Date: Wed Jul 20 06:02:20 2022 -0700 Update port_alias map for Cisco-8102-C64 platform (#6002) Signed-off-by: Longyin Huang [email protected] commit 73ecf5f5937a5b97109f201dbafdb64ce28134c1 Author: Ze Gan <[email protected]> Date: Wed Jul 20 19:58:37 2022 +0800 [getbuild.py]: Download the latest sonic image (#6003) What is the motivation for this PR? To fix the PR: https://github.com/Azure/sonic-mgmt/pull/5746 which can only download the partially succeeded image. How did you do it? Fetch both the succeeded and partially succeeded build id and choose the newer one as the latest sonic image. How did you verify/test it? show version in vsonic neighbor devices. Signed-off-by: Ze Gan <[email protected]> commit 862ed0a8d4ad464cec7404ddaded53abb96c3ba6 Author: Nana@Nvidia <[email protected]> Date: Wed Jul 20 10:00:05 2022 +0800 [po_update]fix local variable referenced before assignment issue (#5950) In the test_po_update_io_no_loss, there is try-finally block, some local vars are defined in the try, and used in the finally, is there is some exception before the local var assigned, then it will throw exception local variable 'xxx' referenced before assignment commit c5eff2fd9de1f778fcc76945dc00dd799678cd43 Author: Zhaohui Sun <[email protected]> Date: Wed Jul 20 09:10:22 2022 +0800 Add radv test script into PR test (#5999) What is the motivation for this PR? radv test is missed from PR test. How did you do it? Add radv/test_radv_ipv6.py into kvmtest.sh to cover it PR test. How did you verify/test it? run test_radv on vms-kvm-t0: radv/test_radv_ipv6_ra.py::test_radv_router_advertisement[active-standby] PASSED [ 25%] radv/test_radv_ipv6_ra.py::test_solicited_router_advertisement[active-standby] PASSED [ 50%] radv/test_radv_ipv6_ra.py::test_unsolicited_router_advertisement_with_m_flag[active-standby] PASSED [ 75%] radv/test_radv_ipv6_ra.py::test_solicited_router_advertisement_with_m_flag[active-standby] PASSED [100%] Signed-off-by: Zhaohui Sun <[email protected]> commit cda47e9aa63780c5832a32b23936f8bc5fe72059 Author: Anton Ptashnik <[email protected]> Date: Tue Jul 19 11:47:34 2022 +0300 skipped QoS multi-asic tests before swapping syncd (#5757) What is the motivation for this PR? Avoid any setup steps execution for a skipped test How did you do it? Made skip conditions to execute before any setup steps How did you verify/test it? py.test --inventory=../ansible/lab,../ansible/veos --testbed_file=../ansible/testbed.csv --module-path=../ansible/library -v -rA --topology=t1,any qos/test_qos_masic.py commit 5068d9da0a144ef9d176b787e24421232c3ba3d4 Author: Ying Xie <[email protected]> Date: Mon Jul 18 18:09:16 2022 -0700 [vlan/dualtor] address a few test issues (#5998) What is the motivation for this PR? vlan test is failing due to various variable validity issues. Also noticed lingering vlan configurations after tests. How did you do it? Saving the mux config will address the inconsistent mux config issue, however, it saves the in-memory change causing tests relying on config reload to recovery to fail. Back off this change since sanity check can make mux config consistent. Mark global variable properly. Initialize variables properly. How did you verify/test it? run vlan tests, most tests passed except one due to known image issue. No longer see lingering vlan configuration after test is done. Signed-off-by: Ying Xie [email protected] commit 2e16669df14e6fd52c9d35c27f1b6ae6de64b73f Author: Ying Xie <[email protected]> Date: Mon Jul 18 18:08:49 2022 -0700 [radv] fix fixture scope issue (#5996) What is the motivation for this PR? test_radv is failing on dualtor testbed due to fixture scope conflicts. How did you do it? toggle_all_simulator_ports_to_upper_tor is a function level fixture, it cannot be used by a module level fixture. Use it at per test basis. How did you verify/test it? run test_radv on a dualtor testbed: radv/test_radv_ipv6_ra.py::test_radv_router_advertisement[active-standby] PASSED [ 25%] radv/test_radv_ipv6_ra.py::test_solicited_router_advertisement[active-standby] PASSED [ 50%] radv/test_radv_ipv6_ra.py::test_unsolicited_router_advertisement_with_m_flag[active-standby] PASSED [ 75%] radv/test_radv_ipv6_ra.py::test_solicited_router_advertisement_with_m_flag[active-standby] PASSED [100%] Signed-off-by: Ying Xie [email protected] commit 60ac30557c35cc212a647b3add5083ebeadd4141 Author: Ze Gan <[email protected]> Date: Tue Jul 19 08:05:42 2022 +0800 [ptf]: Enhance ptf with macsec (#5987) What is the motivation for this PR? Got few unexpected commands like sonic-db-cli APPL_DB HGETALL 'MACSEC_PORT_TABLE:Ethernet164 after issuing warm-reboot. Any commands issued during shutdown process are subjected to fail/surprises. How did you do it? If the MACsec wasn't enabled, don't try to load MACsec info. How did you verify/test it? Check testcases: ecmp.inner_hashing.test_wr_inner_hashing.TestWRDynamicInnerHashing.test_inner_hashing Signed-off-by: Ze Gan <[email protected]> commit d9da0ea38000bf6faea3609e14a2bcc749a3b6bd Author: SuvarnaMeenakshi <[email protected]> Date: Mon Jul 18 16:15:55 2022 -0700 [multi_asic]: Add support to get multi-asic VS build. (#5988) What is the motivation for this PR? Update getbuild script to get sonic multi-asic KVM image. Depends on pipeline change to build multi-asic KVM image Azure/sonic-buildimage#11453. How did you do it? Update script to take a new parameter num_asic to download multi-asic KVM images sonic-4asic-vs.img.gz and sonic-6asic-vs.img.gz or by default assume that it is a single asic VS image. How did you verify/test it? Tested out to get the right image based on num_asic: python3 tests/scripts/getbuild.py --platform vs --branch master --num_asic 6 --buildid 122463 (using the buildid of run of Azure/sonic-buildimage#11453 PR) commit 48b24da1594f7644c12cf7b87cc51efc31ac48d2 Merge: 0a96bf75 478d765d Author: Shahzad Iqbal <[email protected]> Date: Mon Jul 18 12:53:31 2022 -0700 Merge branch 'bfd_test_update' of https://github.com/siqbal1986/sonic-mgmt into bfd_test_update commit 0a96bf7523d4d252aef645a44cc88af8389cac59 Author: Shahzad Iqbal <[email protected]> Date: Mon Jul 18 12:52:54 2022 -0700 This changes includes the following. 1) Fixes concerns raised in review. 2) Fixes a bug which disabledthe scale test 3) Code clean up and reorg. 4) Add testing of Admin Down and suspend session 5) Uses pytest parameter options to test ip v4,v6 and dut,ptf init variation. commit 4fee0685cdec49248b0809356e20c426262438b7 Author: SuvarnaMeenakshi <[email protected]> Date: Mon Jul 18 11:44:43 2022 -0700 Modify topo for msft_four_asic_vs testbed (#5983) What is the motivation for this PR? While running multi-asic VS tests on Azure Pipeline, we see that fpmsyncd/syncd/orchagent crashes with Redis busy error log. ul 6 11:15:32.533912 vlab-08 ERR bgp2#fpmsyncd: :- checkReplyType: Expected to get redis type 4 got type 6, err: BUSY Redis is busy running a script. You can only call SCRIPT KILL or SHUTDOWN NOSAVE. Jul 6 11:15:32.533912 vlab-08 ERR bgp2#fpmsyncd: :- poll_descriptors: readData error: Expected to get redis type 4 got type 6, err: BUSY Redis is busy running a script. You can only call SCRIPT KILL or SHUTDOWN NOSAVE.: Input/output error Jul 6 11:15:32.533912 vlab-08 ERR bgp2#fpmsyncd: :- checkReplyType: Expected to get redis type 4 got type 6, err: BUSY Redis is busy running a script. You can only call SCRIPT KILL or SHUTDOWN NOSAVE. Jul 6 11:15:32.713647 vlab-08 ERR snmp#python3: :- guard: RedisReply catches system_error: command: SELECT 4, reason: BUSY Redis is busy running a script. You can only call SCRIPT KILL or SHUTDOWN NOSAVE.: Input/output error Jul 6 11:15:32.814734 vlab-08 INFO bgp2#/supervisord: fpmsyncd terminate called without an active exception Jul 6 11:15:34.022281 vlab-08 INFO bgp2#supervisord 2022-07-06 11:15:34,001 INFO exited: fpmsyncd (terminated by SIGABRT (core dumped); not expected) Test pipeline where this error was seen: Pipelines - Run build_SuvarnaMeenakshi.sonic-buildimage (2)_vs_azp2_20220706.1 (azure.com) If we reduce the number of external VMs to 2 neighbors, this issue is not seen. How did you do it? Reduce the number of external BGP neighbors to 2 for t1-8-lag topology which is used by msft_four_asic_vs testbed. Add a new function to run test_bgp_fact test so that kvmtest.sh can be used in multi-asic PR checker to run a basic test case. This can further be extended with more test in future. How did you verify/test it? Have a test pipeline with fewer asics and ensure fpmsyncd crash is not seen. Tested on multi-asic vs: ./kvmtest.sh -en -T multi-asic-t1-lag-pr vms-kvm-four-asic-t1-lag vlab-08 -d /var/src/sonic-mgmt commit 73cf7a444daa7d8c71699f211509ca2498ef480d Author: Antonh <[email protected]> Date: Mon Jul 18 21:15:02 2022 +0300 [sensors] update sensors data for kernel 5.10 (#5922) For kernel 5.10 the sensor alarms we have for some pmbus drivers are in1_lcrit_alarm / in1_crit_alarm in2_lcrit_alarm / in2_crit_alarm In kernel 4.19 we only have the following in1_alarm in2_alarm Signed-off-by: Anton <[email protected]> commit 569193234556f3930ae9c09ca93f68e6101e7052 Author: Nana@Nvidia <[email protected]> Date: Tue Jul 19 02:13:58 2022 +0800 [sFlow] Add import for subprocess in sflow_test (#5937) In the test file, the subprocess is used without importing this lib - What is the motivation for this PR? In the test file, the subprocess is used without importing this lib - How did you do it? Add the import for subprocess - How did you verify/test it? Run the sflow test, and no exception. commit f9cfdecde63858bc82a770c8b7dbb8b087278827 Author: Junchao-Mellanox <[email protected]> Date: Tue Jul 19 02:09:02 2022 +0800 [loganalyzer] Make queue stats log ignore regular expression more general (#5942) Change queue stats log ignore regular expression for PR Azure/sonic-sairedis#1073. The new regular expression will be generic enough to cover both old log and new log format. - What is the motivation for this PR? Change queue stats log ignore regular expression for PR Azure/sonic-sairedis#1073. The new regular expression will be generic enough to cover both old log and new log format. - How did you do it? Make queue stats log ignore regular expression more general - How did you verify/test it? Manual test commit 2907a79c6d4747c10a672feb4e7a847f5376ace6 Author: Cong Hou <[email protected]> Date: Tue Jul 19 02:03:42 2022 +0800 Add a check for PCIE_DEVICES table status before getting the key (#5944) Before getting the key of PCIE_DEVICE table, check if the table is ready. The reason is if the previous test has a config reload in teardown, even if the critical processes are all running, it still takes some time to restore all the data tables including the PCIE_DEVICE. There is a chance the table is still empty when the pcie test starts to run and try to get the key. - What is the motivation for this PR? Add a check for PCIE_DEVICES table status before getting the key to improve test stability. - How did you do it? Add a function to check the PCIE_DEVICES table status, and wait until the table is not empty for at most 30 seconds. - How did you verify/test it? Run regression test on all Nvidia platforms with 202012 and 202205 version. All passed. commit 32713fd3f4da51c19e0a4f3368ded1fa5c479265 Author: Antonh <[email protected]> Date: Mon Jul 18 21:00:22 2022 +0300 [drop_counters] fix parsing of fanout trunk ports (#5967) The previous parsing of interface name was not support the split interfaces. iface = 'ethernet 1/26/2' expected: fanout_trunk_port = '26/2' actual: fanout_trunk_port = '2' now the actual as expected. Signed-off-by: Anton <[email protected]> commit a2ee1c60f58ff1cdc8c2af673991f3bb6df02cf2 Author: Nana@Nvidia <[email protected]> Date: Tue Jul 19 01:59:18 2022 +0800 Add missed tbinfo fxiture in test_turn_on_off_psu_and_check_psustatus (#5990) tbinfo is used in the PR #5968, but the fixture is not added in the test case function. - How did you do it? Add the tbinfo fixture in the test_turn_on_off_psu_and_check_psustatus - How did you verify/test it? Run the test_turn_on_off_psu_and_check_psustatus, test pass commit d3f8c069909b2aed888de798a546255efb1b4725 Author: abdosi <[email protected]> Date: Mon Jul 18 07:10:54 2022 -0700 Added Deployment_id in Asic metadata. (#5957) Why I did: As describe here: Azure/sonic-buildimage#11407 commit 857fa0fbd511c035e9eb051fb4ea4a8d9f0a441f Author: Zhaohui Sun <[email protected]> Date: Mon Jul 18 16:57:49 2022 +0800 [test_gap] Add two test cases to check if the status of lag interfaces is in sync with state_db (#5952) What is the motivation for this PR? PR test didn't catch the issue Azure/sonic-buildimage#11255. test_lldp.py run to success because netdev_oper_status of interfaces is up in state_db. But in physical testbed, netdev_oper_status of some interfaces is down and it impacts the lldp behavior. So test_lldp.py caught it. Root cause is the netdev_oper_status in state_db is not synced with the real status of interface. If the image has this issue, no matter you shutdown or no shutdown interface, the netdev_oper_status can't be updated. How did you do it? Add two cases in test_lag_2.py to check if the status of lag interfaces is in sync with state_db. test_lag_db_status is to check if status of lag's member interfaces is synced with port's netdev_oper_status in state_db. It can run on any topology. 1). Check if status of interface is in sync with state_db after bootup. 2). Check if status of interface is in sync with state_db after shutdown/no shutdown. test_lag_db_status_with_po_update is to check if status of lag's member interfaces is synced with port's netdev_oper_status in state_db with add/delete operation for lag member. It only supports t1-lag topology. Remove port member from portchannel Shutdown this port to check if status is down Add this port back into portchannel and check if status is synced Retrieve lag_facts after shutdown interface and check if status is synced No shutdown this port to check if status is up Retrieve lag_facts after no shutdown interface to check if status is synced Since there are so many interfaces in t1-lag, we can define --completeness_level confident to pick up some interface to run it. How did you verify/test it? run test_lag_2.py::test_lag_db_status and test_lag_2.py::test_lag_db_status_with_po_update Signed-off-by: Zhaohui Sun <[email protected]> commit f7ea48387b0c294e1f64ba1275b5977ff3f15d96 Author: jingwenxie <[email protected]> Date: Mon Jul 18 15:39:44 2022 +0800 Skip GCU tests in platform with duplicate lanes config (#5985) Summary: Temp skip GCU tests in platform with duplicate lanes config. What is the motivation for this PR? Some platforms contain ports that share lanes which will fail GCU verification. Temporary skip to avoid nightly failure and push a fix later. commit a0be1192c04ba0b13969acbd300107aa311bf699 Author: ppikh <[email protected]> Date: Mon Jul 18 09:07:31 2022 +0300 [conditional_mark] Improved import logic in conditional_mark plugin (#5953) What is the motivation for this PR? To avoid usage of unneeded plugins How did you do it? Fixed import logic, now it just do: from tests.common.testbed import TestbedInfo How did you verify/test it? Executed test which should be skipped and another which should not be skipped. Checked that logic works(test which should be skipped - skipped, another test executed) commit e20c09ec8c56df933eda750022a1565dfa06714f Author: Nana@Nvidia <[email protected]> Date: Mon Jul 18 12:44:16 2022 +0800 Change the sleep to wait until (#5949) What is the motivation for this PR? Fix the test failure caused by the sleep time is not enough. Failed: "crm_stats_ipv4_neighbor_available" counter was not incremented Failed: "crm_stats_nexthop_group_available" counter was not incremented How did you do it? Use wait until instead of sleep when check the states in test_crm.py How did you verify/test it? Run the test_crm.py test serveral times , and it pass commit 1a8f829053e3e6040ed7b72a4bd42920b95a59e7 Author: vperumal <[email protected]> Date: Mon Jul 18 08:23:13 2022 +0530 Updating print command in test_lag_2.py (#5685) Co-authored-by: Perumal Venkatesh <[email protected]> commit 031510b6c2a85e14d6d85c6fd3e9ac1fd840a8dc Author: Andrii-Yosafat Lozovyi <[email protected]> Date: Sun Jul 17 06:56:25 2022 +0300 [auto-ts] Updated regex in test_auto_techsupport (#5924) Summary: In PR - 10433 were added changes that add automatic techsupport invocation in case memory usage is too high. After this PR output of Cli cmd's (show auto-techsupport global, show auto-techsupport-feature, show auto-techsupport history) changed and TC's are failing to parse the output on SONiC images SONiC.master.112803-dirty-20220621.160636 or higher. This PR updates the regex to properly parse the output in both cases (when TC will run on Sonic image that doesn't include changes in 10433 or in case when Sonic image have this changes SONiC.master.112803-dirty-20220621.160636 or higher) What is the motivation for this PR? Update the regex to properly parse the output of cli cmd's on newer SONiC images Signed-off-by: Andrii-Yosafat Lozovyi <[email protected]> commit a6ecc5c2a4c4d543dd25c8d9cf53bcf01752229f Author: Ying Xie <[email protected]> Date: Sat Jul 16 17:09:19 2022 -0700 [mux config] make sure mux config is consistent (#5989) What is the motivation for this PR? Some tests save configuration during tests, if they happen to change mux configuration, then they could cause 2 ToRs having different saved mux configurations, e.g. one ToR manual and the other ToR Auto. This won't cause issue until another test issues config reload on both ToRs. Then the mux configuration will be different, and the previous sanity check is unable to recover. How did you do it? save mux config after restoring auto mode. display the difference of mux config when sanity check fails. restore mux config to auto on both ToRs when sanity check fails. How did you verify/test it? Run test on a testbed with inconsistent configurations, the sanity check is now able to recover the testbed. Run test toggles the mux configuration. Verity that the saved configuration is consistent after the test. Signed-off-by: Ying Xie [email protected] commit 7d3cd5126b7935d3d1389deeb183178e7897cb14 Author: Longxiang Lyu <[email protected]> Date: Sat Jul 16 04:45:11 2022 +0800 [dualtor-mixed] Fix mux config minigraph generation issue (#5973) What is the motivation for this PR? On dualtor-mixed testbeds, the SoC IPs and server IPs have some overlaps: port state ipv4 ipv6 cable_type soc_ipv4 ---------- ------- --------------- ----------------- ------------- --------------- Ethernet4 auto 192.168.0.2/32 fc02:1000::2/128 active-active 192.168.0.3/32 Ethernet8 auto 192.168.0.3/32 fc02:1000::3/128 active-active 192.168.0.5/32 Ethernet12 auto 192.168.0.4/32 fc02:1000::4/128 active-active 192.168.0.7/32 Ethernet16 auto 192.168.0.5/32 fc02:1000::5/128 active-active 192.168.0.9/32 Signed-off-by: Longxiang Lyu [email protected] How did you do it? Use mux_cable_facts as the only resource to get both server IP and soc IP. How did you verify/test it? Run deploy-mg on both dualtor and dualtor-mixed testbeds. Signed-off-by: Longxiang Lyu <[email protected]> commit 18001afe050be78ee2512a6752c5a7812c43b665 Author: wenyiz2021 <[email protected]> Date: Fri Jul 15 11:24:10 2022 -0700 [MASIC] [master] Skip gbsyncd in modify_syslog_rate_limit (#5963) * Update sonic.py Add gbsyncd in DEFAULT_ASIC_SERVICES as it is new default service in master image * Revert back the adding of gbsyncd to DEFAULT_ASIC_SERVICES, as gearbox json file is missing, causing gbsyncdmgrd exited. If adding this, sanity check won't pass because gbsyncd processes are not running * Skip gbsyncd for modify_syslog_rate_limit * skip for multi-asic duts only, do not impact single-asic * Remove added empty line Co-authored-by: root <root@wenyi-vm-01.pdewprncsagu3n1l4b35v5n55g.cx.internal.cloudapp.net> commit 478d765d028e63a1935d594d89e33c7e2561b467 Author: siqbal1986 <[email protected]> Date: Fri Jul 15 11:12:53 2022 -0700 Update tests/bfd/test_bfd.py acceptable Co-authored-by: Ihor Chekh <[email protected]> commit dbf2c44c392052c8a43e733ff8fcb7700a850a97 Author: Antonh <[email protected]> Date: Fri Jul 15 17:09:42 2022 +0300 [test_route_perf] increase route change check timeout (#5940) Summary: The Dynamic Inner Hashing tests were sensitive to the order in a teardown. Sometimes the order of fixtures was wrong. This led to the failure of the next tests. To add stability were changed the setup and teardown methods were. To use the functions instead of fixtures and config reload for all system cleaning. What is the motivation for this PR? stabilize the test How did you do it? changed the setup and teardown methods Signed-off-by: Anton <[email protected]> commit f7bde1e9403ce0dfb89638a7ec8026ef01c32408 Author: Andrii-Yosafat Lozovyi <[email protected]> Date: Fri Jul 15 05:32:10 2022 +0300 Fix test_route_flap case (#5773) What is the motivation for this PR? Fix issue when TC was executed on t1 topo and fix withdrawn and announce operations. How did you do it? How did you verify/test it? Run test case on t0, t1, t1-lag topology: route/test_route_flap.py::test_route_flap PASSED Signed-off-by: Andrii-Yosafat Lozovyi <[email protected]> commit 23840716d7ba6614f2d6892c93c697032caa85db Author: tjchadaga <[email protected]> Date: Thu Jul 14 16:51:52 2022 -0700 Add test cases for TSA reload and load_minigraph (#5977) commit 3f10e7d5febd85da0e6da151a9fcfb65453c8d69 Author: abdosi <[email protected]> Date: Thu Jul 14 14:29:36 2022 -0700 Added Support of attribute <Bandwidth> for external link to support (#5955) What/Why I did: Added Support of attribute Bandwidth for external link to support port speed change operation. Ex: port_config.ini (HWSKU default speed) specify port speed as 100G which is mapped to Speed attribute of DeviceInfo but the actual connection is 40G which will get mapped to Bandwidth attribute of Png Section. Value of Bandwidth attribute is coming from device connection (eg between dut and fanout) which will have actual configured speed of the link. How I verify:- Generated the minigraph with this change and did load minigraph and make sure port speed change is taken effect. commit 6d00d44aca81fa6666b53cafb0fffbc6b77cfdaf Author: wenyiz2021 <[email protected]> Date: Thu Jul 14 09:42:38 2022 -0700 [MASIC] Test route perf support for multi-asic (#5820) * Update test_route_perf.py to call per random asichost for multi-asic duts * utility functions move to multi_asic.py to work for both multi/single asic * Utility functions execute per asichost * Parser function get_crm_resources for multi-asic * Fix indent * Fix indent * remove redundant function * Update test_route_perf.py * Update sonic_asic.py * Add back original function deleted by mistake * Update multi_asic.py * Remove empty lines added * Update test_route_perf.py fix enum_rand_one_frontend_asic_index * Update test_route_perf.py fix the new change using asichost.count_routes(ROUTE_TABLE_NAME) * Update test_route_perf.py * Add back try block Co-authored-by: Ubuntu <wenyi@wenyi-vm-01.pdewprncsagu3n1l4b35v5n55g.cx.internal.cloudapp.net> commit c3059eaf55d10f611404f57e0a8cc9d6ba6fad00 Author: Ye Jianquan <[email protected]> Date: Thu Jul 14 09:37:56 2022 -0700 [test_configurable_drop_counters]Ignore loganalyzer exceptions (#5804) What is the motivation for this PR? Fix loganalyzer exception of test_configurable_drop_counters. How did you do it? For the error log 1-3: I checked the trigger of the logs, likely it's because of the double delete([sonic-clear fdb all and arp], swssconfig ) of the cleanup commands. The second delete command(swssconfig) is used to remove the static mac record in fdb, I can see a certain record is deleted even though it's saying can't delete a non-exist record. It may be related to the async executing of swssconfig and sonic-clear(https://github.com/sonic-net/SONiC/wiki/SONiC-Clear-FDB-CLI-Design), if I execute 'show arp' and 'show mac' between the two cleanups, the exception will not be raised. Since a double delete has no bad effect on DUT, we can safely ignore it. For the error log 4-5: It should be related to debug things, we can safely ignore it too. So we can ignore the 5 error logs. How did you verify/test it? Run on physical testbeds drop_packets/test_configurable_drop_counters.py::test_neighbor_link_down[PORT_INGRESS_DROPS-L3_EGRESS_LINK_DOWN] PASSED [ 16%] drop_packets/test_configurable_drop_counters.py::test_neighbor_link_down[SWITCH_INGRESS_DROPS-L3_EGRESS_LINK_DOWN] SKIPPED [ 33%] drop_packets/test_configurable_drop_counters.py::test_dip_link_local[PORT_INGRESS_DROPS-DIP_LINK_LOCAL] PASSED [ 50%] drop_packets/test_configurable_drop_counters.py::test_dip_link_local[SWITCH_INGRESS_DROPS-DIP_LINK_LOCAL] SKIPPED [ 66%] drop_packets/test_configurable_drop_counters.py::test_sip_link_local[PORT_INGRESS_DROPS-SIP_LINK_LOCAL] PASSED [ 83%] drop_packets/test_configurable_drop_counters.py::test_sip_link_local[SWITCH_INGRESS_DROPS-SIP_LINK_LOCAL] SKIPPED [100%] commit 8c6228174a5fc7b9331c7d90490a8a560ec1d120 Author: Xichen96 <[email protected]> Date: Thu Jul 14 23:18:03 2022 +0800 [platform_tests]Fix psu test for mgmttstor (#5968) * Fix psu test for mgmttstor MgmtTsToR has 4 power outlets but only 2 is visible from the DUT. The other 2 outlets are reserved for C0 device. Assume last 2 outlets are for C0 on MgmtTsToR Signed-off-by: Xichen Lin <[email protected]> commit 005ca69468a6816188841d3f02608998a5a3ccdb Author: Nana@Nvidia <[email protected]> Date: Thu Jul 14 23:17:12 2022 +0800 Fix the test_po_update_io_no_loss test unstable issue (#5979) In the original code, the value of the cmd is: ip -4 route show | grep -w 10\\.0\\.0\\.25 | wc -l it will also match the "193.10.0.0/25 proto bgp src 10.1.0.32 metric 20 " After the code changed the cmd value will be ip -4 route show | grep -w 10\\\\.0\\\\.0\\\\.25 | wc -l then it will only match the "nexthop via 10.0.0.25 dev PortChannel1020 weight 1" which is expected commit 57a13e6a97d0bedf7ff114ea1729618132ff7af0 Author: xwjiang-ms <[email protected]> Date: Thu Jul 14 22:56:20 2022 +0800 Add start and end timestamp in test report xml (#5918) What is the motivation for this PR? Add start and end timestamp for testcases would help analysis time cost of testcases, and then help optimize testcases. How did you do it? 1. Record start/end timestamp in runtest makereport, and append to user_properties, so we could get start/end timestamp in test report xml 2. Change junit xml parser to support new xml and json files. Since new properties are in case level, so we consider them as new attributes, which is better to show in kusto How did you verify/test it? Run a test. Any platform specific information? Supported testbed topology if it's a new test case? commit 63fb8f665027cc08303443f1f6cba94f01352e89 Author: Oleksandr Kozodoi <[email protected]> Date: Thu Jul 14 09:51:52 2022 +0300 Added barefoot asic type to conditional mark of sub_port_interfaces TC (#5797) What is the motivation for this PR? Sub port interfaces are supported on Barefoot ASIC. So was added barefoot ASIC type to conditional mark of sub_port_interfaces TC according to changes from #5721 How did you do it? Added barefoot asic type to tests_mark_conditions.yaml file for sub_port TC Signed-off-by: Oleksandr Kozodoi <[email protected]> commit 7ae74e4a857e141fd3b5d14b9f53b65e8800b170 Author: Vaibhav Hemant Dixit <[email protected]> Date: Wed Jul 13 17:35:44 2022 -0700 [advanced-reboot] Add upgrade path data to report and support upload to Kusto (#5976) Append upgrade path data to advanced-reboot test report. Make Kusto upload generic to allow both reboot and upgrade reports upload. At present only reboot data is uploaded to Kusto. To allow better analytics, add upgrade path attributes to report. Added base, target image versions to report. Make report uploader check generic to accept upgrade path report upload. Tested on a physical testbed on AZP pipeline. commit 4f1cee746570d6f4a2ee70007066f3b1a0afbdc2 Author: Ye Jianquan <[email protected]> Date: Wed Jul 13 10:48:35 2022 -0700 [ErrorLog] Ignore teamd portchannel and fdb error log (#5951) Summary: Use loganalyzer ignore marker to ignore 'config load' error logs. Ignore LAG and FDB related error logs found in test case sub_port_interfaces/test_sub_port_interfaces.py::test_admin_status_down_disables_forwarding Jul 8 07:00:39.302949 foo-DUT ERR teamd#tlm_teamd: :- process_add_queue: Can't connect to teamd after 10 attempts. LAG 'PortChannel1.20' Jul 8 07:28:02.105624 foo-DUT ERR swss#orchagent: :- update: Failed to get port by bridge port ID 0x3a00000000083f. How did you do it? For error log 1, Jul 8 07:00:39.302949 foo-DUT ERR teamd#tlm_teamd: :- process_add_queue: Can't connect to teamd after 10 attempts. LAG 'PortChannel1.20' It's raised after the test case creates some portchannel-sub-interfaces. But the portchannel-sub-interfaces have been created successfully: admin@foo:~/tmp/sub_port_interfaces$ show ip int Interface Master IPv4 address/mask Admin/Oper BGP Neighbor Neighbor IP --------------- -------- ------------------- ------------ -------------- ------------- Ethernet80.10 10.0.0.56/31 up/up ARISTA01BT1 10.0.0.57 Ethernet84.10 10.0.0.58/31 up/up ARISTA02BT1 10.0.0.59 Ethernet88.10 10.0.0.60/31 up/up ARISTA03BT1 10.0.0.61 Ethernet92.10 10.0.0.62/31 up/up ARISTA04BT1 10.0.0.63 Ethernet96.10 10.0.0.64/31 up/up ARISTA05BT1 10.0.0.65 Ethernet100.10 10.0.0.66/31 up/up ARISTA06BT1 10.0.0.67 Ethernet104.10 10.0.0.68/31 up/up ARISTA07BT1 10.0.0.69 Ethernet108.10 10.0.0.70/31 up/up ARISTA08BT1 10.0.0.71 Ethernet112.10 10.0.0.72/31 up/up ARISTA09BT1 10.0.0.73 Ethernet116.10 10.0.0.74/31 up/up ARISTA10BT1 10.0.0.75 Ethernet120.10 10.0.0.76/31 up/up ARISTA11BT1 10.0.0.77 Ethernet124.10 10.0.0.78/31 up/up ARISTA12BT1 10.0.0.79 Loopback0 10.1.0.32/32 up/up N/A N/A PortChannel1.20 172.16.0.1/30 up/up N/A N/A PortChannel1.30 172.16.0.5/30 up/up N/A N/A PortChannel1.40 172.16.0.9/30 up/up N/A N/A PortChannel1.50 172.16.0.13/30 up/up N/A N/A PortChannel2.20 172.16.4.1/30 up/up N/A N/A PortChannel2.30 172.16.4.5/30 up/up N/A N/A PortChannel2.40 172.16.4.9/30 up/up N/A N/A PortChannel2.50 172.16.4.13/30 up/up N/A N/A Vlan1000 192.168.0.1/21 up/up N/A N/A docker0 240.127.1.1/24 up/down N/A N/A eth0 10.64.247.28/23 up/up N/A N/A lo 127.0.0.1/16 up/up N/A N/A admin@foo:~/tmp/sub_port_interfaces$ ping 172.16.0.1 PING 172.16.0.1 (172.16.0.1) 56(84) bytes of data. 64 bytes from 172.16.0.1: icmp_seq=1 ttl=64 time=0.095 ms 64 bytes from 172.16.0.1: icmp_seq=2 ttl=64 time=0.078 ms 64 bytes from 172.16.0.1: icmp_seq=3 ttl=64 time=0.124 ms According to： Azure/sonic-buildimage#11218 I think we can safely ignore it in the sub port test case. For error log 2, Jul 8 07:28:02.105624 foo-DUT ERR swss#orchagent: :- update: Failed to get port by bridge port ID 0x3a00000000083f. It's raised after removing a member interface from the vlan, because the deleted interface loses its fdb entry. As for the log analyzer ignore marker in the 'config load', I've to do that slight refactor, otherwise we can't only ignore 2 patterns in ignore_regex_list, but also include all error logs caused by 'config load' How did you verify/test it? Run it on physical testbed: === Running tests in groups === /usr/local/lib/python2.7/dist-packages/ansible/parsing/vault/__init__.py:44: CryptographyDeprecationWarning: Python 2 is no longer supported by the Python core team. Support for it is now deprecated in cryptography, and will be removed in the next release. from cryptography.exceptions import InvalidSignature ================================================================================================================== test session starts =================================================================================================================== platform linux2 -- Python 2.7.17, pytest-4.6.5, py-1.10.0, pluggy-0.13.1 ansible: 2.8.12 rootdir: /var/src/sonic-mgmt-int/tests, inifile: pytest.ini plugins: celery-4.4.7, forked-1.3.0, metadata-1.11.0, xdist-1.28.0, html-1.22.1, repeat-0.9.1, ansible-2.2.2 collecting ... /usr/local/lib/python2.7/dist-packages/ansible/parsing/vault/__init__.py:44: CryptographyDeprecationWarning: Python 2 is no longer supported by the Python core team. Support for it is now deprecated in cryptography, and will be removed in the next release. from cryptography.exceptions import InvalidSignature collected 1 item sub_port_interfaces/test_sub_port_interfaces.py::TestSubPorts::test_admin_status_down_disables_forwarding[port_in_lag] PASSED [100%] --------------------------------------------------------------------------------------------- generated xml file: /var/src/sonic-mgmt-int/tests/logs/tr.xml ---------------------------------------------------------------------------------------------- =============================================================================================================== 1 passed in commit 8028f0d473373f84e89c12898304379e6e00f272 Author: Ye Jianquan <[email protected]> Date: Wed Jul 13 09:03:17 2022 -0700 [RDMA&SNAPPI] Add snappi variables (#5971) What is the motivation for this PR? Fixes snappi related variables not found issue. How did you do it? Add definition of snappi related variables. How did you verify/test it? Run it on physical testbeds, and it works well commit 830d17d2a0404cf841cb5ab9b44489b6a1207a42 Author: Hua Liu <[email protected]> Date: Wed Jul 13 10:05:24 2022 +0800 Replace py-swsssdk with swsscommon (#5926) ### Description of PR Replace sonic-py-swsssdk with sonic-swss-common. ### Type of change - [ ] Bug fix - [x] Testbed and Framework(new/improvement) - [ ] Test case(new/improvement) ### Back port request - [ ] 201911 - [ ] 202012 ### Approach #### What is the motivation for this PR? deprecate sonic-py-swsssdk and switch to sonic-swss-common. #### How did you do it? Replace sonic-py-swsssdk with sonic-swss-common. #### How did you verify/test it? Pass all UT. #### Any platform specific information? #### Supported testbed topology if it's a new test case? ### Documentation <!-- (If it's a new feature, new test case) Did you update documentation/Wiki relevant to your implementation? Link to the wiki page? --> commit 57f404681cbe187ba444d064874cea0b4460472d Author: Zhaohui Sun <[email protected]> Date: Wed Jul 13 08:29:51 2022 +0800 Add rand_one_dut_hostname for fixtures in test_critical_process_monitoring (#5966) What is the motivation for this PR? process_monitoring.test_critical_process_monitoring.test_monitoring_critical_processes always failed with this error “Failed: Failed to find program 'lldp-syncd' in container 'lldp'”. The issue persists for these conditions: Topo: dualtor Image: all, including 202012, 202205 and master DUT: Only happened on Lower Tor It’s easy to reproduce: Login lower Tor, docker exec lldp supervisorctl status, get the lldpd’s PID Kill lldpd with this command: docker exec lldp kill -SIGKILL 24 Run docker exec lldp supervisorctl status again, it threw error xmlrpc.client.Fault, lldpmgrd exited and lldpd container restarted. admin@str2-7050cx3-acs-09:$ docker exec lldp supervisorctl status error: <class 'xmlrpc.client.Fault'>, <Fault 6: 'SHUTDOWN_STATE'>: file: /usr/lib/python3.7/xmlrpc/client.py line: 656 admin@str2-7050cx3-acs-09:$ docker exec lldp supervisorctl status Error response from daemon: Container b95505bce7db735617b96eeb4c1c962daf3b1444f9d8c6949d142a144fa1a788 is not running How did you do it? Fixtures in test_critical_process_monitoring select duthost, but in TC test_monitoring_critical_processes, it selects duthost = duthosts[rand_one_dut_hostname]. If it randomly chooses Lower Tor, auto_restart is not disable for container and container will restart, the test case will fail. So, add rand_one_dut_hostname for fixtures in test_critical_process_monitoring to keep duthost is same for fixtures and TC. How did you verify/test it? Run process_monitoring/test_critical_process_monitoring.py::test_monitoring_critical_processes Signed-off-by: Zhaohui Sun <[email protected]> commit ce6e0bab4034d13905d5c1a4751a12f44171482d Author: Zhijian Li <[email protected]> Date: Wed Jul 13 08:22:22 2022 +0800 [platform] revert skip oom test for celestica-e1031 (#5965) What is the motivation for this PR? Revert the skip of `platform_tests.test_memory_exhaustion` for celestica-e1031. Only one device fails on this test case, we will continue run it on this platform. How did you do it? Update configurations in `tests/common/plugins/conditional_mark/tests_mark_conditions_platform_tests.yaml`. How did you verify/test it? Verified on physical testbeds. Signed-off-by: Zhijian Li <[email protected]> commit 3fa8b1ba3895f548c479a928d509c6f1ebb868c4 Author: jingwenxie <[email protected]> Date: Wed Jul 13 07:53:38 2022 +0800 [tests/generic_config_updater] Add a regex to unblock nightly test (#5964) Summary: This is to unblock nightly PR check failure. What is the motivation for this PR? Recently GCU portchannel interface test randomly failed thus resulting in PR check failure. How did you do it? Add the error message to ignored regex. commit d35c8660e065f323d8c87cabc4327fdc2d46bd74 Merge: 0b548767 56e75df1 Author: Shahzad Iqbal <[email protected]> Date: Tue Jul 12 16:12:21 2022 -0700 Merge branch 'bfd_test_update' of https://github.com/siqbal1986/sonic-mgmt into bfd_test_update commit 0b548767579cbfe0745fad666c3b59d22588e5b9 Author: Shahzad Iqbal <[email protected]> Date: Tue Jul 12 16:11:55 2022 -0700 Fixed an statement bug. commit 56e75df169ac3e274413230a05022509fe570aaa Merge: 9abfcb7d bbd05164 Author: siqbal1986 <[email protected]> Date: Tue Jul 12 15:51:29 2022 -0700 Merge branch 'Azure:bfd_test_update' into bfd_test_update commit 9abfcb7dc5859060177bef3749266eb46e6b470c Author: Shahzad Iqbal <[email protected]> Date: Tue Jul 12 15:50:03 2022 -0700 Updated the BFD test to perform the following. - BFD Session scaling to 128 sessions. - Added v6 scale test along with v4. - Fixed minor issues reaised in review. commit bbd051640e355473de8e045730ed8d81cf9d46d7 Author: Shahzad Iqbal <[email protected]> Date: Tue Jul 12 15:40:59 2022 -0700 Updated the BFD test to perform the following. - BFD Session scaling to 128 sessions. - Added v6 scale test along with v4. - Fixed minor issues reaised in review. commit 26d53370ff7b0887c43776b0f068e327198cbc4b Author: bingwang-ms <[email protected]> Date: Wed Jul 13 01:31:12 2022 +0800 Add 2 test cases for tunnel QoS remapping (#5923) This PR is to add two test cases for verifying tunnel QoS remapping. Test plan: #5508 What is the motivation for this PR? This PR is to add two test cases for tunnel QoS remapping. * TC1 test_encap_dscp_rewrite The test is to verify the dscp rewriting of encapped packets. * TC2 test_bounced_back_traffic_in_expected_queue The test case is to verify the encapped packet is mapped to the correct queue How did you verify/test it? Verified on a dualtor testbed. collected 2 items qos/test_tunnel_qos_remap.py::test_encap_dscp_rewrite PASSED [ 50%] qos/test_tunnel_qos_remap.py::test_bounced_back_traffic_in_expected_queue PASSED [100%] commit f425ce127323f084d9fb6e29ba8c8af6fdfc3544 Author: Zhaohui Sun <[email protected]> Date: Tue Jul 12 18:18:00 2022 +0800 Support ubuntu 22.04 in deployment of testbed (#5917) What is the motivation for this PR? Support ubuntu 22.04 in deployment of testbed. Signed-off-by: Zhaohui Sun <[email protected]> commit 316d6f7b3f68fa411bb8aabc046fb1d7d9af83a7 Author: Oleksandr Kozodoi <[email protected]> Date: Tue Jul 12 12:45:56 2022 +0300 Updated approach for installing nanomsg in copp TC (#5962) Summary: Updated checking availability of the nanomsg in the syncd container Added check Debian version before installation of nano Added approach for disabling bgp before installation of nano What is the motivation for this PR? syncd container was upgraded to Debian 11 Bullseye. test_copp uses Python 2 to install nano in syncd container, but Debian 11 doesn't support Python 2. #5905 PR of sonic-mgmt added approach for installing the correct version of Python, but this doesn't cover the case when python isn't available. So that approach was updated. How did you do it? Updated approach for installing nanomsg in copp TC Signed-off-by: Oleksandr Kozodoi <[email protected]> commit 3cb320972e6b4518080917b5a3b6c5d28d2005d8 Author: jingwenxie <[email protected]> Date: Tue Jul 12 12:49:04 2022 +0800 [tests/override_config_table] Handle dualTor test (#5958) Summary: Make DUT identical when testing on dualTor case. commit 44b1256bf1c09a108527f4ece6182cc43908e61b Author: Zhijian Li <[email protected]> Date: Tue Jul 12 11:04:34 2022 +0800 [test_memory_exhaustion] Fix test failure on DUTs with few free memory What is the motivation for this PR? Some DUTs with few free memory (eg. Celestica-E1031) may reboot before ansible receive the result of `tail /dev/zero`, so we add `sleep 5` to ensure ansible receive the result first. How did you do it? Add `sleep 5` to ensure ansible receive the result before DUT reboot. How did you verify/test it? Verified on Celestica-E1031. Signed-off-by: Zhijian Li <[email protected]> commit a10f1cb8f8f800d5b9dc65f67021980f9dd38ebc Author: Stephen Sun <[email protected]> Date: Tue Jul 12 05:52:14 2022 +0800 Support CLI option to specify the port to test (#5948) Signed-off-by: Stephen Sun <[email protected]> commit 8c2320b2d0476237fb9082e4223692762b8cdce2 Author: Neetha John <[email protected]> Date: Mon Jul 11 14:07:18 2022 -0700 [pfcwd] For zero buffer pfcwd detection logic, verify forward action on Rx (#5665) Signed-off-by: Neetha John <[email protected]> For platforms that use the zero buffer detection logic for pfcwd, modify the testcase to check for ingress traffic getting forwarded. Related to Azure/sonic-swss#2279 How did you verify/test it? Ran the test with these changes on Mellanox platform and it passed commit f4ce5fb80445c40db51f7870bb3d9e34eee3c257 Author: jsanghra <[email protected]> Date: Mon Jul 11 06:49:52 2022 -0700 update pg_watermark_test (#5851) commit ff51d2993af173255aa60630ad6907ad7258a49e Author: Zhaohui Sun <[email protected]> Date: Mon Jul 11 21:29:48 2022 +0800 Revert "Check the version of python in syncd container then run different command (#5905)" (#5961) This reverts commit 79f9277da1b85183c36eb70f23da204fe2cf60f6. commit 935329ba0aa46029ced5f72aaa077e2160514e4f Author: StormLiangMS <[email protected]> Date: Sun Jul 10 22:31:54 2022 -0700 support docker registry (#5929) What is the motivation for this PR? For the QoS test, need to swap syncd during run time, sometimes it is necessary to use the other docker registry for different image versions. In current framework, we can specify the docker registry host per inventory but not per image. In this PR, to add a new yml file which can give an option for a different docker registry only for the QoS swap syncd. How did you do it? How did you verify/test it? PASSED the nightly test to do the swap syncd with public image Any platform specific information? Supported testbed topology if it's a new test case? commit 6c323689b94e808d922a00221efa811ce3c55fbb Author: Ying Xie <[email protected]> Date: Sat Jul 9 07:55:25 2022 -0700 [testbed] add wrapper for redeployinng testbed (#5945) What is the motivation for this PR? Add a wrapper for redeploy a testbed. How did you do it? Calling remove-topo then add-topo. How did you verify/test it? Redeploy a testbed. Signed-off-by: Ying Xie [email protected] commit ee04112b43da01fbc5c5bc352cdacf76bd030b61 Author: Anton Ptashnik <[email protected]> Date: Fri Jul 8 23:44:30 2022 +0300 made a fix for platform_api test - test_chassis.py/test_components (#5496) Summary: fixed test_chassis/test_components platform_api test Fixes # (issue) In case duthost facts has no chassis components the test fails on len(None). Fixed that How did you verify/test it? py.test --inventory=../ansible/lab,../ansible/veos --testbed_file=../ansible/testbed.csv --module-path=../ansible/library -v -rA --topology=t1,any platform_tests/api/test_chassis.py -k test_components commit 9628483cf376859bd5362aeaa4e9b1e48c1169a0 Author: rraghav-cisco <[email protected]> Date: Fri Jul 8 10:37:50 2022 -0700 Addressing 5860. (#5873) commit cf17bddab87773436dc2642c7f860df9231097fe Author: SuvarnaMeenakshi <[email protected]> Date: Fri Jul 8 09:45:54 2022 -0700 Fix container_autorestart test case (#5911) What is the motivation for this PR? #5478 was done to support chassis where the change was done correctly in clear_failed_flag_and_restart function. How did you do it? Make changes in function to use the right service name for multi-asic platform. Make changes in post check function to check for all services in case of multi-asic platform. How did you verify/test it? Verified on multi-asic VS testbed. Tested on single-asic VS testbed. commit 0f1faa27bdb359755f34e09f02b89c41fa76e37f Author: Ye Jianquan <[email protected]> Date: Fri Jul 8 23:53:18 2022 +0800 [RDMA&SNAPPI] Enable snappi test cases in nightly run (#5939) What is the motivation for this PR? Enable snappi test cases on tgen testbeds How did you do it? Enable snappi test cases on tgen testbeds How did you verify/test it? Enable it and will verify it in the future nightly test runs. commit c81ce86c1ced6fbaaa93003d8907710b8919be56 Author: Ying Xie <[email protected]> Date: Fri Jul 8 08:43:42 2022 -0700 [setup container] add option to show setup container execution details (#5946) What is the motivation for this PR? Sometimes setup container takes long time and not sure where it was spending time. How did you do it? Add option -x to enable execution tracing. How did you verify/test it? run setup container script with -x option Signed-off-by: Ying Xie [email protected] commit 373d40fd5f0f72f969c76f005f38e3fb86d8883d Author: Longxiang Lyu <[email protected]> Date: Fri Jul 8 12:25:10 2022 +0800 [dualtor] Add sanity check support for `active-active` ports (#5941) Approach What is the motivation for this PR? Add sanity check support for active-active ports. How did you do it? 1. Start running icmp_responder at session-level is topology is of dualtor-mixed. 2. Check if the active-active port status is either active or standby. 3. Restart the nic_simulator service as recovery if the check fails. How did you verify/test it? Run pretest on both dualtor and dualtor-mixed testbeds commit d4decc671819eff09c52ca4d39a9c54e623b90ed Author: jingwenxie <[email protected]> Date: Fri Jul 8 09:12:51 2022 +0800 [tests/generic_config_updater] implement GCU test NTP_SERVER (#5803) Summary: Testcase of ntp server for generic updater apply-patch What is the motivation for this PR? End to End test support for Generic Updater apply-patch How did you do it? Modify ntp server config to dut and check if config change as expected How did you verify/test it? Run test of sonic-mgmt/tests/generic_config_updater/test_ntp.py on KVM commit f3766d9b04ff0a6ccd3665ff19bd86a88032dd45 Author: Oleksandr Kozodoi <[email protected]> Date: Fri Jul 8 03:22:41 2022 +0300 Moved skip reason of qos_config_updates test case (#5657) What is the motivation for this PR? Added support of other platforms to qos_config_updates test case of generic_config_updater test suite. How did you do it? Moved skip reason for qos_config_updates test case of generic_config_updater test suite from test module to test_mark_conditions YAML file. Also added list of hwsku for barefoot platform to the conditions. Signed-off-by: Oleksandr Kozodoi <[email protected]> commit b0672d2f8fae0574bea575406849d8f2696c89c8 Author: Longxiang Lyu <[email protected]> Date: Fri Jul 8 07:53:33 2022 +0800 [dualtor][minigraph] Support new minigraph schema for SoC IP (#5863) Approach What is the motivation for this PR? Support the minigraph generation for the new schema for SoC IP. Depends on: Azure/sonic-buildimage#11207 Signed-off-by: Longxiang Lyu [email protected] How did you do it? Add the SoC IP related information to Dpg How did you verify/test it? Test together with Azure/sonic-buildimage#11207 to verify the mux cable table generation Any platform specific information? commit 7d2e52379a76b5d793d910fb19690bb5cab03ba7 Author: roman_savchuk <[email protected]> Date: Fri Jul 8 02:39:26 2022 +0300 Added SAI_STP_ATTR_BRIDGE_ID to common ignore loganalyzer (#5792) Summary: When run test container auto restart we do see errors in loganalyzer "sonic WARNING syncd[24]: :- discover: skipping since it causes crash: SAI_STP_ATTR_BRIDGE_ID" This caused by SaiDiscovery.cpp Approach What is the motivation for this PR? Avoid loganalyzer failure due to "crash" in syslog message How did you do it? added message to common ignore file How did you verify/test it? Run container autorestart TC, TC passed commit a0e8e6b40f2d7e42f9e4110fc8907711ba36da5b Author: vdahiya12 <[email protected]> Date: Thu Jul 7 11:56:07 2022 -0700 [nic_simulator] add options to nic_simulator to support keep alive (#5935) Signed-off-by: vaibhav-dahiya <[email protected]> Signed-off-by: vaibhav-dahiya [email protected] This PR is meant to fix gRPC keepalive timeout values. With this change and adding the options to client side, the pings start working for TCP connection for all gRPC channels. This PR is also meant to have a restriction for minimum time between two consecutive pings from a client to 1 second, otherwise server will send a KEEP_YOUR_CALM type exception message. as documented in https://github.com/grpc/grpc/blob/master/doc/keepalive.md GRPC_ARG_HTTP2_MIN_RECV_PING_INTERVAL_WITHOUT_DATA_MS If there are no data/header frames being sent on the transport, this channel argument on the server side controls the minimum time (in milliseconds) that gRPC Core would expect between receiving successive pings. If the time between successive pings is less that than this time, then the ping will be considered a bad ping from the peer. Such a ping counts as a ‘ping strike’. On the client side, this does not have any effect. GRPC_ARG_HTTP2_MAX_PING_STRIKES This arg controls the maximum number of bad pings that the server will tolerate before sending an HTTP2 GOAWAY frame and closing the transport. Setting it to 0 allows the server to accept any number of bad pings commit a7543d6d39f7a83a256f3f8ccfaec631074eecd4 Author: jingwenxie <[email protected]> Date: Thu Jul 7 15:28:43 2022 +0800 [GCU] Improve loopback test in dualTor (#5933) Summary: This PR is to improve lo test to fit dualTor scenarios. What is the motivation for this PR? In dualTor scenarios, one Tor's Loopback0 is 10.1.0.33 which is not same as the default 10.1.0.32. Need to improve test to fit in that. How did you do it? Read loopback config from cfg_facts. How did you verify/test it? Run kvm test. commit a8e40092edcd2860b8d0f38bbb7a41324efa1979 Author: ppikh <[email protected]> Date: Thu Jul 7 09:46:50 2022 +0300 Updated SN2201 platform info (#5896) Updated SN2201 platform info Fixed HwSKU name Fixed PSU capabilities Fixed number of xSFP modules Added new sensor - "Ambient CPU Board Temp" - What is the motivation for this PR? Fix issue with number of xSPF ports and add support for new sensor - How did you do it? Change relevant files with wrong information previously used - How did you verify/test it? Executed platform tests Any platform specific information? Mellanox SN2201 only Signed-off-by: Petro Pikh <[email protected]> commit 8e7358e88e71d4cae27482c14f474fd6d504c9e2 Author: Yang Wang <[email protected]> Date: Thu Jul 7 13:50:45 2022 +0800 add sai adhoc api (#5925) commit 74a38241838de6a66d015ca1edb104c2eea3eecd Author: Richard.Yu <[email protected]> Date: Thu Jul 7 13:50:19 2022 +0800 [SAI-Test]Refactor structure for SAI test (#5919) * [SAI-Test]Refactor structure for SAI test Use different py file for different tests Add more comments Refactor code Test done: Run tests within local sonic-mgmt docker Signed-off-by: richardyu-ms <[email protected]> * refactor code Signed-off-by: richardyu-ms <[email protected]> * refactor Signed-off-by: richardyu-ms <[email protected]> * Update cases_brcm_t0.py add brcm t0 cases * Update sai_infra.py remove useless variable commit 38be43ac50de536e1a55c49e12ce22015aa96146 Author: Yutong Zhang <[email protected]> Date: Thu Jul 7 09:53:00 2022 +0800 Cache dut basic fact using `session.config.cache`. (#5932) Description of PR In order to skip test cases using conditional marks, we need to gather the basic facts of dut, which will cost some time. session.config.cache in pytest can support sharing values between py.test invocations. So we use this to cache the basic facts of dut. What is the motivation for this PR? In order to skip test cases using conditional marks, we need to gather the basic facts of dut, which will cost some time. session.config.cache in pytest can support sharing values between py.test invocations. So we use this to cache the basic facts of dut. How did you do it? Compare the tbname with cached tbname. If they are not the same testbed, clear the cache, get the new basic facts, and put the new tbname and basic facts in cache. If the tbname is same as cached tbname, if we already have dut basic facts in cache, use them ditectly, otherwise, get the basic facts and put them in cache. How did you verify/test it? Running test cases with/without cached tbname and dut basic facts. Signed-off-by: Yutong Zhang <[email protected]> commit 423452503b62996af478141d61ea8f1b1673a5ad Author: Oleksandr Kozodoi <[email protected]> Date: Thu Jul 7 03:43:30 2022 +0300 Added function for removing ACL table in sub-ports TC (#5543) What is the motivation for this PR? Update sub_port_interfaces TC according to changes from #2165 PR of sonic-swss How did you do it? Added function for removing ACL table when ingress/egress ACL binding is configured on LAG members. Signed-off-by: Oleksandr Kozodoi <[email protected]> commit 95e784006ae6e188d075d421018ad945793fdc2a Author: Nikola Dancejic <[email protected]> Date: Wed Jul 6 11:04:15 2022 -0700 [bgp_helpers] wait for swss to start when restarting bgp (#5915) What I did: added an assert to wait for swss to come back up after restarting bgp Why I did it: PR in sonic buildimage adds bgp as a dependent of swss. This was causing PR test failures due to swss not restarting in time in vms-kvm-t1-lag tests. Signed-off-by: Nikola Dancejic <[email protected]> commit 6f73c529b058ceac630351f42000e36af3c79ab2 Author: rraghav-cisco <[email protected]> Date: Wed Jul 6 00:33:04 2022 -0700 Adding support for updating ecn config file based on testcase. (#5850) This PR is to add the following for cisco 8000 platform only: Add support for enabling the ECN marking configs in the DUT. Update the test_dequeue_ecn.py to call the above functions. Restore the configs after the test is done. commit c8d13dbf0cb4b8bd8bf195df8755771c75ea8dca Author: Stephen Sun <[email protected]> Date: Wed Jul 6 14:21:21 2022 +0800 [Mellanox] Use the correct way to check branch in "check_sysfs" (#5913) Use the correct way to check the branch of DUT in "check_sysfs" - What is the motivation for this PR? Use the correct way to check the release of DUT in "check_sysfs" dut.os_version is generated from the branch name. There is no guarantee that branch name always includes the release name. For example, a developer can create a branch my-fix based on 202012 branch. In this case, the test will fail. - How did you do it? - How did you verify/test it? Run sonic-mgmt test which noticed this issue - Any platform specific information? Mellanox Signed-off-by: Stephen Sun <[email protected]> commit 1047e500c626cb0e53be58156e21fa18614cf73e Author: Andrii-Yosafat Lozovyi <[email protected]> Date: Wed Jul 6 04:49:16 2022 +0300 Fix test_dhcpv6_relay_counter, added src MAC to unknown DHCP pkt (#5912) What is the motivation for this PR? Fix dhcp_relay/test_dhcpv6_relay.py::test_dhcpv6_relay_counter TC Summary: dhcp_relay/test_dhcpv6_relay.py::test_dhcpv6_relay_counter failed when checked that counter for relayed Unknown DHCP message didn't increment after unknown pkt was sent How did you do it? How did you verify/test it? Run dhcp_relay/test_dhcpv6_relay.py::test_dhcpv6_relay_counter TC on t0 topo: PASSED dhcp_relay/test_dhcpv6_relay.py::test_dhcpv6_relay_counter Signed-off-by: Andrii-Yosafat Lozovyi <[email protected]> commit 26422604d185d28143e9cd442503ed4eeed2699c Author: wenyiz2021 <[email protected]> Date: Tue Jul 5 09:08:22 2022 -0700 [MASIC] [chassis] Test default route with bgp flap support for multi-asic (#5780) * Update sonic_asic.py * Update test_default_route.py * Update test_default_route.py * Update test_default_route.py * Update multi_asic.py * Update test_default_route.py * Update multi_asic.py * Refine for single-asic case * Refine test_default_route for single-asics * Update test_default_route.py * Update test_default_route.py * Update sonic_asic.py to only check eth0 on multi_asic host commit fd7c25f3978795680d268f2c5ed6c2832a52c496 Author: Xin Wang <[email protected]> Date: Tue Jul 5 09:39:57 2022 +0800 Add support of unconditional mark (#5906) If define something like below in the conditional mark yaml file: ```` test_example.py::test_case1: skip: ``` The conditional mark plugin would fail with something like below: ``` INTERNALERROR> File "/home/ubuntu/code/sonic-mgmt/tests/common/plugins/conditional_mark/__init__.py", line 347, in pytest_collection_modifyitems INTERNALERROR> mark_conditions = mark_details.get('conditions', None) INTERNALERROR> AttributeError: 'NoneType' object has no attribute 'get' ``` It would make more sense for the mark syntax to work. The meaning is obvious: unconditionally add 'skip' mark for the matching test case. This change addressed the issue to improve the conditional mark to support unconditionally add a mark. Signed-off-by: Xin Wang <[email protected]> commit d5b6103bc5a34d628fdfc1e950fd07edff385a66 Author: Dinesh Kumar Sellappan <[email protected]> Date: Mon Jul 4 18:16:42 2022 -0700 LACP Failover Test Scripts (#4851) Add/Remove links physically Add/Remove links from dut LACP Timer Effect commit 6c8d13f96cf52da4e8cc6e4bdd74778fe23c4d08 Author: sanmalho-git <[email protected]> Date: Mon Jul 4 21:09:42 2022 -0400 Skip generic_config_updater tests for T2 topology (#5898) What is the motivation for this PR? As per discussion with MSFT, the tests in generic_config_updater use YANG models which are not currently supported on a T2 chassis. So, need to skip these tests for T2 topology How did you do it? Adding check in tests_mark_conditional to skip generic_config_updater if 't2' in topology How did you verify/test it? Ran generic_config_updater against T2 topology and verified that all the tests are skipped. commit fb5ecc1c35e241e104c4c43d623504ef8348faf6 Author: Anton Ptashnik <[email protected]> Date: Tue Jul 5 04:08:39 2022 +0300 Fix sanity check failure after test_lag tests (#5897) What is the motivation for this PR? test_lag test stops routeCheck Monit service to prevent LogAnalyzer to fail the tests because of "missed_ROUTE_TABLE_routes" logs appearing when the test shutdowns neighbor links. The test starts routeCheck on finish but next tests may fail on sanity check because routeCheck did not have enough time to start. The root cause here is that when the test attempts to start routeCheck on test finish, the service does not start immediately. Monit runs the routeCheck on each 5th cycle, where 1 Monit cycle is set to 1 min by default. So, when the routeCheck is requested to start, it will start only on the next 5th Monit cycle. How did you do it? Fixed the issue by refusing to stop routeCheck and ignoring "missed_ROUTE_TABLE_routes" logs instead. routeCheck Monit config: /etc/monit/conf.d/sonic-host ``` check program routeCheck with path "/usr/local/bin/route_check.py" every 5 cycles ``` /etc/monit/monitrc ``` set daemon 60 ``` How did you verify/test it? py.test --inventory=../ansible/lab,../ansible/veos --testbed_file=../ansible/testbed.csv --module-path=../ansible/library -v -rA --topology=t0,any pc/test_lag_2.py commit f1c6bcededcc1d89dad11ce447070510a6c675b3 Author: Xin Wang <[email protected]> Date: Tue Jul 5 09:04:59 2022 +0800 Optimize DB dump collection for failed test case (#5910) What is the motivation for this PR? The autoused fixture for collecting DB dumps for failed cases need to run some redis-dump commands for each of the DBs. The original implementation called the ansible command module on DUT once for each of the redis-dump command. This is not efficient enough because of the extra ansible overheads. How did you do it? This change updated the code to use "shell_cmds" module to run all the redis-dump commands in a batch. This change also updated the dump file name to include test module name, not just the test case name which could conflict across different test scripts. How did you verify/test it? Tested on vSONiC setup, this change can decrease redis-dump execution time from 13 seconds to 6 seconds. Signed-off-by: Xin Wang <[email protected]> commit 91745a9bc63ffd3fd5dc8180203ca16bc0309c81 Author: tiejunzhang <[email protected]> Date: Tue Jul 5 09:03:24 2022 +0800 enhance to support wan topo (#5891) What is the motivation for this PR? Leverage sonic mgmt framework to support more complex topo and multi-vendor devices How did you do it? Leverage main process of sonic mgmt, and separate deployment logic from current sonic mgmt How did you verify/test it? Use T0 topo to verify code changes Co-authored-by: tiejunzhang <[email protected]> commit 6da07f8477cb8b7b27db3ea027273f85ea9da2c6 Author: Zhaohui Sun <[email protected]> Date: Mon Jul 4 09:18:48 2022 +0800 Fix wrong assert condition in delete_neighbor (#5916) What is the motivation for this PR? test_tunnel_memory_leak failed at assert in delete_neighbor with wrong condition. Neighbor was deleted but still threw exception. How did you do it? Fix wrong assert condition in delete_neighbor How did you verify/test it? run dualtor/test_tunnel_memory_leak Signed-off-by: Zhaohui Sun <[email protected]> commit 23c0aeea82dd9c31014b13b7d0993c805ce94731 Author: tjchadaga <[email protected]> Date: Fri Jul 1 23:07:26 2022 -0700 Skip TSA, TSB error check in no_neighbor test (#5914) commit 73f2dfbf174b3b5917ba8af2a497900703ce519b Author: Zhaohui Sun <[email protected]> Date: Fri Jul 1 19:29:34 2022 +0800 Enhance test_cacl_application to support dualtor (#5895) What is the motivation for this PR? test_cacl_application failed on dualtor, if it randomly selects standby tor to run test case, it will failed with this kind of error Failed: Unexpected iptables rules: set([u'-A DHCP -m mark --mark 0x67024 -j DROP', u'-A DHCP -m mark --mark 0x67012 -j DROP', u'-A DHCP -m mark --mark 0x67003 -j DROP', u'-A DHCP -m mark --mark 0x67002 -j DROP', u'-A DHCP -m mark --mark 0x67022 -j DROP', u'-A DHCP -m mark --mark 0x67020 -j DROP', u'-A DHCP -m mark --mark 0x67018 -j DROP', u'-A DHCP -m mark --mark 0x67014 -j DROP', u'-A DHCP -m mark --mark 0x67017 -j DROP', u'-A DHCP -m mark --mark 0x67006 -j DROP', u'-A DHCP -m mark --mark 0x67016 -j DROP', u'-A DHCP -m mark --mark 0x67019 -j DROP', u'-A DHCP -m mark --mark 0x67021 -j DROP', u'-A DHCP -m mark --mark 0x67008 -j DROP', u'-A DHCP -m mark --mark 0x67001 -j DROP', u'-A DHCP -m mark --mark 0x67010 -j DROP', u'-A DHCP -m mark --mark 0x67013 -j DROP', u'-A DHCP -m mark --mark 0x67011 -j DROP', u'-A DHCP -m mark --mark 0x67009 -j DROP', u'-A DHCP -m mark --mark 0x67007 -j DROP', u'-A DHCP -m mark --mark 0x67004 -j DROP', u'-A DHCP -m mark --mark 0x67015 -j DROP', u'-A DHCP -m mark --mark 0x67023 -j DR How did you do it? The failed reason is on standby tor, it will add one rule DHCP mark rule for each standby interface. Changes in this PR: Add a new fixture duthost_dualtor to return duthost for dualtor test case test_cacl_application_dualtor, add another new fixture expected_dhcp_rules_for_standby to generate expected dhcp mark rules for standby tor. Move pytest.skip to tests_mark_conditions.yaml Add from tests.common.dualtor.dual_tor_common import * , otherwise it will throw error of fixture 'cable_type' not found. How did you verify/test it? Run cacl/test_cacl_application.py cacl/test_cacl_application.py::test_cacl_application_nondualtor SKIPPED [ 16%] cacl/test_cacl_application.py::test_cacl_application_dualtor[active_tor] PASSED [ 33%] cacl/test_cacl_application.py::test_cacl_application_dualtor[standby_tor] PASSED [ 50%] cacl/test_cacl_application.py::test_multiasic_cacl_application[None] SKIPPED [ 66%] cacl/test_cacl_application.py::test_cacl_scale_rules_ipv4 PASSED [ 83%] cacl/test_cacl_application.py::test_cacl_scale_rules_ipv6 PASSED [100%] Any platform specific information? Supported testbed topology if it's a new test case? commit 79f9277da1b85183c36eb70f23da204fe2cf60f6 Author: Zhaohui Sun <[email protected]> Date: Fri Jul 1 16:25:53 2022 +0800 Check the version of python in syncd container then run different command (#5905) What is the motivation for this PR? test_copp failed on 202205 image, because python-pip doesn't exist. How did you do it? syncd container on 202205 image was upgraded to bullseyes on mellanox testbeds. test_copp uses python-pip to install nano in syncd container. Check python version before installation of nano, then run command accordingly. How did you verify/test it? Run copp/test_copp.py Signed-off-by: Zhaohui Sun <[email protected]> commit 642ac708365943d9db503d8d1498454dda6a8501 Author: Xin Wang <[email protected]> Date: Fri Jul 1 15:15:00 2022 +0800 Address the security alert of aiohttp used by test report uploading (#5903) What is the motivation for this PR? Github is giving security vulnerabilities alert for the aiohttp package used by test report uploading. How did you do it? This change addressed the security alert by removing the dependency of this aiohttp package by upgrading the packages depending on it to a higher version. The pytest package is only required for testing. It is not required for test report uploading. Moved it into a separate rquirements_dev.txt file. The README.md file was also updated. Currently the test report uploading tests are broken. Need to address them in separate PR. Signed-off-by: Xin Wang <[email protected]> commit ee963d25ea7d56e7983a24281d9cb65ab585fd61 Author: jingwenxie <[email protected]> Date: Fri Jul 1 09:17:56 2022 +0800 [GCU] Improve GCU test from invalid initial config (#5886) Summary: Improve GCU nightly test when initial config is invalid What is the motivation for this PR? GCU required the ConfigDB to pass YANG validation. But previous failing test may leave invalid residue to ConfigDB. This PR is to make GCU test coud be tested in this case. How did you do it? Check initial ConfigDB and reload with minigraph if it is invalid. How did you verify/test it? Run kvm test with config with YANG validation failed. commit a7430f6459c28fec85d30e1157d5f82a512a47be Author: bingwang-ms <[email protected]> Date: Fri Jul 1 08:50:32 2022 +0800 Restart swss after injecting `DualToR` subtype (#5893) * Restart swss after injecting DualToR subtype commit fba373189187ac0e9ce5741f48b91bfbf64bff05 Author: jingwenxie <[email protected]> Date: Fri Jul 1 08:40:53 2022 +0800 [GCU] Improve GCU test with reduced testing time (#5866) Summary: Improve GCU test with less running time; Resolve dualTor test failure. What is the motivation for this PR? GCU long running tests were skipped. This PR is to add them back with reduced runnning time. Change to choose the idential DUT for test in dualTor. How did you do it? Combining subtests into one test, thus eliminate the Log Analyzer running time. Change duthost fixture to rand_selected_dut. How did you verify/test it? Running kvm test commit 5898d35473fce04ab25ff865c3fec287d935b74e Author: Harish Kalyanaraman <[email protected]> Date: Thu Jun 30 16:02:45 2022 -0400 New T0 Test case to check connectivity of statically added arp entries (#4703) What is the motivation for this PR? This is a new T0 test case that addresses the following gap in coverage: There was an issue where devices connected to ports part of vlan on the DUT could not be reached from outside the VLAN subnet after static ARP/neighbor entries created and deleted multiple times for devices. There was no community test to catch this issue, hence adding it How did you do it? Setup: Get a VLAN ID of a vlan with at least 2 members, ipv4 and ipv6 of the vlan Randomized selection of 2 vlan members from the vlan and get its port index Generate 2 IPv4 and IPv6 addresses in the vlan's network using the port indices Add above IPs to ptf hosts matching the port index Randomized selection of neighbor VM host from the neighbor list Get VM ip details and add route on PTF Test Procedure: Add static arp entry of ipv4 of PTF host 1 and 2 on DUT Add static neighbor entry of ipv6 of PTF host 1 and 2 on DUT Check for connectivity from VM host Delete static neighbor entry of ipv6 of PTF host 1 and 2 on DUT Add static neighbor entry of ipv6 of PTF host 2 and then 1 on DUT Delete and re-add static arp entry of ipv4 of PTF host 1 and 2 on DUT Check for connectivity from VM host How did you verify/test it? Ran test with t0-52 topology commit 5c9cfaff49c5330ed09eccb9c1792c4a6fd12211 Author: Ze Gan <[email protected]> Date: Thu Jun 30 13:31:50 2022 +0800 [test_macsec]: neighbor to neighbor for EOS (#5888) What is the motivation for this PR? The neighbor to neighbor test of MACsec cannot adapt to EOS as neighbor. This PR is for adapting EOS topology. How did you do it? Use EOS commands to configure the route. Signed-off-by: Ze Gan <[email protected]> commit 14590b71a9c1788ea03a19cd7e077962aaaaa318 Author: Yutong Zhang <[email protected]> Date: Thu Jun 30 13:28:18 2022 +0800 Remove key "router_macs" in test_decap.py. (#5890) Description of PR The key router_macs is not defined in fixture setup_teardown, and it will case an error. In master branch, we no more need this key, so remove it. What is the motivation for this PR? The key router_macs is not defined in fixture setup_teardown, and it will case an error. In master branch, we no more need this key, so remove it. How did you do it? Remove the key router_macs in ptf_runner in test_decap.py. Signed-off-by: Yutong Zhang <[email protected]> commit f0b7fdb615e3980127b2637059f457c456e3afef Author: Yutong Zhang <[email protected]> Date: Thu Jun 30 13:27:19 2022 +0800 Fix IP decap on dualtor testbed. (#5885) Description of PR PR #5456 updated the return value of fixture single_fib_for_duts. It is no longer just True of False. The new value will be different strings. The IP_decap_test.py ptf script was not. What is the motivation for this PR? PR #5456 updated the return value of fixture single_fib_for_duts. It is no longer just True of False. The new value will be different strings. The IP_decap_test.py ptf script was not. How did you do it? This change udpated the IP_decap_test.py script accordingly. Signed-off-by: Yutong Zhang <[email protected]> commit ae729a35ebf13e161aa7260a2c58dcdc3fb3934f Author: Ashwin Srinivasan <[email protected]> Date: Wed Jun 29 12:06:16 2022 -0700 Modified code to discount PSU voltage value after turn-off (#5889) Since each vendor's implementation is different, the PSU's voltage value may be non-zero after shutdown. Additionally, the voltage might be non-zero for a few minutes even after shutdown on account of capacitor physics slowly discharging current and bringing the voltage to zero. Accounting for these discrepancies, and seeing as current flow is a much better indictor of the state of the PSU anyway, voltage value is disregarded in making a decision about the power-state of the PSU. commit cb6cd552edbaecc0ef49386e6049e642aa53c2de Author: Qi Luo <[email protected]> Date: Wed Jun 29 11:33:11 2022 -0700 Fix typo in comment and string (#5892) commit 01fb03cb07b761ec05c5363300da66faef0848c9 Author: xwjiang-ms <[email protected]> Date: Wed Jun 29 23:35:08 2022 +0800 Fix ptf log and pcap file can't be fetch issue (#5884) * Fix ptf log and pcap file can't be fetch issue What is the motivation for this PR? 1. ptf collect cannot fetch log and pcap file because it excute before log file generate 2. new log and pcap files will cover old files if filenames are the same defined in testcases How did you do it? 1. fetch after log and pcap files generate whether test fails or not 2. add timestamp to filename in docker, and files in ptf will not be changed How did you verify/test it? Run a test which would call ptf runner Any platform specific information? Supported testbed topology if it's a new test case? commit cde9e34df9fa10eb3270899cc2f9e6ecbb93e474 Author: roman_savchuk <[email protected]> Date: Wed Jun 29 07:40:10 2022 +0300 [ everflow ] enabled everflow per interfaces for barefoot asic (#5495) What is the motivation for this PR? Functional tested by everflow_per_interfaces TC has been added to Barefoot How did you do it? Removed asic type from skip list How did you verify/test it? Run everflow_per_interfaces TC. TC passed commit 3acc5463f118c7caee4186d63bb028fb67d20eba Author: Ze Gan <[email protected]> Date: Wed Jun 29 11:03:18 2022 +0800 [vm_topology]: Fixing split multiple times at retry stage (#5874) What is the motivation for this PR? In the first time, the cmdline or grep_cmd has been split to a list, but at the retry stage, the list object will be applied split operation again so that get the following error. 'list' object has no attribute 'read' How did you do it? Use another variables to record the original variables. Signed-off-by: Ze Gan <[email protected]> commit 21a6bac38630608450fb62ab8a043cdaa155caa6 Author: Junhua Zhai <[email protected]> Date: Wed Jun 29 01:30:03 2022 +0000 Add macsec counter test (#5848) commit 1b063c25657b086d3075193eae49aab13e0ca9f0 Author: ppikh <[email protected]> Date: Tue Jun 28 19:04:48 2022 +0300 [auto-techsupport] Stabilize auto-techsupport tests (#5756) Old logic expected that specific timestamp will be in last log file. Sometimes last log file could be not the same as we expected(near it it but different file with different old timestamp). Now instead of checking that oldest timestamp == expected oldest timestamp - we do check that oldest timestamp in [list of expected(possible) oldest timestamps] - What is the motivation for this PR? Stabilize tests, reduce test failures - How did you do it? Changed logic from: oldest timestamp == expected oldest timestamp to oldest timestamp in [ expected oldest timestamps ] - How did you verify/test it? Executed tests Signed-off-by: Petro Pikh <[email protected]> commit bc274dc26168e3f1771e34303e58c6635c11e66d Author: Ihor Chekh <[email protected]> Date: Tue Jun 28 19:02:50 2022 +0300 [VXLAN_ECMP] Workaround to have separate setup and teardown for each supported encap type (#5858) Separate configuration and teardown for each supported encapsulation type: ['v4_in_v4', 'v4_in_v6', 'v6_in_v4', 'v6_in_v6'] - What is the motivation for this PR? Fix the test issue when two types of tunnels (v4 and v6) configured at the same time - How did you do it? By using an additional fixture, which allows to do setup and teardown for each encap type - How did you verify/test it? By running test in the nightly regression commit 326f9fb08933e33e99515fdc74ed624885b6354a Author: Cong Hou <[email protected]> Date: Tue Jun 28 23:58:03 2022 +0800 Get a valid mac adderss for Mellanox switch (#5869) The host vlan test now configures a new mac address(00:11:22:33:44:55) other than the switch mac on a vlan interface. But for Mellanox asics, there is a restriction that we don't support this kind of mac address, because this address is not under the same prefix(36-38 bits) of the switch mac. So need to add a new logic to get a valid mac address for the vlan interface on Mellanox duts. - What is the motivation for this PR? Run host vlan test successfully on Mellanox asics. - How did you do it? Adding a new logic, if the asic is Mellanox, flap the bits in the last octet to generate a new mac address for the test instead of "00:11:22:33:44:55". - How did you verify/test it? Run test on spc1/spc2/spc3 duts, all passed. commit 20c2aa6a33507f47acacb8f85ca52f1cb48a9318 Author: Cong Hou <[email protected]> Date: Tue Jun 28 23:56:36 2022 +0800 Fix the cpu core number for 4600c in platform info thermal test (#5883) 4600C was using wrong thermal profile and it displays 2 CPU core thermal in show platform temperature output, there should be 4 CPU core thermal. The PR in sonic-buildimage has fixed this: Azure/sonic-buildimage#10258 The sonic-mgmt platform thermal test should also be fixed to align with the output change. - What is the motivation for this PR? Fix cpu core number for 4600c in platform info thermal test. - How did you do it? Correcting the thermal cpu core number for 4600c to 4 in tests/common/mellanox_data.py. - How did you verify/test it? Run test case in master and 202012 branch on 4600c by automation, all passed. - Any platform specific information? The change is only for 4600c. commit 4bd499c20553f49d6fc1023642e4017021f57590 Author: Longxiang Lyu <[email protected]> Date: Tue Jun 28 22:58:15 2022 +0800 [nic_simulator] Correct port id mapping for upper/lower ToR (#5882) Signed-off-by: Longxiang Lyu <[email protected]> commit d19bff8fbfe308fdbd59ec37fc4a27e49e62804c Author: Zhaohui Sun <[email protected]> Date: Tue Jun 28 13:55:23 2022 +0800 Modify neighbor flush in test_tunnel_memory_leak (#5856) What is the motivation for this PR? Test case test_tunnel_memory_leak sometimes failed with error "AssertionError: No dev found for neighbor 192.168.0.3". It's because toggle_all_simulator_ports_to_upper_tor runs after run_arp_responder and garp_service, so low Tor may be active Tor and receives arp responder packets and garp packets before toggling, ip neighbors may exist in low Tor. But we check ip neighbors on upper Tor and delete them. How did you do it? If there is no neighbor on upper Tor, we should not fail the test case, it doesn't impact test function. tunnel_packet_handler will trigger neighbor added back. Just delete neighbor if neighbor exists, otherwise ignore it and continue the test. How did you verify/test it? Run dualtor/test_tunnel_memory_leak.py Signed-off-by: Zhaohui Sun <[email protected]> commit 5aaa607fd2f763a296a53ca7f32ed396ad905e3d Author: Ying Xie <[email protected]> Date: Mon Jun 27 22:04:31 2022 -0700 Revert "loganalyzer ignore (#5819)" (#5880) This reverts commit 51b4b784f5fa6acd255c08f5dfd5d6dc0d67ee7c. commit 45e110414ec6d1656ead4ce0908d46a8eefde2aa Author: Xin Wang <[email protected]> Date: Tue Jun 28 11:51:25 2022 +0800 Fix the fib hash testing on dualtor testbed (#5879) PR https://github.com/Azure/sonic-mgmt/pull/5456 updated the return value of fixture `single_fib_for_duts`. It is no longer just True of False. The new value will be different strings. The fib_test.py ptf script has been updated for this change. However, the hash_test.py ptf script was not. This change udpated the fib_test.py script accordingly. Now the fib hash testing can pass on dualtor testbed. Signed-off-by: Xin Wang <[email protected]> commit ce53d799bc91550afc19eb666242522dd1e126cc Author: StormLiangMS <[email protected]> Date: Mon Jun 27 20:09:36 2022 -0700 [conditional mark] To add test_thermal_control_load_invalid_value_json skip for Cisco (#5875) What is the motivation for this PR? When delete xfail, mis delete one case which is necessary for Cisco devices in PR https://github.com/Azure/sonic-mgmt/pull/5777/files, to add it back in this PR. How did you do it? How did you verify/test it? Any platform specific information? Supported testbed topology if it's a new test case? commit 1282349a75e6975f8bad19a2458aa75def8a2fc5 Author: jingwenxie <[email protected]> Date: Tue Jun 28 10:29:37 2022 +0800 [tests/ecmp/inner_hashing] Add config_reload to restore test environment (#5868) Summary: Add module-level config reload in case it breaks test env. commit 180deb4f0d820613216526745ed9bf9e7108f170 Author: Zhaohui Sun <[email protected]> Date: Tue Jun 28 09:22:51 2022 +0800 migrate sflow and vlan to python3 (#5872) What is the motivation for this PR? migrate sflow and vlan to python3 How did you do it? Check syntax format for slow and vlan code, move them to py3 folder. Add is_python3 parameter for test_sflow.py How did you verify/test it? Run test_sflow.py Signed-off-by: Zhaohui Sun <[email protected]> commit a280c2147c196c970074871190deb49e10243e65 Author: Ye Jianquan <[email protected]> Date: Mon Jun 27 09:56:39 2022 +0800 [pc/test_lag_2.py & enum_dut_portchannel] Support different completeness_level to reduce test time (#5846) * [po/test_lag_2.py && enum_dut_portchannel] Support completeness_level to reduce test time Reduce the test time of pc/test_lag_2.py by supporting different completeness_level Approach What is the motivation for this PR? pc/test_lag_2.py will enumerate all PCs on the DUTs, it could take 52+ mins on a large topology. We need to support different completeness_level, like: Run with 'confident' level every weekday, it enumerates at most 4 PCs. Run with 'thorought' level every weekend, it enumerates all PCs. How did you do it? Modify tests/conftest.py and pc/test_lag_2.py to support different completeness_level. If completeness_level is not set, run as default level('thorough'), it's the same behavior as before this pr. How did you verify/test it? Run on a physical testbed and it runs as expected. Signed-off-by: Jianquan Ye<[email protected]> commit 40bd53e32510fc521636c23a7ec6df913b3efa57 Author: Ye Jianquan <[email protected]> Date: Mon Jun 27 09:55:38 2022 +0800 [log analyzer] Support ignoring syslog temporarily by setting ignore markers (#5865) * [log analyzer] Support ignore log analyzer by decorating a function Enhance log analyzer to support ignoring syslog temporarily by setting ignore markers Provide a new decorator to ignore all the syslog during the decorated function elegantly Ignore the syslog during config_reload What is the motivation for this PR? Loganalyzer captures all the syslog in our test session, and raise exception whenever an unexpected err log is found. Sometimes a config_reload is required during the test case, a config_reload would raise some err log which is expected and acceptable, but the err log significantly misleads the loganalyzer. So we need a mechanism in loganalyzer to be muted temporarily. How did you do it? Enhance log analyzer to support ignoring syslog temporarily by setting ignore markers Provide a new decorator to ignore all the syslog during the decorated function elegantly Ignore the syslog during config_reload How did you verify/test it? Run on physical testbeds, it works as expected. Signed-off-by: Jianquan Ye<[email protected]> commit fd076fd477c8a72abf76d84e85638dc2d4f02b21 Author: judyjoseph <[email protected]> Date: Sat Jun 25 22:27:46 2022 -0700 Fix to get the test_link_flap run with vsonic peer (#5747) * Fix to get the test_link_flap run with vsonic peer commit 8d2646177ed9bfd5febc5e05dd7278b67ee5728d Author: vmittal-msft <[email protected]> Date: Fri Jun 24 17:13:46 2022 -0700 Updated QoS test params to accomodate changes in buffer profiles for TD3 based HWSKUs (#5857) commit 59ff039a13066b0b47135f28029f00a500b70def Author: Cong Hou <[email protected]> Date: Sat Jun 25 01:26:01 2022 +0800 [Mellanox] Add ignore error log for test_show_platform_temperature_mocked (#5814) Some errors are ignored for test test_show_platform_temperature_mocked in a previous PR: #5750 But there was one error missing, need to ignore it too. - What is the motivation for this PR? Add missing error log to ignore for PR: #5750 - How did you do it? Add one new error log to ignore list for case test_show_platform_temperature_mocked - How did you verify/test it? Manual run commit 7eb0e4c04ad9111ed49606f27dd8141bd5b0c261 Author: Cong Hou <[email protected]> Date: Sat Jun 25 01:23:44 2022 +0800 Fix GCU update speed case to use supported speed get from the dut (#5843) The test_update_speed case now uses hard coded speed parameters, the speed params may not be supported by some DUTs (e.g. 20G speed is not supported by all nvidia platforms), need to be fixed. This PR combines multiple commits in #5766 (Closed) - What is the motivation for this PR? The test_update_speed case now uses hard coded speed parameters, the speed params may not be supported by some DUTs(e.g. 20G speed is not supported by all nvidia platforms). In order to run this case on all testbeds, need to use supported speeds got from the dut instead of the hardcoded speeds. - How did you do it? Add a function to get the supported speeds from the dut, and use them in the test case. The fix uses 2 valid speeds(randomly selected from all supported speeds) and 1 invalid speed(the value is not changed) just as it was before. For vs setups, because there is no actual supported speed to get, still use the original hard coded valid speed params(20000, 40000). - How did you verify/test it? Have run this case on 2 different testbeds with nvidia switches, all passed. commit 6dd39e225cf193c5c6faccda76886c17f2a73211 Author: Antonh <[email protected]> Date: Fri Jun 24 20:22:01 2022 +0300 [PBH] enable PBH tests on valid branches only (#5861) Update the ignore of PBH tests. The feature was introduced in branch 202111. All later branches support this feature. Signed-off-by: Anton <[email protected]> commit 233189a7a0f925d0b1dc9871e9ae5ff6d5242d06 Author: xwjiang-ms <[email protected]> Date: Fri Jun 24 17:29:47 2022 +0800 Revert add timestamp to tr.xml and changes in junit xml parser (#5867) What is the motivation for this PR? We found some issues in nightly and smoke test because of adding timestamp and changing in junit xml parser: 1)Some xml report didn't contain "start" but contain "end" 2)Test will fail if xml and json files do not contain "start" and "end" How did you do it? Revert changes and fix How did you verify/test it? Run a test Any platform specific information? Supported testbed topology if it's a new test case? commit 2e675594916c53071e3d873689f7cd2a750f5a38 Author: Ashwin Srinivasan <[email protected]> Date: Thu Jun 23 17:45:02 2022 -0700 Added version check to the setup() function (#5589) The tests within the test_fan_drawer_fans script were all failing because a function call was throwing an exception. On further inspection it was discovered that the function that was being called was an unsupported API in the 201911 and 201811 SONiC OS versions. Code was added to skip those tests if either of these OSes were encountered. commit 64a3a6715a6acdd60eacfee1e57e6790c03524c2 Author: Longxiang Lyu <[email protected]> Date: Fri Jun 24 07:51:51 2022 +0800 [nic_simulator] Duplicate ARP packets to both ToRs (#5862) Approach What is the motivation for this PR? ARP packet from the server should be duplicated to both ToRs. Signed-off-by: Longxiang Lyu [email protected] How did you do it? Add a flow to duplicate the ARP How did you verify/test it? Verify both ToRs has the ARP entry for the server. commit bb6bee2d21157d5641c1f85b4022d8cb0ed0655e Author: bingwang-ms <[email protected]> Date: Thu Jun 23 08:33:15 2022 +0800 Always load_minigraph after acl test (#5852) commit a7f30153cc5287627a18aed5905c7f6c0795655b Author: Vaibhav Hemant Dixit <[email protected]> Date: Wed Jun 22 09:46:52 2022 -0700 [test-gap] Test IPinIP decap after VXLAN setup and teardown (#5834) Fix test-gap to address a recently discovered issue where ipinip decap functionality failed after CPA teardown (teardown of vxlan tunnel + mirror acls). A recent warmboot issue highlighted that decap functionality will fail after CPA (control plane assistant) session was teardown. Although CPA is relevant during warmboot, but this issue does not necessarily require warmboot. Just setting up and teardown CPA session will trigger this. I this PR, a new param vxlan to test_decap is added. Now all the cases in decap will run with or without a prestep of vxlan set-and-unset. Changes made: Before running decap test, if vxlan set_unset knob is enabled, create a CPA session and then tear it down. Also, do not apply the config from template. This will ensure new tables are not created, and hence the test will run with default config loaded on the device. commit 77c11fea447d2985d2da52e5e7acd669d334e761 Author: SuvarnaMeenakshi <[email protected]> Date: Wed Jun 22 09:00:50 2022 -0700 Add loopback 4096 ip address for multi-asic VS hwsku. (#5849) What is the motivation for this PR? The internal BGP sessions were not coming up in multi-asic VS because Loopback4096 interface was not added for ASICs in multi-asic minigraph if ansible/lab was used in inventory file. sonic-mgmt/tests/kvmtest.sh uses ./testbed-cli.sh -t vtestbed.yaml -m veos_vtb deploy-mg vms-kvm-four-asic-t1-lag lab password.txt to generate the minigraph . Rerefence: https://github.com/Azure/sonic-buildimage/blob/87425a5b2b02ecd75f3de7e5174e37e053e31790/src/sonic-bgpcfgd/bgpcfgd/managers_bgp.py#L128 How did you do it? Add loopback_4096 ip addresses for asic in ansible/lab inventory How did you verify/test it? Bring up multi-asic KVM testbed. ./testbed-cli.sh -t vtestbed.yaml -m veos_vtb -k ceos add-topo vms-kvm-four-asic-t1-lag password.txt ./testbed-cli.sh -t vtestbed.yaml -m veos_vtb -k ceos add-topo vms-kvm-four-asic-t1-lag password.txt Ensure all BGP sessions are up commit 774978f6444b5b00b9f55365189f7a2704c5a958 Author: jingwenxie <[email protected]> Date: Wed Jun 22 15:28:26 2022 +0800 Bypass replace_lane to unblock submodule update (#5847) Summary: Suspect test bug in test_eth_interface.py. Skip for now and request further investigation. commit 676a093e98bc41d7245793b5ecc0b947b6c7296c Author: Zhaohui Sun <[email protected]> Date: Wed Jun 22 09:15:03 2022 +0800 Fix lambda TypeError in ptfhost_utils (#5845) What is the motivation for this PR? Fix the issue reported in #4867, failures: TypeError: () takes exactly 2 arguments (1 given) The fix in #5605 still uses lambda which is not easy to read and may have compatibility issue. How did you do it? Use for loop and concentrate string together and append them to a list. How did you verify/test it? run tests/test_qos_sai.py Signed-off-by: Zhaohui Sun <[email protected]> commit 9f7af402de9e8e16a33c81b31581c9e155e55430 Author: xwjiang-ms <[email protected]> Date: Tue Jun 21 18:00:17 2022 +0800 Fix junit xml parser failure in nightly test (#5844) * Fix junit xml parser failure in nightly test What is the motivation for this PR? We added start and end timestamp for testcases in test report, and we will parse start and end timestamp in junit xml parser. However, some tests will not contain start and end timestamp, but the parser will directly parse report file without checking, so it will fail How did you do it? Paser parses start and end timestamp only if they exist How did you verify/test it? Run a test Any platform specific information? Supported testbed topology if it's a new test case? * Fix expression error commit 698e2a4c2951823cfb4c1432407ad9492451ecb7 Author: Ze Gan <[email protected]> Date: Tue Jun 21 15:46:30 2022 +0800 [azurepipeline]: Add MACsec pool back to Azp checker (#5831) What is the motivation for this PR? The t0-sonic pool has been fixed, so add it back to azp checker. How did you do it? Remove continueOnError in run-test-template.yml. Signed-off-by: Ze Gan <[email protected]> commit 63365a1348d4d4256840e14b448a5845cb1efe58 Author: Kevin Wang <[email protected]> Date: Tue Jun 21 13:59:09 2022 +0800 [pfc] Fix compatable issue in unknown_mac (#5841) What is the motivation for this PR? The output is different on master and 202012 with CLI "show mac -p <port>" while the <port> doesn't exist. How did you do it? Use "show mac" rather than "show mac -p <port>", and find the target mac from the output to see if the target mac exists. How did you verify/test it? Run pfc/test_unknown_mac.py Signed-off-by: Kevin Wang <[email protected]> commit 0357de1c79ae2920a039c247dea83a441a0d29e0 Author: Longxiang Lyu <[email protected]> Date: Tue Jun 21 10:34:12 2022 +0800 [dualtor] Fix `test_radv_ipv6_ra` (#5842) Approach What is the motivation for this PR? On dualtor testbed, test_radv_ipv6_ra complains about not able to find fixture cable_type. The root cause is that request.getfixturevalue doesn't support parameterized fixtures. How did you do it? Direct call toggle_all_simulator_ports_to_upper_tor instead. How did you verify/test it? commit bc4cba42997ac6d2f8a4098aeecb8235a5c73a10 Author: Zhaohui Sun <[email protected]> Date: Tue Jun 21 09:19:58 2022 +0800 Check ICMP reachability of server and ptf for testbeds (#5840) Check ICMP reachability of server and ptf for testbeds as well. Add several new columns for testbed health info table in Kusto. The final data looks like this: [ { "TestbedName": "vms1-1", "DutIP": "10.10.10.10", "ServerName": "server-1", "ServerIcmpReachability": 1, "DutName": "str-1", "DutIcmpReachability": 1, "PtfIP": "20.20.20.20", "ServerIP": "30.30.30.30", "DutConsoleReachability": null, "PtfName": "ptf-1", "DutSshReachability": null, "PtfIcmpReachability": 1 } ] How did you do it? Improve health_checker.py to check icmp for all hosts in inventory files. Then parse testbed.yaml file to get all testbeds. Check the ping result of dut/server/ptf from output json file generated by devutils. Each duthost has one row in testbed health info table in Kusto How did you verify/test it? Trigger testbed healht poller pipeline. Signed-off-by: Zhaohui Sun <[email protected]> commit 1307ca3a2a0f11545d464b1cb13e6bad4498bb37 Author: jingwenxie <[email protected]> Date: Sun Jun 19 21:55:41 2022 -0700 Avoid long run test in GCU. (#5816) Summary: Skip long running test in GCU. Will improve later. commit ca257fb953b39a0950104f18689be9e24244882e Author: Longxiang Lyu <[email protected]> Date: Mon Jun 20 11:25:07 2022 +0800 [dualtor][active-active] Fix `restart-ptf` (#5826) What is the motivation for this PR? On a dualtor-mixed testbed, the show arp command output: ``` root@demo-switch:~# show arp Address MacAddress Iface Vlan ------------ ----------------- -------------- ------ 10.0.0.61 2a:77:d0:fe:c3:08 PortChannel103 - 10.0.0.63 12:3b:84:02:53:9b PortChannel104 - 10.206.144.1 00:00:5e:00:01:90 eth0 - 10.206.144.7 06:60:bc:7b:e3:e7 eth0 - 192.168.0.3 b6:6e:70:63:9a:c8 Ethernet4 1000 192.168.0.4 92:75:ca:f9:ab:02 Ethernet8 1000 192.168.0.5 b6:6e:70:63:9a:c8 Ethernet4 1000 192.168.0.7 b6:6e:70:63:9a:c8 Ethernet4 1000 192.168.0.10 ae:42:2e:65:27:05 Ethernet20 1000 192.168.0.13 b6:6e:70:63:9a:c8 Ethernet4 1000 192.168.0.17 b6:6e:70:63:9a:c8 Ethernet4 1000 192.168.0.19 b6:6e:70:63:9a:c8 Ethernet4 1000 192.168.0.20 22:58:c8:d8:d6:0a Ethernet40 1000 192.168.0.21 b6:6e:70:63:9a:c8 Ethernet4 1000 192.168.0.24 ca:77:84:eb:7c:0c Ethernet48 1000 Total number of entries 27 ```` All arp entries from the netns are received from the same interface Ethernet4 the root cause is that restart-ptf will try to add ports to netns and assign IP address to it, when restart-ptf tries to assign IP addresses to interfaces in the netns, it will flush the address first, which will break the policy-based routing. Signed-off-by: Longxiang Lyu [email protected] How did you do it? 1. enable restart-ptf to remove/add netns, so the routing policies in the netns will be resumed. 2. modify the routing table logic to check the routing table existence first before adding it. How did you verify/test it? run restart-ptf multiple times and verify the arp: commit 630d2003ef20c186ad6bb9c6e677b05a05269d38 Author: Yutong Zhang <[email protected]> Date: Mon Jun 20 09:26:42 2022 +0800 Get token from environment variable for accessing Github (#5767) Description of PR We can use github issues as conditions in tests/common/plugins/conditional_mark/tests_mark_conditions.yaml. If the status of issue is closed, we will not skip the test case. Due to the lack of git credentials, we can not visit github and consider all the issues as open before. In this pr, we add the git credentials into environment variables, so we can get the real status of issue. And the status of this issue is closed, we will not skip this test case. What is the motivation for this PR? We can use github issues as conditions in tests/common/plugins/conditional_mark/tests_mark_conditions.yaml. If the status of issue is closed, we will not skip the test case. Due to the lack of git credentials, we can not visit github and consider all the issues as open before. In this pr, we add the git credentials into environment variables, so we can get the real status of issue. And the status of this issue is closed, we will not skip this test case. How did you do it? Set the user name and api token in pipeline/library. Pass them to tests/kvmtest.sh, and export them as environment variables. How did you verify/test it? Running the test and the error will not appear again. Signed-off-by: Yutong Zhang <[email protected]> commit d019f2769035befe42a1fc15a0137eb0891c0914 Author: Stephen Sun <[email protected]> Date: Sun Jun 19 19:36:39 2022 +0800 Optimize the logic to compare the routing entries between ASIC_DB and the prefixes pushed to DUT (#5769) - What is the motivation for this PR? Optimize the logic to compare the routing entries between ASIC_DB and the prefixes pushed to DUT - How did you do it? The idea is to extract the prefixes from the output of the DUT and to avoid using json.load for each routing entry to use set instead of list to check difference. O(n*n) => O(n*lg(n)) How did you verify/test it? Run the test. Signed-off-by: Stephen Sun <[email protected]> commit 4c99c91cba5fb882d12c550c2be6b85cc95e187d Author: Sudharsan Dhamal Gopalarathnam <[email protected]> Date: Sun Jun 19 04:30:36 2022 -0700 [config reload]Fixing config reload tests to handle failure error return (#5783) - What is the motivation for this PR? When config reload command fails it would return a non zero error code. Without module_ignore_errors the ansible invoke command will crash. Additional checks are introduced when checking for delayed services to ensure only enabled services are checked. - How did you do it? Modify config reload tests to have module_ignore_errors when config reload is expected to be failed. - How did you verify/test it? Run the config reload tests. commit c92cae0193c4e850e100e6408e02029811e5568a Author: Ying Xie <[email protected]> Date: Sun Jun 19 01:05:00 2022 -0700 [vlan util] update Vlan utility according to config fact format (#5839) What is the motivation for this PR? VLAN test is failing on dual tor testbeds due to extra keys in the VLAN_INTERFACE dict. How did you do it? Explicitly looking for IPv4 VLAN subnet and return the found IP. How did you verify/test it? Run vlan tests on dualtor testbed and passed. Signed-off-by: Ying Xie [email protected] commit 7ce95a530c35eee342538856fdf750d1cb319b33 Author: Shahzad Iqbal <[email protected]> Date: Fri Jun 17 16:36:16 2022 -0700 Updated the BFD test to perform the following. - BFD Session scaling to 100 sessions. - Added testcases to verfiy both IPV4 and V6 session creation in both orders - DUT Initiation followed by PTF - PTF session creation followed by DUT. - Modified the test to use batch commands instead of single ones to improve test performance. commit d518188636ff9f4b007be0c18399308441e4fd1b Author: Shahzad Iqbal <[email protected]> Date: Fri Jun 17 16:36:16 2022 -0700 Updated the BFD test to perform the following. - BFD Session scaling to 100 sessions. - Added testcases to verfiy both IPV4 and V6 session creation in both orders - DUT Initiation followed by PTF - PTF session creation followed by DUT. - Modified the test to use batch commands instead of single ones to improve test performance.
1 parent 983283f commit 5c15ba1

File tree

279 files changed

+12822
-4268
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

279 files changed

+12822
-4268
lines changed

‎.azure-pipelines/pr_test_scripts.yaml

+116
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,116 @@
1+
t0:
2+
- arp/test_arp_dualtor.py
3+
- arp/test_neighbor_mac.py
4+
- arp/test_neighbor_mac_noptf.py
5+
- bgp/test_bgp_fact.py
6+
- bgp/test_bgp_gr_helper.py::test_bgp_gr_helper_routes_perserved
7+
- bgp/test_bgp_speaker.py
8+
- bgp/test_bgpmon.py
9+
- bgp/test_bgp_update_timer.py
10+
- container_checker/test_container_checker.py
11+
- cacl/test_cacl_application.py
12+
- cacl/test_cacl_function.py
13+
- cacl/test_ebtables_application.py
14+
- dhcp_relay/test_dhcp_relay.py
15+
- dhcp_relay/test_dhcpv6_relay.py
16+
- generic_config_updater/test_aaa.py
17+
- generic_config_updater/test_bgpl.py
18+
- generic_config_updater/test_bgp_prefix.py
19+
- generic_config_updater/test_bgp_speaker.py
20+
- generic_config_updater/test_cacl.py
21+
- generic_config_updater/test_dhcp_relay.py
22+
- generic_config_updater/test_eth_interface.py
23+
- generic_config_updater/test_ipv6.py
24+
- generic_config_updater/test_lo_interface.py
25+
- generic_config_updater/test_monitor_config.py
26+
- generic_config_updater/test_ntp.py
27+
- generic_config_updater/test_portchannel_interface.py
28+
- generic_config_updater/test_syslog.py
29+
- generic_config_updater/test_vlan_interface.py
30+
- iface_namingmode/test_iface_namingmode.py
31+
- lldp/test_lldp.py
32+
- monit/test_monit_status.py
33+
- ntp/test_ntp.py
34+
- pc/test_po_cleanup.py
35+
- pc/test_po_update.py
36+
- platform_tests/test_advanced_reboot.py::test_warm_reboot
37+
- platform_tests/test_cpu_memory_usage.py
38+
- process_monitoring/test_critical_process_monitoring.py
39+
- radv/test_radv_ipv6_ra.py
40+
- route/test_default_route.py
41+
- route/test_static_route.py
42+
- show_techsupport/test_techsupport_no_secret.py
43+
- snmp/test_snmp_cpu.py
44+
- snmp/test_snmp_default_route.py
45+
- snmp/test_snmp_interfaces.py
46+
- snmp/test_snmp_lldp.py
47+
- snmp/test_snmp_loopback.py
48+
- snmp/test_snmp_pfc_counters.py
49+
- snmp/test_snmp_queue.py
50+
- ssh/test_ssh_ciphers.py
51+
- ssh/test_ssh_limit.py
52+
- syslog/test_syslog.py
53+
- system_health/test_system_status.py
54+
- tacacs/test_accounting.py
55+
- tacacs/test_authorization.py
56+
- tacacs/test_jit_user.py
57+
- tacacs/test_ro_disk.py
58+
- tacacs/test_ro_user.py
59+
- tacacs/test_rw_user.py
60+
- telemetry/test_telemetry.py
61+
- test_features.py
62+
- test_interfaces.py
63+
- test_procdockerstatsd.py
64+
65+
t0-2vlans:
66+
- dhcp_relay/test_dhcp_relay.py
67+
- dhcp_relay/test_dhcpv6_relay.py
68+
69+
t0-sonic:
70+
- bgp/test_bgp_fact.py
71+
- macsec/test_macsec.py
72+
73+
dualtor:
74+
- arp/test_arp_dualtor.py
75+
76+
t1-lag:
77+
- bgp/test_bgp_allow_list.py
78+
- bgp/test_bgp_bbr.py
79+
- bgp/test_bgp_bounce.py
80+
- bgp/test_bgp_fact.py
81+
- bgp/test_bgp_multipath_relax.py
82+
- bgp/test_bgp_update_timer.py
83+
- bgp/test_bgpmon.py
84+
- bgp/test_traffic_shift.py
85+
- configlet/test_add_rack.py
86+
- container_checker/test_container_checker.py
87+
- http/test_http_copy.py
88+
- ipfwd/test_mtu.py
89+
- lldp/test_lldp.py
90+
- monit/test_monit_status.py
91+
- pc/test_lag_2.py
92+
- platform_tests/test_cpu_memory_usage.py
93+
- process_monitoring/test_critical_process_monitoring.py
94+
- route/test_default_route.py
95+
- scp/test_scp_copy.py
96+
- test_interfaces.py
97+
98+
multi-asic-t1-lag:
99+
- bgp/test_bgp_fact.py
100+
- snmp/test_snmp_default_route.py
101+
- snmp/test_snmp_loopback.py
102+
- snmp/test_snmp_pfc_counters.py
103+
- snmp/test_snmp_queue.py
104+
- tacacs/test_accounting.py
105+
- tacacs/test_authorization.py
106+
- tacacs/test_jit_user.py
107+
- tacacs/test_ro_disk.py
108+
- tacacs/test_ro_user.py
109+
- tacacs/test_rw_user.py
110+
111+
multi-asic-t1-lag-pr:
112+
- bgp/test_bgp_fact.py
113+
114+
t2:
115+
- test_vs_chassis_setup.py
116+
- voq/test_voq_init.py

‎.azure-pipelines/run-test-scheduler-template.yml

+54
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
parameters:
2+
- name: TOPOLOGY
3+
type: string
4+
5+
- name: POLL_INTERVAL
6+
type: number
7+
default: 10
8+
9+
- name: POLL_TIMEOUT
10+
type: number
11+
default: 36000
12+
13+
steps:
14+
- script: |
15+
set -e
16+
17+
pip install PyYAML
18+
19+
rm -f new_test_plan_id.txt
20+
python ./.azure-pipelines/test_plan.py create -t ${{ parameters.TOPOLOGY }} -o new_test_plan_id.txt
21+
TEST_PLAN_ID=`cat new_test_plan_id.txt`
22+
23+
echo "Created test plan $TEST_PLAN_ID"
24+
echo "Check https://www.testbed-tools.org/scheduler/testplan/$TEST_PLAN_ID for test plan status"
25+
echo "##vso[task.setvariable variable=TEST_PLAN_ID]$TEST_PLAN_ID"
26+
env:
27+
TESTBED_TOOLS_URL: $(TESTBED_TOOLS_URL)
28+
TENANT_ID: $(TESTBED_TOOLS_MSAL_TENANT_ID)
29+
CLIENT_ID: $(TESTBED_TOOLS_MSAL_CLIENT_ID)
30+
CLIENT_SECRET: $(TESTBED_TOOLS_MSAL_CLIENT_SECRET)
31+
displayName: Trigger test
32+
33+
- script: |
34+
echo "Polling Test plan"
35+
echo "Preparing the testbed(add-topo, deploy-mg) may take 15-30 minutes. Before the testbed is ready, the progress of the test plan keeps displayed as 0, please be patient(We will improve the indication in a short time)"
36+
echo "If the progress keeps as 0 for more than 1 hour, please cancel and retry this pipeline"
37+
echo "TestbedV2 is just online and might not be stable enough, for any issue, please send email to [email protected]"
38+
echo "Runtime detailed progress at https://www.testbed-tools.org/scheduler/testplan/$TEST_PLAN_ID"
39+
python ./.azure-pipelines/test_plan.py poll -i $(TEST_PLAN_ID)
40+
env:
41+
TESTBED_TOOLS_URL: $(TESTBED_TOOLS_URL)
42+
displayName: Running test
43+
timeoutInMinutes: 300
44+
45+
- script: |
46+
echo "Try to cancel test plan $TEST_PLAN_ID, cancelling finished test plan has no effect."
47+
python ./.azure-pipelines/test_plan.py cancel -i $(TEST_PLAN_ID)
48+
condition: always()
49+
env:
50+
TESTBED_TOOLS_URL: $(TESTBED_TOOLS_URL)
51+
TENANT_ID: $(TESTBED_TOOLS_MSAL_TENANT_ID)
52+
CLIENT_ID: $(TESTBED_TOOLS_MSAL_CLIENT_ID)
53+
CLIENT_SECRET: $(TESTBED_TOOLS_MSAL_CLIENT_SECRET)
54+
displayName: Cancel running test plan

‎.azure-pipelines/run-test-template.yml

+15-5
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,13 @@ parameters:
1212
- name: section
1313
type: string
1414
default: ''
15+
- name: image
16+
type: string
17+
default: sonic-vs.img.gz
18+
values:
19+
- sonic-vs.img.gz
20+
- sonic-4asic-vs.img.gz
21+
- sonic-6asic-vs.img.gz
1522

1623
steps:
1724
- script: |
@@ -37,7 +44,7 @@ steps:
3744
- script: |
3845
sudo mkdir -p sonic-vm/images
3946
sudo mkdir -p sonic-vm/disks
40-
sudo cp -v ../target/sonic-vs.img.gz sonic-vm/images/sonic-vs.img.gz
47+
sudo cp -v ../target/${{ parameters.image }} sonic-vm/images/sonic-vs.img.gz
4148
sudo gzip -fd sonic-vm/images/sonic-vs.img.gz
4249
username=$(id -un)
4350
sudo chown -R $username.$username sonic-vm
@@ -56,13 +63,16 @@ steps:
5663

5764
- script: |
5865
pwd
59-
66+
6067
parent_dir=$(basename $PWD)
61-
docker exec sonic-mgmt-2 bash -c "/var/src/$parent_dir/tests/kvmtest.sh -en -T ${{ parameters.tbtype }} -d /var/src/$parent_dir ${{ parameters.tbname }} ${{ parameters.dut }} ${{ parameters.section }}"
68+
docker exec -e GIT_USER_NAME=$GIT_USER_NAME -e GIT_API_TOKEN=$GIT_API_TOKEN sonic-mgmt-2 bash -c "/var/src/$parent_dir/tests/kvmtest.sh -en -T ${{ parameters.tbtype }} -d /var/src/$parent_dir ${{ parameters.tbname }} ${{ parameters.dut }} ${{ parameters.section }}"
69+
env:
70+
GIT_USER_NAME: $(GIT_USER_NAME)
71+
GIT_API_TOKEN: $(GIT_API_TOKEN)
6272
displayName: "Run tests"
63-
${{ if eq(parameters.tbtype, 't0-sonic') }}:
73+
${{ if eq(parameters.tbtype, 'multi-asic-t1-lag-pr') }}:
6474
continueOnError: true
65-
75+
6676
- script: |
6777
# save dut state if test fails
6878
virsh_version=$(virsh --version)

0 commit comments

Comments
 (0)