[TD4] svi tagged member is unable to ping peer

[philo-micas]

1.Description

Broadcom platform TD4 chip (BCM56780_A0), the svi port's tagged member is unable to ping peer.

topo

TD4 Device (faulted)
Ethernet145
    |
    |
    |
    |
Ethernet321
TH4 Device (Auxiliary device)

2.Steps to reproduce

TD4 Device Configuration

admin@sonic:~$ sudo config vlan add 10
admin@sonic:~$ sudo config vlan mem add 10 Ethernet145
admin@sonic:~$ sudo config inter ip add Vlan10 104.0.0.1/24

admin@sonic:~$ show vlan b
+-----------+--------------+-------------+----------------+-------------+-----------------------+
|   VLAN ID | IP Address   | Ports       | Port Tagging   | Proxy ARP   | DHCP Helper Address   |
+===========+==============+=============+================+=============+=======================+
|        10 | 104.0.0.1/24 | Ethernet145 | tagged         | disabled    |                       |
+-----------+--------------+-------------+----------------+-------------+-----------------------+

TH4 Device Configuration

admin@sonic:~$ sudo config vlan add 10
admin@sonic:~$ sudo config inter switchmode trunk vlan-range 10 Ethernet321
admin@sonic:~$ sudo config inter ip add Vlan10 104.0.0.2/24

admin@sonic:~$ show vlan b
+-----------+--------------+-------------+----------------+-------------+------------+-----------------------+
|   VLAN ID | IP Address   | Ports       | Port Tagging   | Proxy ARP   | Proxy ND   | DHCP Helper Address   |
+===========+==============+=============+================+=============+============+=======================+
|         1 |              | Ethernet321 | untagged       | disabled    | disabled   |                       |
+-----------+--------------+-------------+----------------+-------------+------------+-----------------------+
|        10 | 104.0.0.2/24 | Ethernet321 | tagged         | disabled    | disabled   |                       |
+-----------+--------------+-------------+----------------+-------------+------------+-----------------------+

3.Observed behavior

1）

TD4 Device pings TH4 Device:

root@sonic:/home/admin# ping 104.0.0.2

On the TH4 device, using the tcpdump command, ICMP reply packets are observed:

admin@sonic:\~\$ sudo tcpdump -i Ethernet321
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on Ethernet321, link-type EN10MB (Ethernet), snapshot length 262144 bytes
14:26:04.863433 IP 104-0-0-1.lightspeed.wchtks.sbcglobal.net > 104-0-0-2.lightspeed.wchtks.sbcglobal.net: ICMP echo request, id 48721, seq 32, length 64
14:26:04.863479 IP 104-0-0-2.lightspeed.wchtks.sbcglobal.net > 104-0-0-1.lightspeed.wchtks.sbcglobal.net: ICMP echo reply, id 48721, seq 32, length 64
14:26:05.887427 IP 104-0-0-1.lightspeed.wchtks.sbcglobal.net > 104-0-0-2.lightspeed.wchtks.sbcglobal.net: ICMP echo request, id 48721, seq 33, length 64
14:26:05.887473 IP 104-0-0-2.lightspeed.wchtks.sbcglobal.net > 104-0-0-1.lightspeed.wchtks.sbcglobal.net: ICMP echo reply, id 48721, seq 33, length 64

But the TD4 device's ping has no response：

root@sonic:/home/admin# ping 104.0.0.2
PING 104.0.0.2 (104.0.0.2) 56(84) bytes of data.
^C
--- 104.0.0.2 ping statistics ---
44 packets transmitted, 0 received, 100% packet loss, time 43975ms

2）

TH4 Device pings TD4 Device:

root@sonic:/home/admin# ping 104.0.0.1

On the TD4 device, using the tcpdump command, no ICMP packets are observed:

admin@sonic:~$ sudo tcpdump -i Ethernet145
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on Ethernet145, link-type EN10MB (Ethernet), snapshot length 262144 bytes
14:44:48.485722 STP 802.1w, Rapid STP, Flags [Proposal, Learn, Forward, Agreement], bridge-id 8000.00:27:36:53:73:d0.8020, length 36
14:44:50.485718 STP 802.1w, Rapid STP, Flags [Proposal, Learn, Forward, Agreement], bridge-id 8000.00:27:36:53:73:d0.8020, length 36
14:44:52.485705 STP 802.1w, Rapid STP, Flags [Proposal, Learn, Forward, Agreement], bridge-id 8000.00:27:36:53:73:d0.8020, length 36
14:44:54.485696 STP 802.1w, Rapid STP, Flags [Proposal, Learn, Forward, Agreement], bridge-id 8000.00:27:36:53:73:d0.8020, length 36

And the TH4 device's ping has no response too.

4.Expect behavior

When the TD4 device pings the TH4 device, the following expected behavior should be observed:

root@sonic:/home/admin# ping 104.0.0.2
PING 104.0.0.2 (104.0.0.2) 56(84) bytes of data.
64 bytes from 104.0.0.2: icmp_seq=1 ttl=64 time=0.197 ms
64 bytes from 104.0.0.2: icmp_seq=2 ttl=64 time=0.182 ms
64 bytes from 104.0.0.2: icmp_seq=3 ttl=64 time=0.169 ms
64 bytes from 104.0.0.2: icmp_seq=4 ttl=64 time=0.177 ms

In fact, after manually installing an L2 station entry with VLAN field 10 on the TD4 device, the expected behavior can be observed.

5.Output of show version

SONiC Software Version: SONiC.master.782655-f8215d553

SONiC Software Version: SONiC.master.782655-f8215d553
SONiC OS Version: 12
Distribution: Debian 12.9
Kernel: 6.1.0-22-2-amd64
Build commit: f8215d553
Build date: Tue Feb 25 14:54:04 UTC 2025
Built by: azureuser@b3513a6fc000002

Platform: x86_64-micas_m2-w6520-24dc8qc-r0
HwSKU: M2-W6520-24DC8QC
ASIC: broadcom
ASIC Count: 1
Serial Number: N/A
Model Number: N/A
Hardware Revision: N/A
Uptime: 13:29:10 up 0 min,  1 user,  load average: 1.87, 0.50, 0.17
Date: Sun 01 Dec 2024 13:29:10

Docker images:
REPOSITORY                    TAG                       IMAGE ID       SIZE
docker-syncd-brcm             latest                    f37322365786   763MB
docker-syncd-brcm             master.782655-f8215d553   f37322365786   763MB
docker-gbsyncd-broncos        latest                    817c149f19b2   352MB
docker-gbsyncd-broncos        master.782655-f8215d553   817c149f19b2   352MB
docker-gbsyncd-credo          latest                    064a213cf0aa   326MB
docker-gbsyncd-credo          master.782655-f8215d553   064a213cf0aa   326MB
docker-orchagent              latest                    d48cdc23d058   356MB
docker-orchagent              master.782655-f8215d553   d48cdc23d058   356MB
docker-nat                    latest                    e28968bba823   346MB
docker-nat                    master.782655-f8215d553   e28968bba823   346MB
docker-fpm-frr                latest                    56b39006f552   377MB
docker-fpm-frr                master.782655-f8215d553   56b39006f552   377MB
docker-dhcp-relay             latest                    b98f7abc30ce   322MB
docker-macsec                 latest                    321b5e1b62db   346MB
docker-sonic-mgmt-framework   latest                    f4e0d7ce0343   402MB
docker-sonic-mgmt-framework   master.782655-f8215d553   f4e0d7ce0343   402MB
docker-snmp                   latest                    d1a69462f35c   358MB
docker-snmp                   master.782655-f8215d553   d1a69462f35c   358MB
docker-teamd                  latest                    0ca5fb9f1adf   343MB
docker-teamd                  master.782655-f8215d553   0ca5fb9f1adf   343MB
docker-platform-monitor       latest                    e2387fe2ecfa   434MB
docker-platform-monitor       master.782655-f8215d553   e2387fe2ecfa   434MB
docker-sflow                  latest                    27b2bd759647   344MB
docker-sflow                  master.782655-f8215d553   27b2bd759647   344MB
docker-router-advertiser      latest                    6479c3ca40c1   314MB
docker-router-advertiser      master.782655-f8215d553   6479c3ca40c1   314MB
docker-lldp                   latest                    d7c72f0b4efe   359MB
docker-lldp                   master.782655-f8215d553   d7c72f0b4efe   359MB
docker-mux                    latest                    4dc29febd0ce   365MB
docker-mux                    master.782655-f8215d553   4dc29febd0ce   365MB
docker-sonic-gnmi             latest                    8c6ecdba2e12   425MB
docker-sonic-gnmi             master.782655-f8215d553   8c6ecdba2e12   425MB
docker-eventd                 latest                    193a493556e1   314MB
docker-eventd                 master.782655-f8215d553   193a493556e1   314MB
docker-database               latest                    ce4dd0702870   322MB
docker-database               master.782655-f8215d553   ce4dd0702870   322MB
docker-sonic-bmp              latest                    f31020a186d1   315MB
docker-sonic-bmp              master.782655-f8215d553   f31020a186d1   315MB

[philo-micas]

Issue Analysis
This issue is caused by the community version of SAI code and the chip characteristics of TD4. You can see that if we normally create an SVI interface:

admin@sonic:~$ sudo config vlan add 995
admin@sonic:~$ sudo config interface ip add Vlan995 10::1/64

Then, use the following command to view the chip's L2 station table. You will see that the current community SAI handles the TD4 chip by creating an L2 station table entry with a VFI field of 0x3e3 (995) and a VLAN field of 0 when creating SVI port 995.
admin@sonic:~$ bcmcmd "l2 station show"

However, due to the characteristics of the TD4 chip, in the current default underlay mode, the TD4 chip indexes the L2 station table entry using the "MAC" field + "VLAN" field for tagged packets.
But the L2 station table entry issued by community SAI , as shown in the picture above, lacks the correct setting for the "VLAN" field. Therefore, when the ping packet enters the port, it fails to find any matching L2 station table entry during the "lookup L2 station table entry" step, resulting in a failure to hit L3, and thus the ping failed.

Temporary Solution
Since we cannot modify the SAI code of the community version, we can currently only manually issue an L2 station table entry with a "VLAN" field of 995 to ensure that the packet with 995 vlan tag hits L3, thereby making the ping successful:
Step 1: find the sysmac with a type of static and a port number of 0 by “l2 show”command

Step 2：Use this sysmac to add an L2 station table entry with an index of VLAN 995 to the chip through the SDK command
bcmcmd "l2 station add MACaddress=00:d0:f8:22:38:e4 Vlanid=995 VlanidMask=0xfff"

Step 3：l2 station show again to confirm the successful issuance of the L2 station table entry.
admin@sonic:~$ bcmcmd "l2 station show"

Step 4 (Optional)：When you delete the SVI port, you also need to delete the manually added L2 station table entry; otherwise, this entry will continue to occupy an L2 station capacity until reboot or reload; Deleting this l2 station entry need to specify the SID of the entry, which can be viewed using the "l2 station show" command from the step 3.
admin@sonic:~$ bcmcmd "l2 station delete ID=4"

[philo-micas]

@adyeung please take a look at this issue, thanks!

[adyeung]

@philo-micas please file CSP for SAI issue