Replies: 3 comments 6 replies
-
Can we nic-loopback it?Looking through the git history for So, after following the Chipyard Repo Setup docs, then: MAKEFLAGS=-j`nproc` make -C sims/verilator VERILATOR_THREADS=1
make -C tests nic-loopback.riscvfollowed by: time ./sims/verilator/simulator-chipyard.harness-RocketConfig +permissive +max-cycles=10000000 +permissive-off tests/nic-loopback.riscvproduces this output: Next ideas:
|
Beta Was this translation helpful? Give feedback.
-
|
re:
It does not: class RocketConfig extends Config(
new freechips.rocketchip.subsystem.WithNBigCores(1) ++ // single rocket-core
new chipyard.config.AbstractConfig)But there is a config that seems purpose-built: class LoopbackNICRocketConfig extends Config(
new chipyard.harness.WithLoopbackNIC ++ // drive NIC IOs with loopback
new icenet.WithIceNIC ++ // add an IceNIC
new freechips.rocketchip.subsystem.WithNBigCores(1) ++
new chipyard.config.AbstractConfig)And, luckily, it even produces a simulator with Et voila: Figuring that out took learning to differentiate between the simulator, chipyard's different Configs, and to identify what a "Harness" and "Top" are (roughly) and quite a lot of reading various Makefiles and experimentation with It also still takes two minutes to simulate a fairly simple program, but there's plenty of avenues for exploration there too. I'm definitely attuned to the idea of diving way too deep on "make RTL fast zoom zoom," but I'm reminding myself before focusing too much on improving a single workflow, it's well worth understanding how that workflow fits into our overall goals. |
Beta Was this translation helpful? Give feedback.
-
Can we tuntap it?Since we're going for connecting a user-space program (the simulated risc-v chip) into the network, there's also the matter of getting the other "end" of that virtual link cable connected to something. In Linux, "I want to connect my userspace program to a virtual network device" has a spelling of tun/tap. They're a little awkward to use, but with help from these two resources:
(and more than a few hours of failed experiments), I managed to setupsubnet=0
for mode in tun tap; do
sudo ip tuntap add "${mode}2" mode "${mode}" user $USER
sudo ip addr add 10.0."$(( subnet++ ))".1/24 dev "${mode}2"
sudo sysctl -w net.ipv6.conf."${mode}"2.disable_ipv6=1
sudo ip link set "${mode}2" up
done
sudo ip neigh add 10.0.1.2 dev tap2 lladdr "$(ip --brief link show tap2 | awk '{print $3}' )" nud permanentThat creates a A A run itSave this somewhere, maybe as import struct
from fcntl import ioctl
def open_dev(mode, name=b''):
fd = open("/dev/net/tun", "r+b", buffering=0)
LINUX_IFF_TUN = 0x0001
LINUX_IFF_TAP = 0x0002
LINUX_IFF_NO_PI = 0x1000
LINUX_TUNSETIFF = 0x400454CA
flags = LINUX_IFF_NO_PI
if mode == 'tun':
flags |= LINUX_IFF_TUN
else:
flags |= LINUX_IFF_TAP
ifs = struct.pack("16sH22s", name, flags, b"")
ioctl(fd, LINUX_TUNSETIFF, ifs)
return fd
import sys
if sys.argv[1] not in ['tap', 'tun']:
print(f'invalid mode: {sys.argv[1]}', file=sys.stderr)
print(f'usage: {sys.argv[0]} tap|tun [DEV_NAME]', file=sys.stderr)
sys.exit(2)
mode = sys.argv[1]
dev = open_dev(mode, (sys.argv[2:] or [''])[0].encode('utf=8'))
offset = 0 if mode == 'tun' else 14
while True:
data = dev.read(1500)
if len(data) == 0:
break
print(data.hex())
reply = bytearray(data)
ip_pkt = reply[offset:]
# swap source & dest IP (doesn't change checksum)
ip_pkt[12:16], ip_pkt[16:20] = ip_pkt[16:20], ip_pkt[12:16]
# change from ICMP echo request (8) to ip_pkt (0)
ip_pkt[20] = 0
ip_pkt[22] = (ip_pkt[22] + 8) % 256 # fixup checksum
reply[offset:] = ip_pkt
# also swap src & dst mac addrs when we're in 'tap' mode
if mode == 'tap':
reply[0:6], reply[6:12] = reply[6:12], reply[0:6]
dev.write(reply)
print("-> ", reply.hex())Then, run: $ python pingd.py tun tun2
# or, python pingd.py tap tap2
# in another terminal
$ ping -nc2 10.0.0.2
# or, 10.0.1.2 if you ran in `tap` modeAnd you should see familiar ping output: with output similar to the following from the python side: Which we can plug into the handy https://hpd.gasmi.net :
To see that we've got an ICMP echo ping request/reply. notes
cleanupStop any running python programs (lest you see for mode in tun tap; do sudo ip tuntap del "${mode}2" mode "${mode}"; donenext steps ?
wrapping up / what was it all forSo, we pinged it! Now, "all" we have to do is "simply" replace our More concretely (and the reason I've taken at least 2x as long to write this up as it did to get a little janky prototype working), this can act as a point of reference when something goes wrong in hooking up the verilog RTL simulation. If it works with Footnotes
|
Beta Was this translation helpful? Give feedback.
-
Yes, we can! (Simulate it)Adding this config somewhere in the Chipyard Cinematic Universe (I put it in PeripheralDeviceConfigs.scala): class TapNICRocketConfig extends Config(
new chipyard.harness.WithSimNetwork ++
new icenet.WithIceNIC ++
new freechips.rocketchip.subsystem.WithNBigCores(1) ++
new chipyard.config.AbstractConfig)Reading from the bottom up (because that's how scala evaluates it), we've got:
With that in hand, we can run: (why With everything set up, that produces a These are very similar to "can we tuntap it" above; the last bit compiles the Now we can run: Which reads: "run the simulator, letting me pass arbitrary plusargs, throw "netdev=tap0" into the void and hope SimNetwork reads it, turn strict argument parsing back on, and simulate running the bare-metal binary After quite a while (~80s on my machine), that'll print out something like (Note the Hooray, a 2 second ping to localhost! The first one takes a bit longer because, unlike the simpler Finally, if you wanted to see the The real standout, of course, is the iTLB misses: Potential next avenues (in increasing order of difficulty):
|
Beta Was this translation helpful? Give feedback.
-
|
lol yeah, as a very scientific test I tried with |
Beta Was this translation helpful? Give feedback.
-
|
What is a network tap? From https://chipyard.readthedocs.io/en/stable/Generators/IceNet.html |
Beta Was this translation helpful? Give feedback.
-
|
I have answered myself sort of https://en.wikipedia.org/wiki/Network_tap Seems like a network tap is a wire that duplicates a signal that you can read from? Why do we say "network tap" instead of "test point" or anything else? |
Beta Was this translation helpful? Give feedback.
-
|
what's the diff between getting an FPGA to "accelerate" the simulator and just running the thing on the FPGA? Like if we have the FGPA why not just put the ping/RISC-V example on there directly? |
Beta Was this translation helpful? Give feedback.
-
|
re: network taps; that's neat: I only knew the compound term "tap device", which means something more like "a part of the Linux kernel that emulates a physical wire (
It's a good question: one that seems like it'd let us get at some of the interesting tradeoffs to talk about. I don't really have the expertise to break it down yet, but some things I've noticed are:
Together, those suggest to me that there's a development/production "seam" here: you build an RTL model that is intended to be turned into a bitstream for a particular shipping FPGA, but getting feedback about what's going wrong is slow and sucks, so you find a way to speed it up... and hey, you've got all this experience with FPGAs, and probably some pretty powerful ones sitting right there! In other words, I think the FPGA-accelerated version is intended to be identical to the software RTL simulation (modulo the time-domain effects): it's purely deterministic, you can sample/trace any waveform without disturbing the system in any way, etc. Putting the model onto the FPGA doesn't express those same properties, but it's also probably faster/cheaper/lower power (in some sense). |
Beta Was this translation helpful? Give feedback.
-
Tracking notes/discussion thread for the goal: "Can we run Chipyard's pingd test?"
The second question is on hold for now, as I'm lacking the desire to get too into part selection yet—e.g. none of the diligent dev gear has two ethernet ports, the minimum required for an interesting router (though a single 10/100MB port would work for a ping test).
Pinging the simulated SoC
In order to ping the SoC from my Linux host, we'd have to connect the I/O somehow from the outside world. In other contexts (e.g. containers) I've done this with a veth interface, but there's a whole bunch of options for "thing that can be passed to the
-Iflag of ping."The harder challenge is probably going to be figuring out how to get the
verilatorsimulation host to set itself up to translate the SoC's read attempts to interact with theveth(or whichever). That's going to reward understanding at least a little bit about what the IceNet "thing" is, and what I/O "looks like".Some possible resources:
Beta Was this translation helpful? Give feedback.
All reactions