[help] testing userspace TCP

[commmrade]

As I was implementing TCP on top of TUN in user-space I remembered that there is a thing called tests and that you should kinda write them. So I decided to find some tools that could help me with this and I came across packetdrill. That looked like a perfect tool for my case so I decided to try it out, which did not really work out, because I don't really know how to test user-space TCP implementations with this and would like some help.

Basically, my TCP implementation creates a TUN interface (called tun1) and traffic must go through this TUN to get to the TCP. I decided to use wire server and wire client to make this happen and this "kinda" works. My TCP implementation is able to receive SYNs, for example, but when it sends a SYNACK, packetdrill does not receive this and times out. I have no idea how to fix it.
.pkt is this:

    0 socket(..., SOCK_STREAM, IPPROTO_TCP) = 3
  +.1...0.200 connect(3, ..., ...) = 0
   +0 > S 0:0(0) <...>
  +.1 < S. 0:0(0) ack 1 win 4380 <mss 1460>
   +0 > . 1:1(0) ack 1

Also, I used these commands:

sudo ./packetdrill/packetdrill --wire_server &
sudo ./packetdrill/packetdrill --ip_version=ipv4 --local_ip=10.0.0.1 --remote_ip=10.0.0.2 --connect_port=8090 --wire_client_dev=tun1 tcp/blocking/blocking-connect.pkt

[nealcardwell]

Can you please share a design diagram depicting (or a more detailed text description describing) how your user-space TCP stack works, and how you are intending to use packetdrill? It sounds "inverted" from how I would expect things to work.

A couple notes:

(1) Normally a tun device is used to inject packets into and sniff packets from the Linux TCP stack. It is not normally used to inject packets into and sniff packets from a user-space TCP stack. Can you please explain how you are using the tun device in greater detail?

(2) When you say "My TCP implementation is able to receive SYNs", this seems strange when combined with the test script you mention, because in that test script packetdrill is expecting the TCP implementation under test to send a SYN, not receive a SYN.

(3) If you want to debug a packetdrill scenario of any kind, I would suggest running the following command on all the machines that are involved (client and server side, if using remote mode):

sudo tcpdump -XXX -n -c 2 -e -i any port 8080

[nealcardwell]

If you are looking for more info about packetdrill, I recommend reading the following slides and/or watching the following talk:

https://netdevconf.info/0x19/sessions/tutorial/tutorial-using-packetdrill-to-write-automated-tests-for-the-linux-networking-stack.html

Otherwise, the best solution is probably to read the code, or send your question to the packetdrill mailing list at: packetdrill@googlegroups.com

[commmrade]

Can you please share a design diagram depicting (or a more detailed text description describing) how your user-space TCP stack works, and how you are intending to use packetdrill? It sounds "inverted" from how I would expect things to work.

A couple notes:

(1) Normally a tun device is used to inject packets into and sniff packets from the Linux TCP stack. It is not normally used to inject packets into and sniff packets from a user-space TCP stack. Can you please explain how you are using the tun device in greater detail?

(2) When you say "My TCP implementation is able to receive SYNs", this seems strange when combined with the test script you mention, because in that test script packetdrill is expecting the TCP implementation under test to send a SYN, not receive a SYN.

(3) If you want to debug a packetdrill scenario of any kind, I would suggest running the following command on all the machines that are involved (client and server side, if using remote mode):

sudo tcpdump -XXX -n -c 2 -e -i any port 8080

I think I understood the idea of how packetdrill is used incorrectly and I also may have had a wrong idea about how TUN works exactly at the time I was writing this issue.
If I was to describe how I think my TCP implementation works now I would think of this:

So my TCP impl. and Linux Net. Stack act like different end systems, since as far as my understanding goes, TUN is a virtual network device.

As for packetdrill, I imagine it works like this (I asked an AI to draw this, but that is kinda exactly my understanding):

Now, as for how I wanted to use packetdrill
I wanted it to kinda use my TCP's TUN device and inject packets there, so my TCP can process them and send something back, which I would then validate using something like this, for example, +.1 < S. 0:0(0) ack 1 win 4380 <mss 1460> (i may have confused '<' and '>'). Now, I have to assume that this is not possible, since my TCP impl. "owns" this TUN and only it can write to it and then read from it. So I figure that the only way to test the TCP impl. is to use raw sockets, so the kernel can route packets to my TCP's TUN device. But packetdrill can't do this, correct? Or there may be some dark linux magic that I am not aware of to make this work.

Also, thanks for your quick response :)