TCP VS UDP
And why TCP is unsuitable for games
OK, so I did some comparisons using a server of mine.
The thing is, TCP and UDP ARE pretty much equal when it comes to transportation times across the internet.
In my tests, I sent 10,000 messages across about 50km (my server is located about 50km from where I am now).
The results:
UDP: Average round-trip time: 30ms
TCP: Average round-trip time: 500ms
Now you’re saying WTF???? YOUR TEST IS WRONG!! But allow me to explain.
The reason the TCP tests were so awful, is, well, here, take a glance at numerics of the results first.
UDP RESULTS:
DIFF=0.0278434801501
DIFF=0.0305130913992
DIFF=0.0277071201201
DIFF=0.027180660817
DIFF=0.0300143798507
DIFF=0.0273463786314
DIFF=0.0335492180775
DIFF=0.0280604430992
DIFF=0.0330231491187
DIFF=0.0273172101218
// And it looks the same for all 10000 messages.. no huge latencies at all.
// these are in SECONDS, so it translates to about 30ms per datagram.
// BTW, in this test, I sent 10000 messages and received 9994 of them.
Now for the TCP results..
// First few, at the beginning when the application first launched…
DIFF=0.0373954983804
DIFF=0.0373870345209
DIFF=0.0373785706613
DIFF=0.0373701068017
DIFF=0.0373617621515
DIFF=0.0373534175012
// HMM, that looks pretty much the same as TCP
// Scrolling down a bit… it gets worse..
DIFF=0.0669155569774
DIFF=0.0669068546993
DIFF=0.066898510049
// further down still..
DIFF=0.0961061779911
DIFF=0.0960975949222
DIFF=0.0960886542255
// ok, further down…
DIFF=0.232780874127
DIFF=0.232772767895
DIFF=0.232756436223
// and take a look at THIS interesting section ..
DIFF=2.25679475173
DIFF=2.25677114829
DIFF=2.2567289482
DIFF=2.17196494445
DIFF=1.80040103301
DIFF=1.10465389594
DIFF=0.114454568475
DIFF=0.114441693871
DIFF=0.114434064477
So what’s going on here? What’s happening, especially in the very last block where the round-trip time varies from 2.35 seconds to 0.11 seconds is the PRIMARY REASON why I wouldn’t recommend to use TCP for games, EVER.
Its because of the BUFFERING. You know that when you call .send() on a TCP socket, TCP WILL NOT NECESSARILY DELIVER THE MESSAGE IMMEDIATELY! TCP has a buffer BOTH at the sender (messages might pile up for a number of seconds before actually being fired off across the net) __AND__ at the recipient (messages might stock up for a number of seconds before the remote user’s TCP subsystem alerts HIM that you have sent some bytes). This is precisely what makes TCP so inappropriate for use for games: it might withhold the data from you, even if its already at your machine, as you can see here, for up to 2 seconds (or more!).
In the context of a fast paced game, this will give a stupid 2 second lag in game play (likely the game will freeze up for the player, as if he had temporarily disconnected.. depending on how you programmed it of course) which really is not good.
The TCP system is basically like having customs at the airport. Before you can travel, you have to pass through customs, and after you arrive, you also have to pass through customs. You might be held up at customs (the outgoing buffer) for very short (“anything to declare? no? good!”) or you might be there for hours (seconds). You then get into the same plane as the UDP packets, travel across the internet, and when you arrive at the remote host, you as a TCP message must go through customs AGAIN (while the UDP packet simply runs off into the crowd). UDP packets don’t have customs (buffering). But TCP packets do. And because of the customs (buffering) there’s this UNPREDICTABLE, UNCONTROLLABLE (just like customs!!) extra wait that makes TCP totally unsuitable for games.
UDP, as you can see from the results listed above (0.03s, 0.03s…), CONSISTENTLY delivers the messages in 30ms .. UDP is completely stateless and connectionless – you fire off a message, and away across the net it goes. UDP has no sense of buffers or any of the heavyweight stuff that TCP employs. When you use UDP to send messages across the internet, the only delay that will be encountered as the messages goes across the web is because of congestion on the web itself. A UDP message WILL NOT be held up in some buffer EITHER on the local machine from which you send it, or at the remote host that receives it.
Often when people build games, you see them building a TCP-like system (with message acknowledgement and re-sending in case the message doesn’t arrive), but you might wonder WHY they bother to you see people using UDP if they need message acknowledgement. Well, the reason is simple. More CONTROL. UDP doesn’t have that annoying buffer! When you implement “TCP in UDP”, you are in complete control and can achieve the same reliability you get with TCP using UDP. And its worth it.
Conclusions: On Windows it SEEMS that TCP “gets lazy” after about 10 seconds of using it. When a program is sending thousands of really small (8-byte, in this case) messages, the TCP implementation in Windows XP really starts to buffer the packets up and you start to see big honking 1024k messages being transmitted, which ultimately translates to ADDITIONAL DELAY for your game.
So, this is the PRIMARY REASON that TCP isn’t suitable for games. Its not that “TCP is slower” in any absolute sense – actual transmission speeds between TCP and UDP are pretty much the same across the same segment of the ‘net. WHAT IS different is that TCP MAY WITHHOLD THE DATA FROM YOU for longer than you might like in its buffers. And that (the buffering) is what makes TCP unsuitable for games. You can’t control the buffering of TCP – its a FEATURE, not a bug/problem.
The scripts used are below. You launch the server script on your server, then you run the client script and tell it whether you want (U)DP test to run (type U) or the TCP test to run (T).
The UDP test takes about 100 seconds to run, because there is a 0.01s pause between the .send for each of the 10000 messages (something about the UDP buffer overflowing and getting MASSIVE packet loss if you try and send UDP packets too rapidly).
SERVER SCRIPT
Run on the server you are using. It launches BOTH a UDP listener AND a TCP listener (on separate threads), so you can run either test by running this single server script.
#SERVER SCRIPT
#Python 2.5
from socket import *
from select import select
from struct import *
from time import *
import thread
import sys
TCPServer = socket( AF_INET, SOCK_STREAM )
TCPPort = 10007
TCPDone = False
UDPServer = socket( AF_INET, SOCK_DGRAM )
UDPPort = 10008
UDPDone = False
""" UDP server startup """
def UDPServerStart():
global UDPDone
try :
# just bind UDP socket
UDPServer.bind( ( '', UDPPort ) )
print '\nUDP socket bound successfully on port %d' % UDPPort
except :
#only get here if something bad happened in try block above
print "\nServer couldn't startup! Reason:",sys.exc_info()
UDPServer.close()
sys.exit() # bail if failed
while( UDPDone is False ):
try :
( buf, senderAddr ) = UDPServer.recvfrom( 1024 )
#print 'I got something'
except :
print '\nudp: could not recvfrom'
break
# now just return that message to the sender
UDPServer.sendto( buf, senderAddr )
# we are done when the message is a 0
if( buf[ 0 ] == 0 ):
UDPDone = True
#end while
print '\nShutting down the UDP server'
UDPServer.close()
""" TCP Server startup """
def TCPServerStart():
global TCPDone
try :
#bind
TCPServer.bind( ( '', TCPPort ) )
print '\nTCP socket bound on port %d' % TCPPort
#listen
print '\nTCP socket is listening'
TCPServer.listen( 5 )
#Accept
(sock,senderAddr) = TCPServer.accept()
print '\nTCP socket has accepted connection',senderAddr
except :
#only get here if something bad happened in try block above
print "Server couldn't startup! Reason:",sys.exc_info()
TCPServer.close()
sys.exit() # bail if failed
# now
TCPDone = False
while( TCPDone is False ):
try :
buf = sock.recv( 1024 )
except:
print '\nproblem in tcp receive'
TCPDone = True
break #will drop us out of the loop
# now just return that message to the sender
sock.send( buf )
##print 'Got',buf
# we are done when the message is a 0
if( len(buf) == 0 or buf[ 0 ] == 0 ):
TCPDone = True
# end while
print '\nshutting down the connected socket'
sock.close()
print '\nShutting down the TCP server'
TCPServer.close()
def Unstick():
udpunsticker = socket( AF_INET, SOCK_DGRAM )
udpunsticker.sendto( '', ( '127.0.0.1', UDPPort ) )
udpunsticker.close()
# unstick sticky socket
tcpunsticker = socket( AF_INET, SOCK_STREAM )
# connect to MAINSOCKET to basically unstick (unblock) it
tcpunsticker.connect( ('127.0.0.1', TCPPort) )
tcpunsticker.close()
# end def Unstick
def main():
global TCPDone
global UDPDone
# Start up 2 threads, one for the TCP server, and one
# for the UDP server.
# thread.start_new_thread( RunGame, ( player1sock, player2sock ) )
thread.start_new_thread( TCPServerStart, () )
thread.start_new_thread( UDPServerStart, () )
inputs = raw_input("Press any key to shut down")
TCPDone = True
UDPDone = True
Unstick()
main()
CLIENT SCRIPT
Launch after already having launched SERVER SCRIPT on your server
#Python 2.5
from socket import *
from select import select
from struct import *
from time import *
import thread
import sys
class Packet :
def __init__( self, ID ) :
self.ID = ID
self.constructionTime = clock()
def packMe( self ) :
packed = pack( 'ff', self.ID, self.constructionTime )
return packed
def unpackMe( self, data ) :
unpacked = unpack( 'ff', data )
return unpacked
def getDataAsTuple( self ):
return ( self.ID, self.constructionTime )
LocalIPAddress = '127.0.0.1'
RemoteIPAddress = '127.0.0.1' # CHANGE TO YOUR REMOTE SERVER ADDRESS
TCPSocket = socket( AF_INET, SOCK_STREAM )
TCPPort = 10007
UDPSocket = socket( AF_INET, SOCK_DGRAM )
UDPLocalPort = 10006
UDPRemotePort = 10008
Done = False
cumDiff = 0
cumMsgs = 0
StartTime = clock()
SaveOver = ''
TotalReceivedMessages = 0
def TCPTest():
global cumDiff
global cumMsgs
global SaveOver
global TotalReceivedMessages
def TCPListener() :
global cumDiff
global SaveOver
global TotalReceivedMessages
output = open( "tcplisteneroutput.txt", "w" )
output.write( 'Listener start'+ '\n' )
while( Done is False ) :
try :
data = TCPSocket.recv( 1024 )
datalen = len(data)
output.write( "Now I got "+str(data)+" which is "+str(datalen)+" bytes."+ '\n' )
except :
output.write( 'I couldnt recv '+str(sys.exc_info())+ '\n' )
# This is some packet we already sent
# and its being echo'd back to us here
# what time is it NOW?
timeNow = clock()
# the difference is how long it took to transport
# there and BACK
packet = Packet( 1 )
# Now, if there IS something saved over from last time, prepend
# those elements to the array.
if len( SaveOver ) is not 0 :
# prepend those leftovers
data = SaveOver + data
# now compute if there's any
# left over on THIS iteration
overChop = datalen % 8
# save over
lastGood = datalen - overChop
SaveOver = data[lastGood:]
try :
output.write( "Processing " + str( datalen-overChop ) + "\n" )
for i in range( 0, datalen - overChop, 8 ) :
unpacked = packet.unpackMe( data[i:i+8] )
diff = timeNow - unpacked[1]
output.write( "DIFF="+str(diff)+ '\n' )
cumDiff += diff
TotalReceivedMessages += 1
except :
output.write( 'problems unpacking'+str( sys.exc_info())+ '\n' )
output.close()
f = open( "tcpTest.txt", "w" )
# connect to the TCP server and rapidly
# fire off messages. the server will send
# the messages back to you, so you can measure
# round-trip time
f.write( "TCP test begin"+ '\n' )
try :
# connect to the server
addr = ( RemoteIPAddress, TCPPort )
TCPSocket.connect( addr )
f.write( 'TCPSocket connected to '+str(addr)+ '\n' )
except :
#only get here if something bad happened in try block above
f.write( "Server couldn't startup! Reason: " + str(sys.exc_info())+ '\n' )
TCPSocket.close()
sys.exit() # bail if failed
# start up the listener thread
thread.start_new_thread( TCPListener, () )
### TCP TEST
for i in range( 1, 10000 ) :
f.write( 'sending packet '+str(i)+ '\n' )
packet = Packet( i )
# serialize and send that packet, also saving it
# so we can know when it came back.
data = packet.packMe()
try :
TCPSocket.send( data )
cumMsgs += 1
except :
f.write( 'data send failed!' + str(sys.exc_info() )+ '\n' )
#Done = True
sleep( 2 )
# done now, so print the average.
avg = cumDiff / TotalReceivedMessages
endStr = "cumDiff="+str(cumDiff) + \
" TotalReceivedMessages="+str(TotalReceivedMessages)+ \
' The TCP average transport time was '+str(avg)
f.write( endStr + '\n' )
print endStr
f.close()
return #/TCPTest
def UDPTest():
print "UDP test begin"
global cumDiff
global cumMsgs
global TotalReceivedMessages
def UDPListener() :
global cumDiff
global TotalReceivedMessages
output = open( "udplisteneroutput.txt", "w" )
output.write( 'UDP listener start'+ '\n' )
while( Done is False ) :
try :
( data, senderAddr ) = UDPSocket.recvfrom( 1024 )
output.write( "Now I got "+str(data)+" which is "+str(len(data))+" bytes, from "+str(senderAddr)+ '\n' )
except :
output.write( 'I couldnt recv'+str( sys.exc_info() )+ '\n' )
# This is some packet we already sent
# and its being echo'd back to us here
# what time is it NOW?
timeNow = clock()
# the difference is how long it took to transport
# there and BACK
packet = Packet( 1 )
try :
# all packets from UDP will always be the same length.
unpacked = packet.unpackMe( data[0:8] )
diff = timeNow - unpacked[1]
output.write( "DIFF="+str(diff) + '\n')
cumDiff += diff
TotalReceivedMessages += 1
except :
output.write( 'problems unpacking '+str( sys.exc_info() )+ '\n' )
# connect to the TCP server and rapidly
# fire off messages. the server will send
# the messages back to you, so you can measure
# round-trip time
f = open( "udpTest.txt", "w" )
f.write( "UDP test begin"+ '\n' )
try :
# bind this UDP socket.. because we want to be
# able to receive on it. we still could if
# we didn't bind, but it would be a random port
# and the ports are usually all closed.
addr = ( '', UDPLocalPort )
UDPSocket.bind( addr )
f.write( 'UDPSocket BOUND to '+str(addr)+ '\n' )
except :
#only get here if something bad happened in try block above
f.write( "Server couldn't startup! Reason: "+str(sys.exc_info())+ '\n' )
UDPSocket.close()
sys.exit() # bail if failed
# start up the listener thread
thread.start_new_thread( UDPListener, () )
### UDP TEST
for i in range( 1, 10000 ) :
f.write( 'sending packet '+str(i)+ '\n' )
packet = Packet( i )
# serialize and send that packet, also saving it
# so we can know when it came back.
data = packet.packMe()
try :
UDPSocket.sendto( data, ( RemoteIPAddress, UDPRemotePort ) )
print 'Sending',data
sleep( 0.01 ) # if you send packets too rapidly
# you will get severe packet loss (possibly due to
# UDP buffer overflow at the remote host
cumMsgs += 1
except :
f.write( 'data send failed! '+str( sys.exc_info() )+ '\n' )
#Done = True
sleep( 5 )
# done now, so print the average.
avg = cumDiff / TotalReceivedMessages
endStr = "cumDiff="+str(cumDiff) + \
" TotalReceivedMessages="+str(TotalReceivedMessages)+ \
' The UDP average transport time was '+str(avg)
print( endStr + '\n' )
f.write( endStr + '\n' )
return #/TCPTest
def UnstickTCP():
# unstick sticky socket
tcpunsticker = socket( AF_INET, SOCK_STREAM )
# connect to MAINSOCKET to basically unstick (unblock) it
tcpunsticker.connect( ('127.0.0.1', TCPPort) )
tcpunsticker.close()
def UnstickUDP():
udpunsticker = socket( AF_INET, SOCK_DGRAM )
udpunsticker.sendto( '', ( '127.0.0.1', UDPPort ) )
udpunsticker.close()
# end def Unstick
# rapidly send out
def main():
inputs = raw_input("(T)CP or (U)DP?")
if( inputs.startswith( 'T' ) ) :
TCPTest()
Done = True
elif( inputs.startswith( 'U' ) ) :
UDPTest()
Done = True
main()
raw_input("Press enter to continue . . .")