Small Fish, Big Pond

Besides gambling and overpriced shows Vegas is famous for their all you can eat buffets (among other things). Let’s imagine for a moment you’re at a buffet and you eat until you’re full, but a guy sitting next to you eats 4 times as much before he stops. Would you feel slighted that he somehow cut in on your share? You had all you wanted, he had all he wanted, and you both paid the same amount, were you slighted because his consumption was more than yours?

How about the buffet owners? If they still have so much food left over after you and your partner ate your fill they have to throw it out were they slighted by his high consumption?

The big question being is did he “hog” all the food? Personally I’m kind of an efficiency junkie so my answer is no, he got what he needed without negatively impacting anybody else. The only loss here was the providers overabundance of food, perhaps an overcalculation on their part but this way all their customers were happy.

You may be wondering what this has to do with technology but Benoît Felten has pointed out the same thing, except in the definition of a”bandwidth hog” on the internet. ISPs are claiming that bandwidth hogs take more than their fair share of bandwidth at the expense of others, this way people who use too much can be charged more. But if you pay for all you can eat and the provider has bandwidth to spare are you a hog or just making use of excess bandwidth?

Benoît is challenging the ISPs to prove that bandwidth hogs are a problem. (follow up here).

I can’t directly answer Benoît’s challenge but as I work for an ISP, in the cloud, where all the backend bandwidth lies and I can tell you with some hands on experience that at least on our network bandwidth hogs are not a problem and doesn’t create noticeable impact on the network users.

But before shouting the inevitable “Aaah HA!” to ISPs for lying to us for their own profit, bandwidth hogs DO impact the network at peak times, just not noticeably or in the way we would expect. You see we create complex Cost of Service rules across all the routers in the cloud that guarantees our customers have X amount of bandwidth minimum based off what they pay, and then Y amount of excess bandwidth if its available. This creates more of a flow change during peak times rather than a roadblock. Most people think of network congestion the same way they do a traffic jam. If one big semi is taking up most of the road the smaller cars get crowded behind it and soon you’re 20 minutes late to work because traffic came to a halt.

Network congestion doesn’t work like that.

When traffic starts to fill up the available bandwidth (traffic lanes in the above analogy) traffic doesn’t bunch up behind the heavy data streams (semis) instead the bandwidth is narrowed down. So the cars don’t bunch up behind the semi, the semi is made skinnier and longer taking up less bandwidth but over a longer period of time. In this way if somebody is streaming a hulu video and using up all the bandwidth, your mission critical email won’t be blocked all traffic will just be stretched out a bit and the email will take a bit longer to send. However considering that web traffic and emails are fractions of a fraction the size of a high def web stream even a doubling of the time taken to send it will be virtually un-noticeable. We’re talking about 60ms to load to 120ms to load.

The bigger the file transfer the more noticeable the change, so the only people who will have a noticeable impact on performance are those people who are hogging the most bandwidth.

And in real life production it’s amazing watching the latency change as we grant that guaranteed amount of bandwidth instead of just having a company use the excess bandwidth. At first a new router is just using the excess bandwidth on the cloud which may be very minimal at peak times. As soon as they’re Cost of Service and Committed Information Rates are integrated into the bandwidth there is a very noticeable change is transfer speeds.

The amount of bandwidth being on the cloud used is usually quite high simply because people are using up the excess (thus less lines are “dark” or under utilized) so sites that don’t have their guaranteed COS values set have a hard time competing with sites that have guaranteed speeds+the excess. Thus a site with no COS may get 40% of their max circuit speeds, once they’re integrated with everybody else that usually bounces up above 90%. It’s only the rare time that EVERYBODY maxes out their connections and speeds drop across the board to the guaranteed speeds (usually around 75% of the circuit capability).

Another good comparison is the planned state of the “smart” electric grid which is basically going to work as out internet does now. By monitoring data flow all across the net high congestion times can be rate limited a few percentage points to smooth out the bursts of data. There is an impact to users but the less data needed the less noticeable the impact, a page that takes 1 second to download will still seem about the same if it takes 1.2 seconds to download.

So bandwidth hogs do exist and at the same time their use a a term by the ISPs are a concoction to get more money.

Businesses have been living under this model for decades, and users have been to but it’s only recently we’re regularly using enough bandwidth that we’re starting to notice our impact. In the future we will probably see terms such as “Guaranteed Class of Service” and “Committed Information Rates” show up more in consumer internet plans, although I hope they use terminology that it’s easier for consumers to understand. But it’s best now if you start to erase from your mind the idea that you purchased a 5mbps connection cable and that you’re guaranteed to be able to use that at full throttle all the time.

At the same time be aware that data plans that are tiered by total bandwidth used is just a marketing way that companies can dig more out of heavy users. The solution that I hope we see in the future is simply the way corporate connections go, and that is that you pay for a line capable of X bandwidth, with a guarantee of Y bandwidth.

via ars

I mentioned it once before in a blog post, and I’ve also said it in numerous forum and blog comments. Deep Packet inspection is not new, it’s not rare, and it’s not limited in scope. Ars Technicia reports that OpenDPI an open-source DPI engine has revealed their source code to allay fears that personal info is retained. To give you an idea what protocols were talking about here:

The OpenDPI engine will identify a huge list of non-encrypted protocols, however:

• P2P File Sharing: BitTorrent, eDonkey , KaZaa/Fasttrack, Gnutella, WinMX, DirectConnect, AppleJuice, Soulseek, XDCC, Filetopia, Manolito, iMesh, Pando
• Voice over IP: SIP, IAX, RTP
• Instant Messaging: Yahoo, Oscar, IRC, unencrypted Jabber, Gadu!Gadu, MSN
• Streaming Protocols: ORB, RTSP, Flash, MMS, MPEG, Quicktime, Joost, WindowsMedia, RealMedia, TVAnts, SOPCast, TVUPlayer, PPStream, PPLive, QQLive, Zattoo, VeohTV, AVI, Feidian, Ececast, Kontiki, Move, RTSP, SCTP, SHOUTcast
• Tunnel Protocols: IPsec,GRE, SSL, SSH, IP in IP
• Standard Protocols: HTTP, Direct download links (1-click file hosters), POP, SMTP, IMAP, FTP, BGP, DHCP, DNS, EGP, ICMP, IGMP, MySQL, NFS, NTP, OSPF, pcAnywhere, PostgresSQL, RDP, SMB, SNMP, SSDP, STUN, Telnet, Usenet, VNC, IPP, MDNS, NETBIOS, XDMCP, RADIUS, SYSLOG, LDAP
• Gaming Protocols: World of Warcraft, Half-Life, Steam, Xbox, Quake, Second Life

BitTorrent, DirectConnect, ORB, VeohTV, SHOUTcast, IPsec, GRE, SSL, SSH, RDP, WoW. I know a lot of people who don’t think these can be identified but they can. This is also connected directly to your IP address so they will know that YOU sent it, what it program it was, and where it went. Using DPI they may not know what you sent over Bittorrent, but they will know that you did torrent something from point A to B.

I first began learning of DPI when I was on the security team working for Cisco in 2004, Cisco’s implementation is called CBAC and at the time is was installed in most currently sold routers and switches. Now it is in most all Cisco devices and added into many old ones through IOS updates.

DPI isn’t a way to reconstruct data streams and recover files that cross the internet, it’s simply the act of opening up the packet further, more than just seeing the IP addressing or what port it is destined for. DPI looks close enough to see what type of application the packet is used with. Imagine it as opening a traveling packet enough to see that it is a “.doc” file; you can’t tell what the doc files says from this one chunk, just that part of a doc file crossing the net from point A to B.

While this may allay fears that ISPs are reconstructing your data as it passes through their devices it’s almost equally alarming how much control ISP can use to rate limit and control what traffic you send and receive.

Like I said, most all Cisco devices can now check and filter this traffic; even my little 871 at home can do this so you know the big backbone routers can do it too. And being that Cisco devices make up 86% of the internet it’s virtually guaranteed that your data will pass through a CBAC capable device at some point. ISPs currently have the ability to pick and choose what you send over your connection and drop the rest if they want.

The saving grace is that DPI is more processor intensive than basic routing. The more inspection they do the higher the overhead moving data across the network. And as it stands now they’re trying to cut overhead as much as possible. Most backend routers don’t even act as routers so much as highspeed switches, they check once to see the layer 3 IP address of a packet then send the it and the rest of the datastream on it’s way.

Hopefully most ISPs will keep their lines an un-policed cloud where people can use the connection they pay for in the method of their choosing. But as the backend gets filled with bandwidth intensive Hulu streams, and torrent downloads they may put filters on these kinds of traffic to allow only permitted traffic through. The point is that everything is already in place, the only thing that need to change is ISP corporate policies.