I Have a Need, a Need for Download Speed
Should you try to speed up consecutive Steam downloads with a cache server?
Thanks to Wendell from Level1Techs for his all of his help on this project and pointing us in the right direction!
A few years ago, we were fortunate enough to get a fiber internet connection installed at the PC Perspective office. Capable of 1Gbps download speeds and about 250Mbps upload, we were excited at the possibilities that laid ahead.
However, when you have access to a very fast internet connection, you begin to notice that the bottleneck has shifted from your connection to the servers on the other side of the content delivery networks (CDNs) that power the internet. While these CDNs have very fast links to the internet, they generally limit bandwidth so that there is more speed to go around to multiple people at the same time.
DL from Steam at 101 MB/s! So fast that the SSD is having trouble keeping up. Able to grab 30+ GB in under 5 mins. pic.twitter.com/ptHUyQVHJr
— Ryan Shrout (@ryanshrout) September 3, 2014
A look back at what once was
One of the services that we found would max out our connection was Steam. Since we download a lot of PC games at the office, it was a nice benefit to have an internet connection as fast as our NICs could handle, and that the Steam CDNs would serve us at our maximum potential. In fact, the bottleneck shifted over to storage performance, as the random writing nature of Steam thrashed our SSDs at the time.
By no stretch of the imagination is 60MB/s slow.. but what happened to our 100MB/s!
Unfortunately, this has ceased to remain the case. At some point, Steam downloads started getting slower on our same internet connection. Not only did storage utilization during a Steam download start to increase, but also CPU usage, pointing to a potential change in how Steam distributed their data. While downloads on our high-end systems fell to around 50-60MB/s, systems with less CPU horsepower started to see speeds fall to 20-30MB/s. All hope was lost for fast game downloads.. or was it?
Recently, Wendell from Level1Techs mentioned on Twitter that they were running a local Steam caching server on their network with great success. After some guidance from Wendell, we decided to tackle this project and see if it would help our specific scenario.
The Setup
The theory behind caching Steam downloads is simple. While you might not get the appearance of it from the Steam client, all that's happening is a bunch of HTTP requests to Valve's CDN. Simply speaking, this means, that once you know the addresses of the servers Steam uses for downloading, you can monitor the traffic coming from them, save the data to a cache file, and then when it is needed again, simply deliver the local copy instead of going out to the internet to fetch it.
In order to be able to hijack this traffic, you need to intercept the DNS requests for Valve's CDN and point them to your local HTTP server configured for delivering the data.
The steamcache project on Github does just this, through the use of preconfigured Docker containers.
To accomplish this, you need to run two Docker containers. The first is the steamcache "generic" container, which provides a preconfigured nginx-powered HTTP proxy to intercept and store the relevant data.
Second, you need to be running the steamcache-dns container to redirect the DNS requests to your local server. This requires your computer sending all of it's DNS requests through to Docker container. This can be configured either by changing the local DNS server configuration on your computers or by changing the DNS server configuration in your router.
Running a local DNS server will provide additional speed benefits to activities such as web browsing since your computer no longer needs to make a request to an external DNS server for every possible HTTP request. This means a lower access time for all HTTP requests.
We aren't going to get into the minutia of configuring Docker for people who have never used it, as there are much better tutorials for that on the Internet by people who understand the process way better than we do. The notes on the steamcache website are a good place to start if you are thinking of doing this yourself.
It's worth noting that the steamcache generic container provides support for a multitude of other download services, like Uplay, Origin, and even Windows Updates. For now, we are sticking with strictly caching Steam downloads. We don't download a lot of games from Uplay or Origin, and when we do, they still manage to fully utilize our 1Gbps internet connection.
As time goes on, and the Steam caching proves to be stable, we might look into adding more cached services.
The Hardware
In order to run the docker containers, we turned to our new Ryzen 7 1700-powered QNAP TS-877 NAS (full review on this to come soon!). While it might seem like a bit of an oddity to run this sort of thing on a NAS, the desktop CPU, 10 Gigabit NIC, and available 2.5-in drive bays for SSDs made it a great pick.
For the cache storage, we are using two 1TB Samsung 840 EVO SATA SSDs in a RAID 0 configuration. While we wouldn't normally recommend a RAID 0 configuration in production, all this will be doing is holding a download cache of files we can easily get back if needed, so we aren't necessarily worried about the potentials of data loss.
Our Linux environment is provided by an Ubuntu 16.04 VM running in the KVM-based QNAP Virtualization Station. While the QNAP has native support for Docker containers running outside of a VM, we wanted to be able to move the VM image to other hardware if needed, and the QNAP Container Station software didn't seem to be very well documented at this point.
Results
With our Steam caching server in place, we first started by downloading several games to a local machine. Since this is the initial download, and there is no data already cached, it will not be any faster than downloading games without the cache enabled. However, in our experiences, this seeding of data to the cache also isn't any slower than the normal process of downloading the game.
To evaluate the performance of the cache, we then downloaded the same games on another PC. In order to eliminate any potential performance impacts on our client, we used an overkill system with an 8-core/16-thread Core i7-5960X CPU, 10Gbit NIC, and an Intel Optane 900P SSD.
With a hard limit of about 250 MB/s (!!!), now, the bottleneck seems to be the on the server side. When downloading at around 250MB/s, the Ryzen 7 1700 CPU in our QNAP NAS seems to be maxing out a single thread of the processor, within our Ubuntu VM.
While we could move this setup to yet another dedicated server with faster single-threaded performance, for now, we are happy with our results. Considering most of our machines aren't equipped with 10Gbit NICs in the office yet, being able to max out our Gigabit connection again is good enough for now.
It does seem like a bit of overkill for an office of less than 5 people to have a Steam caching server, but given what we do it make sense for us. For organizers who are running a LAN party, this would be a great solution to prevent game downloads/updates from saturating your link to the Internet.
I’m a little unclear why a
I’m a little unclear why a Steam-specific cache was chosen instead of just a regular http caching proxy that would take care of caching anything http without having to configure specific services.
Are you trying to download
Are you trying to download the entire internet, or just play some games?
Point is, you can more easily
Point is, you can more easily set up a less-specific web cache server that will not only maintain cache of all Steam games, but also Origin, Uplay, YouTube, Netflix, OS updates, and more. You can control what gets cached, not cached, or even blocked.
Would you like faster Steam,
Would you like faster Steam, or faster Internet which includes faster Steam? A traditional cache would reduce your download for the Internet as a whole. Why limit the speedup/download-reduction to only Steam when it is actually EASIER to reduce download and speed up everything?
The one catch is that caching everything would mean more disk space, but if you have enough disk space to cache a lot of Steam content, you probably also already have enough disk space to cache years of non-Steam and non-video content.
However, I will say that the general trend of the past couple years is “https everywhere”, which stops proxy caches from being effective for that content. Before reading this article, I would have guessed that Steam is surely https as well.
Are there similar service for windows platform?
Could you please link any
Could you please link any proxyes or tutorials for me to set up one of these (http caching proxy)? I’m getting a NAS and i have poor internet connection, and people here see a lot of the same content.
There are also some Docker
There are also some Docker containers for unRAID now, so it is pretty easy to set up on an existing server if you have one already. Guide here: https://squishedmooo.com/how-to-install-steamcache-on-unraid/
For some reason mine is not working for steam, but it is working for battle.net and Origin. Not sure what the problem is.
Why didn’t you just backup
Why didn’t you just backup steam games to a local file server and load them over 10gb lan?
Yes, exactly what Sean said.
Yes, exactly what Sean said. It’s a far more convoluted process to build a server to intercept steam requests than it is to just use the backup / restore feature that’s built into the client. You don’t need any fancy software and you can save the files anywhere you’d like.
One reason to do this is for
One reason to do this is for instance if you’re hosting a LAN party and you don’t know what games you’re going to be downloading before it’s underway. The first download of any game will be “slow”, but the following downloads will not stress the internet connection and it causes no disruption of the user experience for downloading a game from steam. Which is arguably just as important as most people are not tech savvy enough to be able to just “connect to this network location and use the steam restore feature on it”.
We’ve been doing it for years in our group, and it works really well for the purpose. And we don’t want to have all that data backed up.
So instead of fixing the
So instead of fixing the problem in one place with 2 commands you’d rather go to each individual client and run a manual backup/restore? That’s way more convoluted.
PCPer personal are the most
PCPer personal are the most relatable bunch of people in the world.
“Do you also have a gigabit connection? Are you also suffering from bellow gigabit download speeds? We have the solution for you!”
Jokes aside, I also suffer from having a gigabit connection. I’m glad this article addressed us 1% ers.
This article actually
This article actually addresses way more users.
Consider my case: We host a LAN party about 3 times a year. Something like 20 people show up, we try to have people agree on what games to play beforehand, but there’s always someone who hasn’t downloaded their game, there’s always some other game that noone thought of, or a free weekend of something that we want to try out.
So now we’re on a 50mbit connection and 20 people that need to download a 20gb game… well, one person downloads the game, the other 19 people wait till the download is done and then use the LAN cache and get around gigabit speeds from the cache. And they don’t need to do anything special, like access a network location and use the restore feature in steam. No, just click “install” on their game in steam and everything just works like normal.
You don’t even need 20 people or a fast connection for it to be a cool idea. The fewer you are arguably it becomes easier to do something else, like direct network share, but there’s a point where LAN-caching makes perfect sense.
Dear Malurt,
Could you please tell me what the speeds are of the first download? and what specs are you using for the server?
I’d kill to have a gigabit
I’d kill to have a gigabit connection.
It sucks being 100+ miles from the nearest ISP that offers that service.
I think its worse to be 1
I think its worse to be 1 mile from it and be unable to get it. Like me
I’ve got you beat. I have the
I’ve got you beat. I have the empty pipe already dug and placed up against my exterior wall. The only fiber company allowed to blow fiber through it won’t don it because I’m too close the the county border and they don’t want to annoy the competing fiber company on the other side of the red line of the map (which is themselves). *sigh*
same here… cant wait for it in my area.
I just copy my games from
I just copy my games from drive to drive /shrug
Do the caching servers also
Do the caching servers also cache and keep track of updates? if not then it will not work for many games that are regularly updated, e.g., a few months old copy of ark survival evolved, will mean a 50+GB download due to how they update their game.
Yes, it keeps track of
Yes, it keeps track of everything basically, so if you only need one patch you’re fine but if your friend shows up and hasn’t update their laptop in 6 months it will also have those in the cache.
You know what would be a nice
You know what would be a nice addition for those of us that are really lazy… a link to the time code in the weeks podcast in each story you talk about.. somewhat reverse of what’s in the podcast linking back to each story.
This solution is great!
But
This solution is great!
But unfortunately, it will not work for “epic games” like fortnite but only for games hosted on steam.
Is there any alternative, like creating an update patcher? It would be great of a solution is posted for this issue.
Thanks in advance.
Well the upside here is that
Well the upside here is that the type of person who will set this up and can actually afford the NAS isn’t 12 years old so he won’t have any interest in Fortnite
Waste of time. I get 80/90
Waste of time. I get 80/90 mb/s directly from my Cable Modem over 1000 down 100 up connection.
This is a very empowering concept. Would anyone know the logistics of setting up a public server where people with slow or small internet connections could rent space and download their large files either via 10 gigabit connection, or the more popular ones could be stored on a petabyte nvm caching server?
Anyone tried SteamOS?