What's Necessary to Get Full Performance Out of a Solid State Drive?

The storage hardware in the new consoles opens up new possibilities, but on its own the hardware cannot revolutionize gaming. Implementing the new features enabled by fast SSDs still requires software work from the console vendors and game developers. Extracting full performance from a high-end NVMe SSD requires a different approach to IO than methods that work for hard drives and optical discs.

We don't have any next-gen console SSDs to play with yet, but based on the few specifications released so far we can make some pretty solid projections about their performance characteristics. First and foremost, hitting the advertised sequential read speeds will require keeping the SSDs busy with a lot of requests for data. Consider one of the fastest SSDs we've ever tested, the Samsung PM1725a enterprise SSD. It's capable of reaching over 6.2 GB/s when performing sequential reads in 128kB chunks. But asking for those chunks one at a time only gets us 680 MB/s. This drive requires a queue depth of at least 16 to hit 5 GB/s, and at least QD32 to hit 6 GB/s. Newer SSDs with faster flash memory may not require queue depths that are quite so high, but games will definitely need to make more than a few requests at a time to keep the console SSDs busy.

The consoles cannot afford to waste too much CPU power on communicating with the SSDs, so they need a way for just one or two threads to manage all of the IO requests and still have CPU time left over for those cores to do something useful with the data. That means the consoles will have to be programmed using asynchronous IO APIs, where a thread issues a read request to the operating system (or IO coprocessor) but goes back to work while the request is being processed. And the thread will have to check back later to see if a request has been fulfilled. In the hard drive days, such a thread would go off and do several non-storage tasks while waiting for a read operation to complete. Now, that thread will have to spend that time issuing several more requests.

In addition to keeping queue depths up, obtaining full speed from the SSDs will require doing IO in relatively large chunks. Trying to hit 5.5 GB/s with 4kB requests would require handling about 1.4M IOs per second, which would strain several parts of the system with overhead. Fortunately, games tend to naturally deal with larger chunks of data, so this requirement isn't too much trouble; it mainly means that many traditional measures of SSD performance are irrelevant.

Microsoft has said extremely little about the software side of the Xbox Series X storage stack. They have announced a new API called DirectStorage. We don't have any description of how it works or differs from existing or previous console storage APIs, but it is designed to be more efficient:

DirectStorage can reduce the CPU overhead for these I/O operations from multiple cores to taking just a small fraction of a single core.

The most interesting bit about DirectStorage is that Microsoft plans to bring it to Windows, so the new API cannot be relying on any custom hardware and it has to be something that would work on top of a regular NTFS filesystem. Based on our experiences testing fast SSDs under Windows, they could certainly use a lower-overhead storage API, and it would be applicable to far more than just video games.

Sony's storage API design is probably intertwined with their IO coprocessors, but it's unlikely that game developers have to be specifically aware that their IO requests are being offloaded. Mark Cerny has stated that games can bypass normal file IO, elaborated a bit in an interview with Digital Foundry:

There's low level and high level access and game-makers can choose whichever flavour they want - but it's the new I/O API that allows developers to tap into the extreme speed of the new hardware. The concept of filenames and paths is gone in favour of an ID-based system which tells the system exactly where to find the data they need as quickly as possible. Developers simply need to specify the ID, the start location and end location and a few milliseconds later, the data is delivered. Two command lists are sent to the hardware - one with the list of IDs, the other centring on memory allocation and deallocation - i.e. making sure that the memory is freed up for the new data.

Getting rid of filenames and paths doesn't win much performance on its own, especially since the system still has to support a hierarchical filesystem API for the sake of older code. The real savings come from being able to specify the whole IO procedure in a single step instead of the application having to manage parts like the decompression and relocating the data in memory—both handled by special-purpose hardware on the PS5.

For a more public example of what a modern high-performance storage can accomplish, it's worth looking at the io_uring asynchronous API added to Linux last year. We used it on our last round of enterprise SSD reviews to get much better throughput and latency out of the fastest drives available. Where old-school Unix style synchronous IO topped out at a bit less than 600k IOPS on our 36-core server, io_uring allowed a single core to hit 400k IOPS. Even compared to the previous asynchronous IO APIs in Linux, io_uring has lower overhead and better scalability. The API's design has applications communicating with the operating system in a very similar manner to how the operating system communicates with NVMe SSDs: pairs of command submission and completion queues that are accessible by both parties. Large batches of IO commands can be submitted with at most one system call, and no system calls are needed to check for command completion. That's a big advantage in a post-Spectre world where system call overhead is much higher. Recent experimentation has even shown that the io_uring design allows for shader programs running on a GPU to submit IO requests with minimal CPU involvement.

Most of the work relating to io_uring on Linux is too recent to have influenced console development, but it still illustrates a general direction that the industry is moving toward, driven by the same needs to make good use of NVMe performance without wasting too much CPU time.

Keeping Latency Under Control

While game developers will need to put some effort into extracting full performance from the console SSDs, there is a competing goal. Pushing an SSD to its performance limits causes latency to increase significantly, especially if queue depths go above what is necessary to saturate the drive. This extra latency doesn't matter if the console is just showing a loading screen, but next-generation games will want to keep the game running interactively while streaming in large quantities of data. Sony has outlined their plan for dealing with this challenge: their SSD implements a custom feature to support 6 priority levels for IO commands, allowing large amounts of data to be loaded without getting in the way when a more urgent read request crops up. Sony didn't explain much of the reasoning behind this feature or how it works, but it's easy to see why they need something to prioritize IO.


Loading a new world in 2.25 seconds as Ratchet & Clank fall through an inter-dimensional rift

Mark Cerny gave a hypothetical example of when multiple priority levels are needed: when a player is moving into a new area, lots of new textures may need to be loaded, at several GB per second. But since the game isn't interrupted by a loading screen, stuff keeps happening, and an in-game event (eg a character getting shot) may require data like a new sound effect to be loaded. The request for that sound effect will be issued after the requests for several GB of textures, but it needs to be completed before all the texture loading is done because stuttering sound is much more noticeable and distracting than a slight delay in the gradual loading of fresh texture data.

But the NVMe standard already includes a prioritization feature, so why did Sony develop their own? Sony's SSD will support 6 priority levels, and Mark Cerny claims that the NVMe standard only supports "2 true priority levels". A quick glance at the NVMe spec shows that it's not that simple:

The NVMe spec defines two different command arbitration schemes for determining which queue will supply the next command to be handled by the drive. The default is a simple round-robin balancing that treats all IO queues equally and leaves all prioritization up to the host system. Drives can also optionally implement the weighted round robin scheme, which provides four priority levels (not counting one for admin commands only). But the detail Sony is apparently concerned with is that among those four priority levels, only the "urgent" class is given strict priority over the other levels. Strict prioritization is the simplest form of prioritization to implement, but such methods are a poor choice for general-purpose systems. In a closed specialized system like a game console it's much easier to coordinate all the software that's doing IO in order to avoid deadlocks and starvation. much of the IO done by a game console also comes with natural timing requirements.

This much attention being given to command arbitration comes as a bit of a surprise. The conventional wisdom about NVMe SSDs is that they are usually so fast that IO prioritization is unnecessary, and wasting CPU time on re-ordering IO commands is just as likely to reduce overall performance. In the PC and server space, the NVMe WRR command arbitration feature has been largely ignored by drive manufacturers and OS vendors—a partial survey of our consumer NVMe SSD collection only turned up two brands that have enabled this feature on their drives. So when it comes to supporting third-party SSD upgrades, Sony cannot depend on using the WRR command arbitration feature. This might mean they also won't bother to use it even when a drive has this feature, instead relying entirely on their own mechanism managed by the CPU and IO coprocessors.

Sony says the lack of six priority levels on off the shelf NVMe drives means they'll need slightly higher raw performance to match the same real-world performance of Sony's drive because Sony will have to emulate the 6 priority levels on the host side, using some combination of CPU and IO coprocessor work. Based on our observations of enterprise SSDs (which are designed with more of a focus on QoS than consumer SSDs), holding 15-20% of performance in reserve typically keeps latency plenty low (about 2x the latency of an idle SSD) without any other prioritization mechanism, so we project that drives capable of 6.5GB/s or more should have no trouble at all.

Latency spikes as drives get close to their throughput limit

It's still a bit of a mystery what Sony plans to do with so many priority levels. We can certainly imagine a hierarchy of several priority levels for different kinds of data: Perhaps game code is the highest priority to load since at least one thread of execution will be completely stalled while handling a page fault, so this data is needed as fast as possible (and ideally should be kept in RAM full-time rather than loaded on the fly). Texture pre-fetching is probably the lowest priority, especially fetching higher-resolution mimpaps when a lower-resolution version is already in RAM and usable in the interim. Geometry may be a higher priority than textures, because it may be needed for collision detection and textures are useless without geometry to apply them to. Sound effects should ideally be loaded with latency of at most a few tens of milliseconds. Their patent mentions giving higher priority to IO done using the new API, on the theory that such code is more likely to be performance-critical.

Planning out six priority classes of data for a game engine isn't too difficult, but that doesn't mean it will actually be useful to break things down that way when interacting with actual hardware. Recall that the whole point of prioritization and other QoS methods is to avoid excess latency. Excess latency happens when you give the SSD more requests than it can work on simultaneously; some of the requests have to sit in the command queue(s) waiting their turn. If there are a lot of commands queued up, a new command added at the back of the line will have a long time to wait. If a game sends the PS5 SSD new requests at a rate totaling more than 5.5GB/s, a backlog will build up and latency will keep growing until the game stops requesting data more quickly than the SSD can deliver. When the game is requesting data at much less than 5.5GB/s, every time a new read command is sent to the SSD, it will start processing that request almost immediately.

So what's most important is limiting the amount of requests that can pile up in the SSD's queues, and once that problem is solved, there's not much need for further prioritization. It should only take one queue to put all the background, latency-insensitive IO commands into to be throttled, and then everything else can be handled with low latency.

Closing Thoughts

The transition of console gaming to solid state storage will change the landscape of video game design and development. A dam is breaking, and game developers will soon be free to ignore the limitations of hard drives and start exploring the possibilities of fast storage. It may take a while for games to fully utilize the performance of the new console SSDs, but there will be many tangible improvements available at launch.

The effects of this transition will also spill over into the PC gaming market, exerting pressure to help finally push hard drives out of low-end gaming PCs, and allowing gamers with high-end PCs to start enjoying more performance from their heretofore underutilized fast SSDs. And changes to to the Windows operating system itself are already underway because of these new consoles.

Ultimately, it will be interesting to see whether the novel parts of the new console storage subsystems end up being a real advantage that influences the direction of PC hardware development, or if they end up just being interesting quirks that get left in the dust as PC hardware eventually overtakes the consoles with superior raw performance. NVMe SSDs arrived at the high end of the consumer market five years ago. Now, they're crossing a tipping point and are well on the way to becoming the mainstream standard for storage.

What To Expect From Next-gen Games
POST A COMMENT

200 Comments

View All Comments

  • PeachNCream - Friday, June 12, 2020 - link

    What makes a console a good piece of hardware is that I do not have to even care what is inside the box that runs the games since the games that I buy for said console will just work as intended. Every generation has extensive discussion about hardware and the usual comparing and penis measuring between brands. It all ends up being meaningless when you pick up a controller and play a game on it which is far and above the hassle of PC gaming in terms of equipment apathy. In short, IDGAF what drive is inside the thing. Insert disc (or download digital copy) and game on! Reply
  • shabby - Friday, June 12, 2020 - link

    Games will work just as intended? Are you suggesting console games never suffer from fps dips below 30fps or have lower quality graphics to hit 30fps? Reply
  • close - Friday, June 12, 2020 - link

    If you played on both console and PC then you know what that means. I do that and yes, on the console it's a lot more hassle-free. Every game is optimized for *that* configuration, perhaps with minor variations between the 2-3 cycle refreshes. No need to worry about "does it work better on the Radeon? Should I use the older driver? Adjust graphics settings to determine which one has the best graphics for *your* PC. Etc. It extracts the most out of that hardware.

    On my PC (with a far better config than any console) Assassin's Creed Odyssey freezes for a second (not a figure of speech) every once in a while even if it runs flawlessly otherwise. On the Xbox it works just fine. On the PC I had to adjust graphics because the highest level was just a tad too much but the next lower level left too much performance on the table, so I had to tweak a full screen (or 2) of settings about shadows, aliasing, and whatnot to get "the best for my setup". Maybe. I don't know, I can't be bothered to spend half a day experimenting. Now subjectively it looks better on the Xbox with a 4K screen than on the PC with a 1440p screen.

    On the other hand it also means 5 years from now you have an old GPU that you can't swap out. But for console gamers that's a small price to pay.
    Reply
  • Retycint - Saturday, June 13, 2020 - link

    You don't need to adjust the graphics configuration or mess with drivers because the option has been taken away from you. Instead of having the choice to slightly lower texture quality to ensure smooth gameplay, for instance, the game devs decide that a dip below 30fps is acceptable and push it onto you.

    Very strange to describe the lack of choices as "it just works"
    Reply
  • PeachNCream - Saturday, June 13, 2020 - link

    That's a pretty weak argument that doesn't take anything away from the original point of not having to care about the underlying hardware and being assured that a game purchased for a particular console will work properly on said console. Reply
  • vol.2 - Saturday, June 13, 2020 - link

    "will work properly" is what people are taking issue with. the argument is that there are games that run crappy on consoles. don't try to pretend that there haven't been plenty of prime examples over the past 10 years or so of games that are poorly optimized for consoles.
    On a PC, when that happens, you tweak settings until it's playable. On a console you have a shitty experience. Or maybe you just don't care for whatever reason. That's the point. Is a deal breaker for you, I guess not. It is for some.
    Reply
  • close - Sunday, June 14, 2020 - link

    I find this suspicious given that I play on both platforms and if anything, games are optimized *for consoles*. From performance, to gameplay, and controls. Devs make *a lot* more money from console sales since piracy is pretty much non existing so it makes sense they'd focus on them.

    As an anecdote I found far, far more games misbehaving on PC than on the Xbox. Between driver issues, "optimized for the that other hardware that you don't have" issues, trying to balance all the setting (most of which most people have no idea about), auto detect settings that "guess" what you should use based on some too simplistic checks, and basic game bugs I faced more problems on the PC. Yes, there are issues on consoles but the argument "well you have issues on both therefore the experience must be the same or worse on consoles" comes almost exclusively from people who "read about it", "heard from someone", etc. but never actually tried both for any meaningful period of time.

    Yes, for all intents and purposes games "will work properly" and with less hassle on consoles in more cases than on the PC. And I say this as someone who can definitely handle whatever a computing system can throw at them better than 99.9999% of the population. But that doesn't mean I *want* to have to handle that, especially on what's supposed to be my fun, hassle free time.

    Not trying to convince you to use either but if if you want to share your opinion the least you can do is be fully informed.
    Reply
  • hecksagon - Sunday, June 14, 2020 - link

    Couple counter arguments here.

    Just because the console game works as intended doesn't mean it doesn't run like shit. No developer ever wanted its fans to play their game at 30fps, just like no developer wanted their fans to play their game at 720p. Unfortunately with consoles its always been compromise. You either optimize for image quality or optimize for steady gameplay, or meet somewhere in-between.
    Reply
  • hecksagon - Sunday, June 14, 2020 - link

    Apparently I'm too stupid to find the edit button.

    The other point is that PCs, while more complicated, offer expanded options thanks to that complication. Want high refresh rate gaming? Not gonna get it on a console not only due to performance but due to HDMI limitations. Want to play an RTS? Good luck with a controller. Want an extra wide monitor to handle that increased FOV you are playing at? Your resolution isn't supported on consoles. VR? Don't waste your time with a console. Modding games? Gonna get you banned on a console. Emulation? Not likely on a console, better go get yourself a Playstation Mini.
    Reply
  • lunarx3dfx - Monday, June 15, 2020 - link

    I think what we have here is a classic case of one screen and two different movies. One argument is being made on subjective experience, and the other is being made from an objective point of view. I could be wrong, but I don't think OP is stating that consoles are more powerful, but more the subjective opinion that the just download game and play is more convenient for them and for a lot of people.

    As a dad I can understand this sentiment, which is why since my daughter was born I too have spent more time gaming on console. I find the lack of choice when it comes to performance settings as a bonus because I am that person who, if given the choice, will spend ages tweaking settings to get the best image quality at 60 fps. This however resulted in the limited amount of time I have available to play games being eaten up by time spent tweaking.

    In contrast, on my Xbox I just download the game and play. Is it an objectively better experience than running the game at maxed out settings on a PC at 60+ fps? No. Not at all. But the convenience is a factor in my overall enjoyment these days. However, there are some types of games I will NEVER play on a console. Doom for isntance. That is a PC only game in my eyes. You just can't beat a keyboard and mouse for that game. Period.

    I think where this argument always ends up coming from is personal opinion based on what is important to the end user. PCs are so much easier to build, setup, and play games on than in the 80's and 90's when I first got into PC gaming, but for many users it's still intimidating. That's why the console market is so large, and will continue to be. At the end of the day, many consumers still prefer the ease of setup and use that consoles afford, and you can't really put a price on that. A lot of people want to just take the machine out of the box, plug in two cables, download a game, and get playing.
    Reply

Log in

Don't have an account? Sign up now