Xbox 1080p Streaming Uses IIS, Smooth Streaming, But Not Silverlight

Microsoft has given me a few more details surrounding how they are managing to stream 1080p videos on the Xbox 360 with consumers only needing a 4Mbps connection. Microsoft said that Zune Video on Xbox LIVE takes advantage of Microsoft's Smooth Streaming technology, which is a feature of IIS Media Services and that Zune Video was built from the ground-up for the Xbox 360 platform. While the Silverlight plug-in also takes advantage of Smooth Streaming technology, Silverlight is not being used on the Xbox platform.

Microsoft says that Zune video on Xbox LIVE features a state-of-the-art encoder and high performance decoder that allows them to offer instant on 1080p video and 5.1 surround sound with Smooth Streaming playback and smooth fast-forward and rewind functionality. They aren't giving out any details yet on how the videos are encoded and what the settings are, but I expect we'll hear more about that when the service launches to the public. Microsoft also said that they recommend a minimum of 4 Mbps for the full 1080p/5.1 experience but the great thing about the technology is that videos will automatically scale to the best possible resolution based on the speed of the users internet connection.

I can't wait until Microsoft is willing to give out more technical details around the Zune Video technology and how it works. A 1080p video stream, with only a 4Mbps connection, is something this industry has never seen. It really is a big deal.

Comments

[Kieran Kunhya]

"A 1080p video stream, with only a 4Mbps connection, is something this industry has never seen. It really is a big deal."

Didn't Vudu do something similar?

What they've done here I suspect is run a strong film grain removal filter to remove the hard-to-compress film grain. They've then probably lowpassed it to remove all the high frequency details as has been done on DVDs and some Blu-Rays for a while now. Depending on the strength of said filters the content at worst has the same amount of detail as a DVD and at best about 720p of resolution. This is probably coupled with some boosting of the contrast/brightness to give it an "HD-pop". Also most films aren't full 1920x1080 because of the aspect ratios and they may have also done an "HD-lite" like TV stations do by offering content at 1440x1080 or 1280x1080 because humans are less sensitive to horizontal resolution. Because of the DRM we'll probably never know.

If they ever do release any info you can be sure that it will be coated in a shroud of marketing fluff. Having said that, credit to them for getting it deployed having jumped through all the DRM hoops.

Even without such aggressive filtering described above it's been possible to do 1080p at low bitrates; it's just that as you've said many times, finding someone to pay for this is difficult, along with getting a workable DRM solution for PPV content.

[Dan Rayburn]

Hi Kieran, VUDU says that that their HD encoding starts at 4.5Mbps and goes up to 9Mbps. So while their video quality is also exceptional, it's at a higher bitrate than the Xbox 1080p content.

[Mike Assel]

Dan, just curious if you've heard whether or not Microsoft plans on marketing this technology in a set top box only format. I'm specifically thinking of the Netflix player from Roku.

[Dan Rayburn]

I don't know. I have asked Netflix if they plan to support this, but so far, no comment from them.

[Alex Zambelli]

Kieran,

FYI, Smooth Streaming somewhat changes the classic video encoding paradigm. Smooth Streaming not only allows the player to change data rates (and therefore quality) on the fly in response to network conditions, but it also allows the video encoding parameters to change (at encoding time, of course) in response to content complexity. So measuring effective resolution isn't as straightforward as with regular video - the resolution could literally change every few seconds depending on content complexity.

The Smooth Streaming encoder (same one powers Zune/Xbox encoding as well as pro solutions like Inlet Armada) can decrease the spatial resolution of a particular GOP whenever the data rate becomes too low to produce acceptable video quality without noticeable artifacts. What does that mean? A video might be encoded as 1920x1080 at 4.5 Mbps and yield great quality at that rate for most of its duration, but during particularly complex scenes which would otherwise require significantly higher bitrates to produce the same level of quality - it reduces the encoded resolution to maybe 1440x1080, 960x540 or 1280x720 in order to avoid compression artifacts at 4.5 Mbps for that scene. The net result is a softer looking picture some of the time - but one consistently free of artifacts.

So the accurate description would be to say that a 1080p Xbox title is encoded at 1080p *most of the time* but with occasional scenes at lower resolutions.

[Fanfoot]

@Alex: I don't think the tracks in Smooth Streaming work that way. I think each track (bitrate) has a fixed set of encoding parameters, and if you want to change those parameters you have to change tracks. So a track that starts out at 1280x720 is always 1280x720. Course you can still switch to a different stream with a different frame size based on your network characteristics at the time. Also, remember how quantization works--if the bit rate must stay constant and the complexity goes up you effectively get blockier video anyway, which is the equivalent (somewhat) of reducing the resolution of the image anyway.

Anyway, other than the client controlled changing of the bit rate/track stuff, there is nothing else magically new here. The Smooth Streaming stuff is still based on the same h.264 video encoding as ALL those other services out there. So whereas AT&T U-Verse uses 7.5Mbps or higher to deliver an HD stream (about 1/2 the bandwidth Comcast would allocate for an HD MPEG-2 stream at 14.4-15Mbps), Microsoft is doing it with 4Mbps. Sure the encoders are still evolving, but it AIN'T gonna look as good.

Anyway, you can do 1920x1080 at ANY bit rate. It'll just look crummy at lower ones. Whether this looks good in ACTION SCENES or sequences where the CAMERA IS ROTATING or when displaying FIRE, RAIN or WATERFALLS is yet to be determined. I'm sure they can do great shots of largely static content. Those'll look really really great. But they don't tell you anything about what a football game will look like.

[Ben Waggoner]

@Fanfoot,

Rest assured, when it comes to compression, Alex ALWAYS knows what he's talking about :).

This technology is very similar to the Smooth Streaming encoding in Expression Encoder 3, which does dynamically lower resolution for high-motion scenes which would have artifacts at full resolution, but where motion blur hides the reduced resolution. It's quite effective.

[Alden Torres]

@Fanfoot

Alex is absolutely right in the reasoning. You can do all this sort of acrobatics from GOP to GOP. However, it remains a huge technical challenge, also you have a trade off between the GOP size and the smoothness of the streaming.

@Ben Waggoner

I follow all your comments on doom9 regarding Smooth Streaming. I was not aware of the nice feature you mention for EE3. Is this a feature available for both VC1 and H264? Do I have some sort of control in this heuristic? Thanks.