Published on: Aug 18, 2023
Introduction
This blog post is aimed at evaluating the benefits and shortcomings of low- and high-frequency wireless communications for head-mounted displays (HMDs). Additionally, I will provide brief insight into the flexibility of extended reality (XR) application requirements, used to loosen the load that XR imposes on wireless networks, and evaluate how XR can benefit from joint low- and high-frequency radio spectrum usage. The reader can find an introduction to XR application requirements and the motivation for enabling wireless XR content transmission in my previous blog posts, [1] and [2], respectively.
Figure 1: VR users employing a cable for reliable high-data content streaming [7].
Providing sufficient data rate
Streaming XR video from a nearby computer or edge node to the HMD requires a throughput rangingfrom 10 Mbps to 100 Gbps, or even more. For example, encoded 1K video rendered at 60 frames per second (FPS) generates data at a rate of roughly 10 Mbps, while 4K raw streaming at 240 FPS corresponds to about 90 Gbps. Such data rates dwarf the typical usage patterns of today’s gadgets (laptops, smartphones, etc.).
The amount of data that a wireless signal can carry is closely related to the available bandwidth. That is, data rate is proportional to the number of unique frequencies that make up the (wireless) signal. In the European Union, there is about 10-times as much unlicensed “harmonised” bandwidth available in the millimeter-wave (mmWave) spectrum, as there is in the frequency range below 15 GHz, where all current Wi-Fi standards operate [3,4]. From another perspective, the entire 0-15 GHz bandwidth corresponds to one tenth of the frequency spectrum considered for the next generation of wireless standards, operating at up to 150 GHz. Hence, high-frequency wireless technology, operating above 15 GHz, inherently provides data rates in the order of Gbps due to the larger available bandwidth. This makes high-frequency wireless technology an interesting option for XR applications.
Nevertheless, sub-15 GHz technology can also offer multi-Gbps data rates by incorporating advanced techniques, such as simultaneous transmission through multiple antennas, scattered around the XR user. However, this requires additional planning and increases costs. Applying the same techniques to mmWave technology would yield data rates in the order of tens of Gbps, on par with those of a dedicated HDMI or display port cable.
Offering the required resilience
A high data rate on its own is not sufficient, as the XR application requires a stable data link to avoid judder (dropped frames). Failing to do so leads to motion sickness and an overall bad user experience.
Since the smaller wavelength (higher frequency) requires proportionally smaller antennas, the total gain of the system drops. To counteract this, most mmWave devices employ antenna arrays. This also stabilizes the wireless channel, mitigating the otherwise more pronounced small-scale data link volatility at higher frequencies, which exceeds the scope of this post. However, the employed antenna arrays yield directional communication, referred to as beams, which easily get misaligned as an XR user moves. To provide reliable communication, the HMD must frequently sense the wireless channel (its environment) and update its beams accordingly. Moreover, an external obstruction, e.g., a person that blocks the main beam, will also result in the need for beam adaptation.
Lower frequencies suffer less from these issues, as the antennas or antenna arrays are typically less directional and can capture more signal components at once, resulting in higher resilience. Moreover, the lower-frequency spectrum is less prone to attenuation by matter (e.g., people) and absorption in gases. However, applying advanced transmission techniques for providing multi-Gbps data rates, such as concurrent transmission through distributed infrastructure antennas, requires precise wireless channel knowledge. As noted earlier, the wireless channel can change rapidly when XR users move and rotate, which requires constant adaptation. This can result in data link outage if the channel is not sensed frequently enough.
Role of low- and high-frequency bands
The ability of lower-frequency technology to withstand adverse conditions, such as blockage and XR user movement, makes it fit for XR applications that require high reliability and less throughput. For example, a robotic arm operator wearing pass-through glasses that display distances and virtual guides. Moreover, the lower-frequency bands can always act as a low-resolution fallback when channel conditions are too adverse for establishing a reliable high-frequency data link.
Contrarily, high-frequency technology can transport high-fidelity XR content in prosperous wireless channel conditions. For example, immersive firefighter training in a controlled environment, i.e., in a dedicated room. Furthermore, by applying some of the previously-mentioned advanced transmission techniques, high-frequency wireless technology can substitute physical cable connections even as HMDs push towards 8K resolution at more than 120 FPS. High-frequency communications can also benefit dense multi-user XR deployments, such as the spectators at a sports event, since the smaller wavelength and more numerous antennas provide better angular resolution. I.e., it is easier to transmit data to a given user, without also sending it to other users and causing interference. Hence, vastly increasing their combined data rate (also called sum rate).
Ideally, XR applications would be able to benefit from both low- and high-frequency spectrum concurrently by transmitting and receiving important time-sensitive data over the more robust lower-frequencies and leveraging the high-frequencies for receiving the high-fidelity video stream. For example, an HMD can relay small amounts of time-sensitive sensor data, used for positioning and pose estimation, to the edge node and receive precise video stream synchronisation data in return, both at lower frequencies. Provided there is sufficient data rate available, the edge node can also transmit a low-resolution duplicate of the video stream over the low-frequency wireless technology, whereas the higher-frequency spectrum is employed for transmitting a high-fidelity video stream to the HMD. In case of sudden outage, e.g., due to a fast rotation, the dropped high-resolution video frames are automatically substituted by their low-resolution counterparts, while video content rendering can continue uninterruptedly since the sensor data still arrives intact at the edge node due to its transmission at lower frequencies.
Loosening the requirements
While data rates surpassing 1 Gbps are generally perceived as the requirement for seamless wireless streaming, there are a few cunning shortcuts that XR systems can leverage.
For starters, the HMD does not have to display full-resolution video data on the entire screen. Instead, it can leverage information about the XR user’s gaze and apply foveated rendering, where only the parts of the scenery that a user is focusing on, are rendered in high-definition. Foveated rendering can reduce the data rate requirement by at least an order of magnitude; however, it’s essential to have precise eye-tracking technology and transmission to the edge processing node with minimal latency in order for foveated rendering to work effectively [5].
Additionally, the XR application can tailor, and possibly relax, the video latency constraint according to each individual user. Therefore, the HMD can increase the video buffer in cases where the user shows no signs of motion sickness, allowing the data rate to drop at times or using the profited time for conducting additional channel estimation.
Such principles are possible by measuring electrodermal activity, which fluctuates according to user arousal and general well-being [6].
Conclusion
Low- and high-frequency technology both have their own benefits and drawbacks, while joint operation might yield the best trade-offs. Focusing on the Wi-Fi family of standards (IEEE 802.11), there is a clear distinction between low- and high-frequency support. The more common Wi-Fi standards all operate below approx. 7 GHz, while the two high-frequency standards (termed WiGig) operate at approx. 60 GHz. Even though they share the same foundations, there is still a significant separation between them, which renders their interplay prohibitively slow and unreliable for XR applications. Hence, research is now exploring ways of interleaving the entire radio spectrum to reap the benefits of both low- and high-frequency wireless communication and cater to recent technological advancements, such as XR, autonomous vehicles, and Industry 4.0. Furthermore, the inclusion of additional sensor data, such as eye tracking and EDA measurements, can alleviate the strain imposed by XR applications on wireless networks and help facilitate their further inclusion in future wireless networks.
If you were able to stick until the end and can’t wait for more content and you also want to know about us and our projects, you can always follow our social media channels.
References
[1] Marinsek A. (2021). Every Millisecond Counts, https://b5g-mints.eu/blog11/
[2] Marinsek A. (2022). Millimeter-wave for transporting extended reality video, https://b5g-mints.eu/blog23/
[3] Commission Implementing Decision (EU) 2019/1345 of 2 August 2019 amending Decision 2006/771/EC updating harmonised technical conditions in the area of radio spectrum use for short-range devices, https://eur-lex.europa.eu/eli/dec_impl/2019/1345/oj
[4] EU Radio spectrum decision living document,
[5] UploadVR (2022). Foveated rendering performance benefits, https://uploadvr.com/quest-pro-foveated-rendering-performance/ https://digital-strategy.ec.europa.eu/en/library/radio-spectrum-decisions
[6] Egan et. al (2016). An evaluation of Heart Rate and Electrodermal Activity as an Objective QoE Evaluation method for Immersive Virtual Reality Environments, IEEE QoMEX, DOI: 10.1109/QoMEX.2016.7498964
[7] Photo by Lucrezia Carnelos on Unsplash
