What is retina display quality? (macOS)
Retina quality passes along retina (i.e. HiDPI) data in the video stream from your Mac's display into your main Immersed display to enhance screen clarity. A higher setting will encode with better quality. It doesn't really affect the performance, but it will increase data size that affects latency in a slow Wi-Fi network.
For example, if your original resolution is 1920x1080, then 0 will render as 1920x1080 pixel and 10 will render as 3840x2160 for retina display.