The Vision
It's simple. A virtual workspace capable of multi-tasking, using the tools and applications I'm already accustomed to.
What this means in practice is a linux desktop environment, ideally (but not necessarily) one that lets me use an existing linux box I have in my home.
Trials and Tribulations
The overarching theme of my experience with the Meta Quest 3 is that, for what I want to use it for, it is so excruciatingly close to being a good solution. The image clarity in VR is great, but the passthrough is only about 80% there -- things are fuzzy and blurry and shift in and out of focus at seemingly random intervals. It's workable, but annoying. The interface is decent, but there are little things here and there that don't work how you intuitively think they would. The room scale tracking is great... but over time it drifts and there's no way to edit an existing boundary, you can only re-create them. And finally, support for remote desktop access for linux is in a rough spot.
Using Official Apps
The only officially supported remote desktop application that provides a linux server at the moment seems to be Immersed. The app itself is decent, providing a rotating selection of free virtual environments with more available for cash. You can create pass-through portals for things in the real world, which is handy for if you need to glance down at your keyboard or reach over and grab your cup of coffee. It also handles audio automagically.
The downsides are the non-FOSS nature of it all, the fact that they only provide an AppImage (which required some tinkering to get working due to a busted CA cert at login for me, for some reason), and the fact that it is a bit of a resource hog, with fluctuations in streaming quality. Sometimes I would get crystal clear, responsive visuals with perfect audio sync, and sometimes it would turn into a slideshow -- all over the course of 10 or 15 minutes, with no apparent rhyme or reason.
Using RDP
RDP ended up being too laggy for practical use in all of the combos of server and client that I attempted. Getting audio working mostly out of the box was nice, though.
Using VNC
VNC offered much better visual clarity and less latency (and better security with the proper configuration), but meant that other solutions had to be found for piping audio over from the remote side. The server used was LibVNC's x11vnc, which I have no problems with. Highly configurable, fast, lightweight, and easy.
For the client side, there are a number of android apps available to sideload onto the quest, of which I tried three. Yet untested were any paid solutions and MultiVNC (which I have heard good things about, but have not found the time to try).
Client: RealVNC
Horrible resource hog, laggy, non-FOSS, better quality gated behind paywall. Dead dove, do not eat.
Client: AVNC
A decent option. Reasonably fast (though still too slow to realistically watch videos with), FOSS, decently configurable. The biggest downside was that mouse support with Quest 3 is lacking. A right-click is mapped to android's "back" key which exits the active VNC session, so all three mouse buttons have to be mapped to left-click. Additionally, I could not find a way to get it to send a held key repeatedly (leading to having to press j-j-j-j-j-j-j-j-j-j-j-j-j to scroll through a document in vim, for example). And since it's pure vanilla VNC, no audio.
Client: bVNC
I unfortunately could not get this one to work no matter what combo of server and client options I tried, the visuals were always a jumbled mess. Maybe if I played around with different versions I could have gotten something stable, but it was a no-go.
Bonus: NoVNC
NoVNC is a web-based front-end for VNC. The visual quality was on-par with AVNC, and it didn't have
any issues passing mouse events through which was very nice. However, because you're relying on the
Quest's built in web browser for a connection, not all keyboard events get sent. No Alt+ combos, no
function keys, and no Ctrl+, and... no backspace. Puts a bit of a damper on coding in vim.
Running X11 on Quest
Yeah... yeah, that's a thing you can do.
Prerequisites
- The ability to sideload apk's on your quest
- A Github account (for downloading artifacts)
- Developer mode enabled for both your Meta account and on your Quest headset
Install and Prep
- Grab the latest termux apk
- Download the appropriate artifact for your device from the most recent successful build of termux-x11 (logging in to GitHub required)
- Sideload both apk's to your Quest
-
Disable Android's phantom process killer, using an adb shell connected to your Quest (steps here)
$ adb shell "/system/bin/device_config set_sync_disabled_for_tests persistent" $ adb shell "/system/bin/device_config put activity_manager max_phantom_processes 2147483647" $ adb shell settings put global settings_enable_monitor_phantom_procs false -
Launch Termux and install x11 and the window manager of your choice (xfce4 in this case)
$ pkg install x11-repo termux-x11-nightly xfce4
Usage
-
Launch Termux and run the following command to start the X server with your chosen desktop environment
$ termux-x11 :1 -xstartup xfce4-session -
Launch the Termux-X11 android app
Et voila, a full desktop environment running on your Quest. It's worth noting that you can connect a bluetooth keyboard and mouse to your Quest to get a more natural experience. However... you're still limited to the Quest 3's native resources which can only get you so far.
My Current Solution
After much, much trial and error, I've settled on a decent compromise between the above options. I
run termux-x11 and launch tigervnc-viewer to VNC in to my main linux box (tunneled over SSH). I
additionally use PulseAudio for sound (also tunneled).
For reasons unfathomable to me, the best remote desktop experience in terms of visual clarity, latency, and keyboard/mouse compatibility is not a native Meta app or even native android, but termux+x11+tigervnc.
Suggested Settings
...specifically for using with an external bluetooth keyboard and mouse.
Change the touchscreen input mode to "Direct touch" in the Termux-x11 settings to enable normal clicking and dragging. Additionally enable "Automatic uploading clipboard to android" to share your clipboard.
The only minor remaining frustration is that, with my chosen bluetooth keyboard (A Thinkpad track point model), scrolling does not work. Still trying to figure that one out.
One of these days I should probably try installing a game onto this thing...