frametop

Remote displays (exploration)

Status: exploration. Spike S1 is done; nothing in Frametop has changed yet. Branch remote-displays, written 2026-10-06.

The idea: show desktops streamed from other machines as Frametop panels that behave like the native screens. They get placement, curve, pinning, layouts, lasers, gaze, and attention-based frame rates. The target is up to 5 remote displays from 2 hosts, with no more headset overhead than 5 native screens.

What the hardware gives us

These were checked on the Frame (SteamOS kernel 6.18, SM8650 / Snapdragon 8 Gen 3, Mesa 26.3 Turnip on Adreno 750).

So a decoded frame needs one GPU pass before SteamVR sees it: NV12 to RGBA, about 0.13 ms of GPU time for a 3440x1440 frame at full clock. It is the same kind of pass as the hand cutouts’ (screens/handcut.cpp), and native screens pay for one of the same size: KWin’s composite of each output.

Is the decoder the bottleneck?

Mostly no. Macroblocks per second against the 7,776,000 budget, at 60 fps:

Displays Share of the decoder
5x 1920x1080 31%
5x 2560x1440 56%
The current layout (3440x1440 + 2x 1440x1920) 32%
5x 2560x1440 at 90 fps 83%
4x 3840x2160 100%
5x 3840x2160 125%, refused
Steam Link VR’s own stream (1152x4608 at 120 Hz) about 32%

Five 1440p displays fit at full rate with room left. Five 4K displays only fit at about 48 fps or less. That number is what gets declared at session start; actual load is lower, because hosts send frames only when the screen changes (see “Keeping the decoder under budget”).

The planned setup

Two hosts that share the same two desk monitors: a Mac (14-inch MacBook Pro) with those two plus its built-in display, and the test PC (Windows, RTX 5090) with the two. Windows and Linux hosts run Vibepollo, which streams real displays and creates virtual ones on demand. The Mac runs Sunshine and is limited to one display for now (see “Host side”). At 60 fps:

Display Share of the decoder
Super ultrawide 5120x1440 (5160x1440 as given; same load within 0.2%) 22.2%
Built-in 3024x1964 (native pixels) 17.9%
Ultrawide 3440x1440 14.9%
Mac, one display 14.9% to 22.2%
The test PC, ultrawide + super ultrawide 37.1%
Both hosts today (three displays) at most 59.4%
Five: the three above plus two 3440x1440 virtual displays on the test PC at most 89.2%
The original five (all three Mac displays + the test PC’s two) 92.2%

Every combination fits under the driver’s limit at 60 fps. The original five stay as the worst case for spike S2, so the Mac can grow past one display later without a new budget. These are declared numbers, with every display changing every frame at once. With damage-driven hosts the real load is far lower.

Notes:

A remote display should cost less than a native one everywhere else:

Per display Native screen Remote display
Apps Run on the Frame’s CPU Run on the host
Composition KWin draws each output on the Adreno GPU One GPU pass per new frame turns the decoder’s NV12 into RGBA
Hand-off to SteamVR XRGB dmabuf, zero copy RGBA UBWC dmabuf, zero copy
Sampled by vrcompositor 4 bytes per pixel The same
New work — Network receive and reassembly, V4L2 queueing

The new costs are CPU for receiving packets, plus the decoder’s and Wi-Fi radio’s power and heat. Heat matters because the SoC slows down when it gets hot. Those are what the spikes need to measure.

Keeping the decoder under budget

  1. Damage-driven hosts. Sunshine and its forks send a new frame only when the captured screen changes, plus duplicates down to minimum_fps_target (default half the stream rate, settable to 1). With minimum_fps_target = 1, a static display costs about one decoded frame a second, the same idea as a native screen with no damage. A display playing video costs full rate, as a native video screen does.
  2. Out of sight means 1 frame per second, whatever the host sends. A display outside your view updates once a second, even when the host is sending 60 fps from a game. See “Displays out of sight” below for how. Concealed panels, and every panel while the desktop is paused for a game, disconnect fully after a grace period. That frees the host’s encoder and the network.
  3. An admission budget. Before connecting, Frametop adds up the declared load of every stream: width/16 × height/16 × fps. If a new stream would go past the driver’s limit, minus whatever else is decoding, Frametop lowers settings before connecting: first fps, then resolution, starting with the displays that get the least attention. Today’s three displays come to at most 59%, which fits next to Steam Link VR’s stream (about 32%). Five displays (89-92%) fit alone but not next to it, so a streamed PC game with five remote panels up would push them down to about 45 fps (or some lower and others higher).
  4. No B-frames, low-latency decode. Sunshine doesn’t send B-frames, so every decoded frame can be shown at once. The decoder gets a small capture pool (6-8 buffers). Valve’s client found the standard DISPLAY_DELAY controls unsupported on this driver (EINVAL in vrlink.txt). They aren’t needed: S1 showed that with Q08C each frame comes out as soon as it’s decoded (linear NV12 holds two more).
  5. Tiling only as a fallback. Valve tiles both eyes into one frame because they always change together. Desktops don’t: one busy display would make the whole canvas decode at full rate, and a hidden tile can’t be skipped. Per-display streams keep each display’s cost proportional to its own changes. Tiling makes sense only for a host with many small, mostly static displays, and Sunshine can’t capture a spanning desktop on Windows anyway.

The Moonlight protocol can’t change resolution, fps, or bitrate mid-stream; that takes a reconnect. With stock hosts, attention changes therefore act on the client: in what gets decoded (2), not in what the host sends.

Displays out of sight

Remote panels follow the native screens’ attention rules (UpdateAttention, vr.cpp:855):

Attention Native screen Remote display
Focused (within 12° of where you look) Full rate Full rate
In view (within 60°) 15 Hz, or full rate while it plays video Full rate. Every frame has to be decoded anyway (below), and the host sends only what changed
Hidden (out of view) 1 Hz 1 Hz, even when the host sends 60

The “video” signal comes for free: a damage-driven host sends frames only when its screen changes, so frames arriving faster than 10 a second for 8 frames in a row mean video, matching VIDEO_COMMITS and VIDEO_HZ in compositor.c.

The stream’s fps is ours to choose. A game running at 144 Hz on the host is captured and sent at the fps the stream asked for (60 at most), so 144 never reaches the decoder.

Why 1 Hz can’t just skip frames. Each frame in a stream is coded as changes to the frame before. Decoding frame 60 needs frames 1 to 59. Throwing away 59 of every 60 frames breaks the chain, and the next frame decodes as garbage. Valve’s client works the same way: on a stall it asks the host for a new full frame (an “IFrame” in vrlink.txt).

With stock hosts (Moonlight protocol): keyframe sampling. While a panel is hidden, ft-stream:

  1. drops every incoming frame before the decoder (moonlight-common-c’s decode callback gets a DECODE_UNIT with frameType; return DR_OK without queueing);
  2. calls LiRequestIdrFrame() once a second, and decodes and shows only the IDR frame that comes back, which needs no earlier frames;
  3. when the panel comes back into view, requests one more IDR and goes back to decoding everything. The last frame, at most a second old, stays up until it arrives, about one round trip plus one frame later.

What that saves and what it doesn’t, for a hidden display whose host sends 60 fps:

Cost Saved?
Decoder work About 59 of 60 frames. One IDR costs a bit more to decode than one change frame
Panel updates in vrcompositor 59 of 60
Network traffic No. The host keeps sending 60 fps, plus one larger IDR a second. A 5120x1440 IDR is a few hundred KB (estimate), so a few Mbit/s extra
Frame CPU to receive, reassemble, and repair packets No. moonlight-common-c still handles every packet. Patching it (it’s GPL, and so is ft-stream) to discard hidden video packets early would cut most of this
Host encoder No

With a host that takes an fps change mid-stream: real 1 Hz. If the host can be told “this display is out of sight, send 1 fps”, the hidden display costs almost nothing anywhere. That’s network, Frame CPU, decoder, and host encoder alike, and no IDRs are needed. The 1 Hz frames are ordinary change frames against the one a second earlier. This needs our own streamer, or a Sunshine fork with a control message that changes the capture rate. Owning a host streamer was rejected for its maintenance cost (see “Alternatives considered”), so this stays a possible upstream contribution. Vibepollo is the likeliest place to propose it: it already extends the protocol, since its own Moonlight fork sends a VRR pacing request that changes how the host captures.

Spike S3 checks the stock-host version: whether Sunshine honours an IDR request every second, how big and how late those IDRs are, and what receiving a hidden 60 fps stream costs the Frame’s CPU.

Proposed architecture

Frametop maintains only the headset side. The hosts run existing, separately maintained streamers that speak the Moonlight protocol: Vibepollo on Windows and Linux, Sunshine on macOS (see “Host side”).

host machine: Vibepollo (Windows, Linux) / Sunshine (macOS), one stream per display
   │ RTSP + RTP over UDP, ENet control
   ▼
ft-stream (one process per remote display, on the Frame, GPLv3)
   moonlight-common-c (protocol) + pairing (moonlight-embedded's libgamestream)
   iris decoder session: OUTPUT = HEVC, CAPTURE = Q08C, VIDIOC_EXPBUF
   GLES pass: Q08C → ring of 3 RGBA buffers (EGL YUV hints = the stream's colour space)
   │ unix socket, SCM_RIGHTS
   ▼
ft-screens (existing, MIT)
   remote screen type: ImportDmabuf once per ring buffer,
   SetOverlayTexture per frame, buffer returned to ft-stream when replaced
   panel input → ft-stream → LiSendMousePositionEvent / LiSendKeyboardEvent / …

Why one process per display. moonlight-common-c keeps global state, so it runs one stream per process. It and libgamestream are GPLv3 while Frametop is MIT, and a helper that talks over a socket keeps the licences apart (this isn’t legal advice). ft-stream lives in its own directory with its own licence file. Separate processes also mean a stalled stream or a decoder reset can’t freeze the native screens. Each process pairs as its own client, which Vibepollo needs anyway: it ties each Remote Monitor to the client that opened it, one role per client, so every display needs its own client certificate and pairing.

Pairing with a host token (chosen 2026-10-07). Since every display is its own client, PIN pairing would mean one PIN and one round of permission clicks per display. Vibepollo has neither a multi-use PIN (its one-time PINs pair one client each) nor a way to pair several certificates at once. It does have scoped API tokens for its Web UI API (POST /api/token, sent as Authorization: Bearer). So the user creates one token on the host, limited to submitting pairing PINs (POST /api/pin), listing clients and setting their permissions (GET /api/clients/list, POST /api/clients/update), listing the host’s monitors (GET /api/display-devices), and placing Remote Monitors (GET/PUT /api/clients/display-layout). A script on the host makes it from the Web UI login (make-frametop-token), and the user enters it once in Frametop’s “add host” dialog. From then on, Frametop makes a client for each new display, pairs it by sending its own PIN, and grants it launch, mouse, and keyboard (Vibepollo gives a new client only list and view). It sizes Remote Monitors like the host’s real monitors, and unpairs a display’s client with that client’s own certificate when the display is removed. The token can’t change the host’s settings and can be revoked in the Web UI. It can change any client’s permissions, though, so Frametop keeps it like a password (a file only the user can read). Vibepollo’s client update replaces the whole client record, so Frametop always sends every field.

Why ft-stream converts. ft-screens then gets RGBA dmabufs, as it does from KWin. Each decoder buffer goes back to the decoder right after the pass, so the decoder’s pool doesn’t depend on what SteamVR still holds. A GPU fault in one stream stays in its own process. ft-stream asks the host for BT.709 limited range (Moonlight’s colour space and range settings) and gives the converter the same as EGL hints. S1 showed all four matrix and range combinations come out right when the hints match the stream.

Opening a display. For a host’s main session, ft-stream launches the host’s desktop app as Moonlight does. For a Vibepollo virtual display it launches the synthetic “Remote Monitor” app (id 2147483505 in remote_session.h), and the requested stream size and fps become the virtual display’s mode. When the stream drops, Vibepollo keeps that display and its windows by default (remote_monitor_disconnect_on_stream_end = false), and ft-stream reconnects with “Resume” (id 2147483501). So a concealed panel can disconnect fully without the host rearranging its windows.

The socket protocol, roughly:

In ft-screens, the explorer’s map gives the seams:

Input. handle_vr_event (compositor.c:333) branches on the screen type. A remote screen sends:

A drag can cross panels, for example a window dragged from one of the host’s displays to another. SteamVR sends a held button’s moves only to the panel where the press began, with coordinates off that panel once the laser leaves it. The panel under the laser gets nothing (S3 measured this). So while a button is held on a remote screen, ft-screens hit-tests the laser (ComputeOverlayIntersection) against the host’s other remote screens and sends the position to the ft-stream of the screen it hits. Moves that land on no panel are dropped, never clamped: clamping pins the host’s cursor to the first display’s edge. The host’s input is shared across its sessions, so Windows sees one drag. But the release goes through the stream the press went through: Vibepollo takes a mouse button’s release only from the client that pressed it (mouse_press_owner in its src/input.cpp). Sent through the display under the laser, it was dropped, and the window stayed on the pointer (found in the headset, 2026-10-07; g_pressed_on in remote.c).

Keyboard focus follows the last panel clicked. When it’s a remote one, send_key and relay_button send evdev codes to its ft-stream, which maps them to Windows virtual-key codes (moonlight-qt has the table). Sunshine on macOS maps the Windows key to Cmd and Alt to Option. The host draws its own cursor into the video, so it lags the laser by one round trip.

Won’t work across the boundary: drag and drop, the clipboard (the Moonlight protocol has none), floating windows, and KWin window rules. A remote display is a picture of another machine’s monitor.

Host side

Decision for v1 (2026-10-07): real monitors only. Vibepollo 2.0.0’s Remote Monitors broke the test PC’s monitor layout every time one was removed (see “S3 results so far”), while streaming a real monitor changes nothing on the host. Frametop can make extra screens in the headset itself, and the feature is mostly for controlling other machines, so v1 streams only a host’s real monitors; virtual displays wait until the Vibepollo bugs are fixed. One host program streams one real monitor (a second client launching the main app joins the display already streaming), so each extra real monitor needs its own host instance: its own config file and ports (port 100 apart) and output_name set to that monitor. On the test PC that’s Vibepollo for the primary (it also serves the Steam Deck and the Mac) and a plain Sunshine instance for the LC34G55T. Plain Sunshine has no scoped API tokens, so a Sunshine instance is paired once by PIN; it has a single Frametop client anyway. Frametop’s host bundle becomes a list of instances, one per real monitor.

The requirement: each host streams its real displays, and virtual displays it creates on demand. Frametop doesn’t maintain a host streamer; it uses existing ones and ships setup notes or scripts for them. State checked 2026-10-06.

Host OS Streamer Displays
Windows Vibepollo 1 main session (real or virtual) + up to 4 virtual Remote Monitors
Linux (Arch, CachyOS; beta) Vibepollo The same, with at most 4 virtual displays at once
macOS Sunshine 1, the Mac’s main display, for now

Windows and Linux: Vibepollo

Vibepollo 2.0.0 (2026-09-30) is a Sunshine fork. One install gives:

That covers virtual displays completely, up to five from one host. Real displays are the limit: only the main session shows one, so one Vibepollo install streams one real display. A second real display at the same time needs a second host instance with its own config file, port, and output_name. On Windows that could be a plain Sunshine instance next to Vibepollo (untested). On Linux, Vibepollo expects to be the only host install on the machine. For the test PC this means one desk monitor streams as the main session and further displays are virtual, unless S3 shows a second instance works. Virtual displays also avoid the input-switching question (see “The planned setup”).

Settings Frametop’s setup notes change from the defaults:

Both platforms send frames only when the screen changes. On Linux the virtual displays use presentation-driven capture: sparse changes are captured at once, and faster ones are coalesced to the stream’s fps. The virtual outputs have no cursor plane, so the cursor is in the video, as with every Moonlight host.

PyroWave, Vibepollo’s wavelet codec, isn’t usable here. It decodes on the client’s GPU and needs hundreds of Mbit/s over wired LAN. Frametop uses HEVC.

Windows (the test PC). The chosen setup (2026-10-06): one desk monitor streams as the main session, a real display (virtual_display_mode = disabled, output_name set to it). The other desk monitor is replaced by a Remote Monitor, a virtual display at that monitor’s size, which Vibepollo releases when Frametop ends the connection (remote_monitor_disconnect_on_client_disconnect = true). A dropped connection keeps it (remote_monitor_disconnect_on_stream_end = false), so a Wi-Fi drop or a panel that disconnects while out of sight doesn’t move its windows; ft-stream releases it explicitly with “Disconnect Monitor” (id 2147483502) when the remote display is removed or Frametop exits. Each Remote Monitor’s place next to the real monitors is set per client in the Web UI. GeForce allows 8 concurrent NVENC sessions per system. Installing Vibepollo and its display driver needs an admin; the test PC’s maptrainer account isn’t one.

How it’s set up on the test PC (2026-10-06). Vibepollo installed over Apollo in C:\Program Files\Apollo (service ApolloService) and kept Apollo’s settings, pairings, and Web UI login. The existing clients (a Steam Deck and a Mac) launch “Desktop” or “Steam Big Picture”, which stream a virtual display with the other monitors turned off (dd_configuration_option = ensure_only_display). Frametop leaves them alone and gets its own app instead:

The config folder is admin-only, so the change is a script for the user to run (D:\remote-displays\vibepollo-setup\run-setup.cmd, with a backup and an optional Web UI password reset). It ran on 2026-10-07. Vibepollo rewrites apps.json each time it starts and adds its built-in “Remote Input” and “Remote Monitor” apps, so any later change must start from the live file.

Linux (beta). x86_64 only: Arch Linux or CachyOS, KDE Plasma 6 on Wayland started by SDDM or Plasma Login Manager, Linux 6.16 or newer with matching headers, and a GPU with hardware H.264 encode (tested on NVIDIA and modern AMD). Virtual displays come from Vibepollo’s own DKMS module, vibeshine_drm, which has four connectors, so at most four virtual displays exist at once. Streaming before login needs NVIDIA. Other distributions aren’t covered; plain Sunshine still streams real displays there, without virtual ones.

Risk. Vibepollo is new, moves fast, and has one maintainer, and its README says about 99% of its code is AI-generated. Frametop depends only on its Moonlight-protocol behaviour and the Remote Monitor app ids, so on Windows Apollo (virtual displays through the SudoVDA driver) or plain Sunshine (real displays) stay as fallbacks.

macOS: Sunshine, one display for now

Vibepollo has no macOS build. Sunshine (v2026.914, labelled experimental on macOS) is the only Moonlight-protocol host for the Mac, so the Mac streams one display: its main display, from one Sunshine instance. The reasons:

The main display is the one with the menu bar (System Settings → Displays). Capture is at backing pixel size, so the built-in display streams at 3024x1964. Closing the lid removes the built-in display, so a stream of it ends. Capture stalls if the display sleeps mid-stream (#5509).

One display can still reach the encoder’s limit. Sunshine forces VideoToolbox’s low-latency mode, which roughly halves throughput (#5814); an M4 Pro managed 0.36-0.46 Gpix/s in that mode. The super ultrawide at 60 fps is 0.44 Gpix/s, so full-motion video on it may need a lower fps or a smaller stream. The other two displays need 0.30 and 0.36 Gpix/s.

What would lift the limit. The mouse fix is libvirtualhid PR #145 (“target configured mouse viewport”, open), which Sunshine’s draft PR #5739 pulls in. Once it ships, spike S4 tries one Sunshine instance per display (own config file, port about 100 apart, output_name) and BetterDisplay virtual displays switched on by global_prep_cmd. Open PR #5817 (ScreenCaptureKit + OBS’s VideoToolbox encoder) may raise the encoder limit. Helping #145 along upstream is the one Mac contribution worth making.

What stock hosts cost us

Limit Effect What can be done
A stream’s fps is fixed at connect Out-of-sight panels save decoder work through keyframe sampling, but the host keeps encoding and sending 60 fps Patch ft-stream’s copy of moonlight-common-c to drop hidden video packets early (saves Frame CPU). A “change rate” control message needs a host upstream; Vibepollo is the likeliest
One stream per display Five ft-stream processes, connections, and client pairings on the Frame Measure the CPU (S3)
One real display per Vibepollo install A second real display needs a second host instance Use virtual displays; S3 tries a second instance on Windows
Cursor drawn into the video Lags the laser by a round trip; each mouse move over a still page becomes a video frame Accept
Mac: one display Sunshine’s mouse reaches only the main display, and there are no virtual displays Upstream fix in progress, then S4
Mac encoder Full-motion video on the super ultrawide at 60 fps is at the limit Lower fps or size for that stream

What Frametop maintains: ft-stream (pairing, decoder, keyframe sampling, input mapping, Remote Monitor launch and resume; the protocol itself is moonlight-common-c’s), the remote screen type in ft-screens, the layout and settings changes, and host setup notes or scripts.

Alternatives considered

Spikes before any Frametop change

Spikes on the Frame run through frame-job --local as a standalone test overlay, with the headset on or with frame-testbench holding its worn state and pose. None of them touches the live desktop. S3 and S4 also need the hosts set up.

# Question Pass
S1 (done) Does a decoded Q08C buffer show in a SteamVR overlay with correct colours? Test pattern clip, BT.709 limited range, then full range Correct colours; under 3% of a core; no extra missed frames (~/.cache/frametop-perf/drops); the decoder’s real output delay
S2 Concurrency: the original five, the worst case (2x 5120x1440, 2x 3440x1440, 3024x1964), fed from files in real time at 60 fps Admitted (92% declared); decode time per frame; SoC temperature and clocks over 10 minutes
S3 Live streams from the test PC (Vibepollo): the main session on a real desk monitor plus two Remote Monitors, through moonlight-common-c + the S1 decoder; then keyframe sampling on a hidden 60 fps stream CPU per stream at real bitrates; glass-to-glass latency; time to picture after LiRequestIdrFrame(); the host honours one IDR request a second; IDR size; Frame CPU for a hidden stream; each Remote Monitor’s mouse lands on its own display; Resume after a dropped stream keeps the windows; whether a second host instance can stream the other real monitor
S4 The Mac: one Sunshine instance on the main display. Once the mouse fix ships: one instance per display, then a BetterDisplay virtual display Mouse and keyboard work; encode rate with full-motion video on the super ultrawide; minimum_fps_target = 1 keeps a static display near 1 fps; later, all three stream at once with the mouse on the right display
S5 The baseline: 5 native screens, one playing video, four static The CPU, GPU time (DRM fdinfo), temperature, and missed-frame numbers that the remote version must match

S1 results (2026-10-06)

S1 passes, with one change to the plan: a GPU pass between the decoder and SteamVR.

stream/spike/ft-dectest (built by stream/build.sh) decodes a clip with the V4L2 stateful API and hands each picture to SteamVR with ImportDmabuf, as the remote screen type would. Clips: a colour test pattern (stream/spike/pattern.py) with a moving box, 10 s at 60 fps, encoded by NVENC HEVC on the test PC (-preset p1 -tune ull -bf 0, as Sunshine-style streaming), in BT.709 and BT.601, limited and full range. The in-headset runs used frame-testbench to hold the worn state and a still pose, with nobody wearing the headset.

Decoding.

  3440x1440, Q08C 3024x1964, Q08C 3440x1440, linear NV12
Decoder’s buffer (coded size) 3456x1440, 7,557,120 bytes 3072x1984, 9,191,424 bytes 3456x1440, 7,467,008 bytes
SteamVR import Works (NV12 + QCOM_COMPRESSED) Works Works (NV12, linear)
Decode time, steady 2.1 ms avg, 2.7 ms worst 2.6 ms avg, 3.3 ms worst 2.2 ms avg, 2.8 ms worst
First picture after 1 input frame 1 input frame 3 input frames
CPU at 60 fps, no conversion 1.2% of a core 1.5% of a core 1.4% of a core

Colours. stream/spike/s1-colours.py shows the clip head-locked, switches the panel between the decoded video and an RGB copy of the pattern every 2.5 s, grabs the headset view (/dev/video99) in both states, and compares each colour patch.

Clip, converted with its own matrix and range Mean error Worst patch
3440x1440, BT.709 limited 1.4 4.0
3440x1440, BT.709 full 1.2 4.0
3440x1440, BT.601 limited 0.2 2.4
3440x1440, BT.601 full 0.3 2.9
3024x1964, BT.709 limited 1.3 4.0

Errors are on the 0-255 scale, over 39-40 patches (27 for 3024x1964, where less of the panel is in view). The near-black steps (0-30) and near-white steps (225-255) all stay apart, so nothing is crushed or clipped. BT.709 sits about 3 low in red throughout, probably a rounding difference between ffmpeg’s and Mesa’s coefficients. That isn’t visible.

Cost of the pass, 3440x1440 at 60 fps:

Measure Result
GPU work About 118,000 cycles a frame: 0.13 ms at the 903 MHz top clock. With the headset idle, the GPU ran at about 230 MHz and was busy 0.51 ms a frame (DRM fdinfo)
Time until the GPU is done 1.1-1.4 ms avg, 2-6 ms worst
CPU, whole ft-dectest process 2.2-3.5% of a core, up from 1.2%
Missed compositor frames, 30 s at 90 Hz 0 of 2,701 with the stream shown, 0 of 2,700 idle

The benchmark is S5 against the same scene on remote displays: total Frame CPU (Frametop + ft-stream), GPU time, missed frames, and SoC temperature all at or below native.

S3 results so far (2026-10-07)

stream/spike/ft-streamtest.cpp is a Moonlight client for one display: moonlight-common-c and libgamestream from moonlight-embedded (pinned in stream/build.sh), S1’s decoder and GPU pass (spike/iris.h), and a SteamVR overlay. It pairs through the host’s API token, launches the primary app or a Remote Monitor, and reports every 2 s. Tests ran from the test PC over the LAN, with the headset held by frame-testbench.

The primary display (5120x1440 at 60 fps, HEVC, 50 Mbit/s asked). It works end to end. The picture in the headset is right, colours included. The test PC’s primary is an HDR monitor, and Vibepollo sends it as SDR Rec. 709, as asked.

  Shown Hidden (keyframe sampling)
Frames received 60 fps 60 fps
Frames decoded and shown 60 fps 1 fps
Network 12-14 Mbit/s the same
ft-streamtest CPU 4.2-4.6% of a core 1.5-1.7% of a core

Vibepollo bug: a stale display slot blocks Remote Monitors. The first Remote Monitor launch failed with 503, “The composed display topology did not apply”, for 40 s of retries. Vibepollo still held a display slot for the Mac’s earlier “Desktop” session (/api/clients/display-layout: the Mac as a client node, its runtime “retryable” with the lease held), though that session’s virtual display was gone. Composing the layout includes every held slot, and the Mac’s can’t be resolved to a device, so every composition fails. The Mac’s session had been cut by a Vibepollo restart and reconnected before it ended. The failed launch still created frametop-2’s virtual display, and “Disconnect Monitor” then reported success without removing it, because the launch never finished. A Vibepollo restart clears both. ft-stream must detect this state, a 503 that persists, and report it rather than retry forever. It’s worth reporting upstream with the steps that cause it.

Remote Monitor (frametop-2, 3440x1440 at 60 fps), after a Vibepollo restart. The launch took 2.3 s (Vibepollo makes the virtual display, then answers), and the first picture came 0.9 s after connecting. The picture was right. An empty, still monitor dropped to 1 fps at once: 0.1 Mbit/s, 0.6% of a core, about 4-5 ms from first packet to panel. Windows moved a window it remembered for that spot onto the new monitor, so a Remote Monitor can take windows off the real screens without being asked.

Vibepollo bug: a dropped Remote Monitor reset the host’s real monitor layout. The test killed ft-streamtest to imitate a dropped connection. Vibepollo kept the Remote Monitor for Resume, as configured, and recomposed the display layout. Windows refused it (SetDisplayConfig: ERROR_INVALID_PARAMETER, “failed to move device … to new origin” for the monitor above the primary). Vibepollo’s recovery (SDC_USE_DATABASE_CURRENT, a topology jog, CDS_RESET) then left Windows with a different layout: the monitor above became the primary, beside the old primary. “Disconnect Monitor” again reported success and did nothing. A Vibepollo restart was needed again.

One more defect that may have contributed: GET /api/clients/display-layout rebuilds Vibepollo’s record of the real monitors without their positions or modes (every monitor at 0,0, 1920x1080; refresh_remote_display_physical_baseline in confighttp.cpp). A layout composed from that record stacks the monitors on one another. Launching a Remote Monitor reads the real positions again (refresh_remote_monitor_baseline in nvhttp.cpp), and the last such call came before the launch here, so this isn’t proven to be the cause. Frametop must not call that endpoint until it’s fixed.

So far, Vibepollo 2.0.0’s Remote Monitors aren’t reliable on the test PC: a stale slot blocks them, a drop can rearrange the host’s real monitors, and “Disconnect Monitor” can’t release a monitor whose ownership was lost. These go upstream with logs before Frametop depends on them. Further Remote Monitor tests on the test PC need the user’s go-ahead, since they can move the user’s own screens.

The cause, and a fix (frametop-vibepollo, 2026-10-07). All three Remote Monitor failures come from one call. When a display role ends, Vibepollo’s coordinator (src/remote_display_topology.cpp) first recomposes the topology without the departing display, and only then removes it. On the test PC, Windows refuses that composed SetDisplayConfig. Then:

Removing the virtual display alone, as a Vibepollo restart does, left the layout intact. The fork Frametop/frametop-vibepollo (GPL-3.0, like Vibepollo), on branch fix/windows-remote-monitor-release, changes two things on Windows:

Four new unit tests cover it, and the coordinator’s 40 tests pass. The plan is to offer the fix upstream once it’s proven on the test PC. Frametop stays MIT: it only talks to the host over the network.

The fix works on the test PC (2026-10-07). The dev build, 2.0.0 plus the fix, was built on the test PC with MSYS2 (UCRT64, as the CI does, without WebRTC, drivers or packaging; D:\vp-build). It links only Windows system DLLs, so it drops into the install as a plain exe swap (D:\vp-build\deploy.ps1; the original is kept as sunshine.exe.2.0.0-original, and -Restore puts it back). Three Remote Monitor cycles left the real layout exactly as it was, with no layout re-apply and no SetDisplayConfig errors in Vibepollo’s log: a clean end, a killed client, and a relaunch after the kill. Before the fix, the clean end reset the layout twice in a row. Restarting Vibepollo for the deploy also ended a leftover Mac session cleanly and brought the real monitors back.

A dropped connection removes the Remote Monitor at once (Vibepollo saw the disconnect within 3 s), even with remote_monitor_disconnect_on_stream_end = disabled. With remote_monitor_disconnect_on_client_disconnect = enabled, a lost connection counts as a client disconnect. Keeping a monitor and its windows across a Wi-Fi drop would need that setting off, and Frametop releasing monitors explicitly with “Disconnect Monitor”. That’s still to test.

The development loop is an incremental Ninja build on the test PC (only changed files recompile), the exe swap through the admin SSH login, and ft-streamtest: a few minutes per change. Pure logic gets unit tests on the Frame.

Real displays, several at once (frametop-vibepollo, 2026-10-07). A new hidden control, “Frametop display” (id 2147483521), streams one of the host’s existing displays, named by the launch argument frametopDisplay (a device id from /api/display-devices). The session captures that output exactly, the way a Remote Monitor captures its virtual display, but nothing is created and the layout is never touched. Each client holds one such stream, and the main app stays free for the Deck and the Mac. On the test PC:

With that, Frametop no longer needs the “Frametop primary display” app; every real display is captured the same way.

Vibepollo bug: one idle HTTPS client froze the API for everyone. While any stream ran, every other client’s HTTPS requests (serverinfo, pairing, launch) waited until it ended, so a second display couldn’t start. The stock 2.0.0 build did the same. A stack dump (MSYS2 gdb attached to the service) showed the HTTPS server’s only I/O thread blocked in a synchronous TLS shutdown: SunshineHTTPS’s destructor (nvhttp.h) sends close_notify and then waits for the client’s. libgamestream leaves its connection idle in curl’s cache (it forbids reuse only on FreeBSD), so the wait lasted as long as the streaming client process. Fixed on both sides:

The 3D mouse, and dragging windows between displays (2026-10-07). The setup had three panels from the test PC: the OLED and the LC34G55T as Frametop displays, and a 2560x1440 Remote Monitor. frame-testbench held the headset, and the 3D mouse was driven through @ft_pointer_helper. The 3D mouse moved and clicked the test PC’s cursor on every panel.

The first version clamped every move to its own panel. A drag from the Remote Monitor to the OLED left the window below the Remote Monitor’s bottom edge, mostly off screen. Removing the Remote Monitor brought it back to the OLED. Drags between monitors that touch in Windows’ layout seemed to work, but only because the cursor, pinned at the shared edge, left the window hanging across it. --input-log showed why. With the button held, SteamVR kept sending the moves to the panel where the press began (for example -213,2093 on the 2560x1440 panel), and sent the panel under the laser no events at all. The release also went to the first panel, off its edge.

The spike’s fix is the routing described under “Input”, with one difference: each panel is its own process. The panel that got the press publishes the laser’s device in a small shared file (/dev/shm/frametop-streamtest-drag). Every other panel hit-tests that device’s ray against itself and moves the host’s cursor while the ray hits it. When hovering, the panel’s own hit test matched SteamVR’s coordinates exactly, so the ray origin and the UV orientation (bottom-left origin, like the mouse events) are right. With the fix, three drags worked:

In each, the window followed the laser across panels and stayed where it was dropped. The release still went through the first panel’s connection and landed right every time.

Windows’ “Remember window locations based on monitor connection” moves windows on its own. When a Remote Monitor appears, Windows puts back windows it remembers there, and when it goes, they move to a real monitor.

In ft-screens (2026-10-07)

The user decided remote displays are Frametop displays, with the same controls as the desktop’s screens and their places saved in profiles. The spike’s standalone overlays are replaced.

What’s built (branch remote-displays, uncommitted):

Tested through frame-testbench, with a --beside instance:

SteamVR’s overlay limit. SteamVR allows 128 overlays in the whole system (k_unMaxOverlayCount), its own included. Each Frametop panel takes six or seven with its controls. The desktop’s floating-window slots made all of theirs at start: eight empty slots held 56. With three screens, a third remote screen got VROverlayError_OverlayLimitExceeded. A floating window’s panel and controls are now made when a window floats on it, and destroyed when it docks (EnsureFloatPanel, DropFloatPanel). This needs a test on the live desktop. The controls of the other panels stay up, invisible until a laser comes near, because SteamVR’s laser hover is what brings them in.

Layouts, profiles and settings (2026-10-07, same branch):

vrcompositor crashed again at ft-screens quit (15:42, SIGBUS), and at 14:34. Both times remote screens were up and the headset was in standby. SteamVR logs “leaving standby” within 2 ms of ft-screens disconnecting, so the compositor wakes in the middle of our cleanup, and it drew a texture that was already freed. The crash took the gamescope session and Steam down with it. The restart at 15:39 under the same conditions didn’t crash, so it’s a race. Hardened, not yet verified: ft-screens now tears SteamVR down first, while every buffer the panels show still exists (before KWin and the streams go). It clears the panels’ textures, destroys the overlays, waits 200 ms, and only then releases the imports; nothing calls SteamVR after VR_Shutdown. Until that’s checked, restart the desktop only with the compositor awake.

Frozen panels, out-of-sight rate, sound (2026-10-07, after the user’s test):

Where the lost frames went, and the dongle (2026-10-07, ~16:40):

Adding a computer: sign in instead of carrying a token (2026-10-07, ~17:00):

The PC side: Frametop host setup (2026-10-07, ~17:40):

Frametop’s build released (2026-10-09):

Tested in the live desktop (frame-testbench, the 3D mouse through @ft_pointer_helper, 2026-10-07):

Still to do:

Open questions

This site is open source. Improve this page.