fix(client): the six Phase 0 touch defects #447

Merged

enricobuehler merged 7 commits from touch/phase-0 into main 2026-08-30 20:38:48 +00:00

Conversation 0 Commits 7 Files Changed 28

+854 -255

enricobuehler commented 2026-08-29 21:29:12 +00:00

Owner

Copy Link

Copy Source

Phase 0 of punktfunk-planning/design/touch-client-overlay.md (§5, §6): the touch defects that are independent of the overlay and should ship first. One commit per work package, plus a merge of main (the Triton PR took C ABI 27 in the meantime, so the touch capability landed as ABI 28).

What changes for a user

WP	Before	After
T3 ea04667f	A three-finger drag scrolled on a Deck (the twins reserve it)	Exactly two fingers scroll; three or more only disqualify the tap
T2 faa8c92c	"Invert scroll" did nothing for two-finger touch scrolling on an iPad or a Deck	The sign is applied where the notch is made, as Android already did
T1 88213b1b	Press-and-hold gave a click and a cursor move — window drag, text selection and file drag were unreachable on every touch client	One finger held still 500 ms presses the left button and drags until it lifts; tap-then-drag stays
T5 3e9b6d12	On a phone in portrait most of the screen (the letterbox bars) was inert	The gesture layer spans the whole container; absolute contacts clamp into the picture
T6 b96d7274	Android could not change the session mode at all	nativeRequestMode JNI
T4 5f5033bc	Selecting Touch mode against a Hyprland/sway/Omarchy host did nothing and said nothing	HOST_CAP2_TOUCH: the host says whether its injector carries touch; each client runs the trackpad model and shows a one-line notice when it does not

Things the build found that the design does not say

• pf-presenter gates punktfunk-core per target in Cargo.toml, so lifting the touch module's platform gate alone did not compile on macOS. Act::wire() moved to input.rs; the engine now depends on nothing but std. Before this, cargo test -p pf-presenter on macOS ran zero tests and passed.
• host_caps2 never reached any client: no getter, no ABI function, no Swift or Kotlin accessor. punktfunk_connection_host_caps2 and PUNKTFUNK_HOST_CAP2_TOUCH are C ABI 28 (additive; 27 went to the SC2 hidout widening on main). The wire is unchanged — the byte has ridden the Welcome since the repeat mark.
• A long press needs a clock: a still finger raises no event. Desktop ticks from the run loop with SDL ticks; Android uses withTimeoutOrNull; Apple a main-queue work item.
• The settings screens cannot grey the Touch row (the host's bits are known only in a session), so T4's user-visible half is in-stream.

Verification

• rustfmt; clippy -D warnings for pf-presenter, pf-client-core, punktfunk-client-session, punktfunk-core, pf-inject (pf-gtkflow container) and punktfunk-host (punktfunk-rust-ci container); rerun green after the merge of main.
• 13 desktop gesture tests (five new), run on macOS for the first time.
• Android: :kit + :app compile (the JNI crate through cargo-ndk, the API-28 import floor check), all unit suites, VideoFitRectTest.
• Swift: PunktfunkKit and PunktfunkClient on iOS, tvOS and macOS.
• Not on glass. The four Phase 0 done-whens in the design's §9 are still open; T1's Android and Apple halves have no other check.

Docs: docs-site/content/docs/input.md gains the fallback paragraph. CHANGELOG is written at release time.

Phase 0 of `punktfunk-planning/design/touch-client-overlay.md` (§5, §6): the touch defects that are independent of the overlay and should ship first. One commit per work package, plus a merge of `main` (the Triton PR took C ABI 27 in the meantime, so the touch capability landed as **ABI 28**). ## What changes for a user | WP | Before | After | |---|---|---| | T3 `ea04667f` | A three-finger drag scrolled on a Deck (the twins reserve it) | Exactly two fingers scroll; three or more only disqualify the tap | | T2 `faa8c92c` | "Invert scroll" did nothing for two-finger touch scrolling on an iPad or a Deck | The sign is applied where the notch is made, as Android already did | | T1 `88213b1b` | Press-and-hold gave a click and a cursor move — window drag, text selection and file drag were unreachable on every touch client | One finger held still 500 ms presses the left button and drags until it lifts; tap-then-drag stays | | T5 `3e9b6d12` | On a phone in portrait most of the screen (the letterbox bars) was inert | The gesture layer spans the whole container; absolute contacts clamp into the picture | | T6 `b96d7274` | Android could not change the session mode at all | `nativeRequestMode` JNI | | T4 `5f5033bc` | Selecting Touch mode against a Hyprland/sway/Omarchy host did nothing and said nothing | `HOST_CAP2_TOUCH`: the host says whether its injector carries touch; each client runs the trackpad model and shows a one-line notice when it does not | ## Things the build found that the design does not say - `pf-presenter` gates `punktfunk-core` per target in `Cargo.toml`, so lifting the `touch` module's platform gate alone did not compile on macOS. `Act::wire()` moved to `input.rs`; the engine now depends on nothing but `std`. Before this, `cargo test -p pf-presenter` on macOS ran **zero** tests and passed. - `host_caps2` never reached any client: no getter, no ABI function, no Swift or Kotlin accessor. `punktfunk_connection_host_caps2` and `PUNKTFUNK_HOST_CAP2_TOUCH` are **C ABI 28** (additive; 27 went to the SC2 hidout widening on `main`). The wire is unchanged — the byte has ridden the Welcome since the repeat mark. - A long press needs a clock: a still finger raises no event. Desktop ticks from the run loop with SDL ticks; Android uses `withTimeoutOrNull`; Apple a main-queue work item. - The settings screens cannot grey the Touch row (the host's bits are known only in a session), so T4's user-visible half is in-stream. ## Verification - rustfmt; clippy `-D warnings` for `pf-presenter`, `pf-client-core`, `punktfunk-client-session`, `punktfunk-core`, `pf-inject` (pf-gtkflow container) and `punktfunk-host` (punktfunk-rust-ci container); rerun green after the merge of `main`. - 13 desktop gesture tests (five new), run on macOS for the first time. - Android: `:kit` + `:app` compile (the JNI crate through cargo-ndk, the API-28 import floor check), all unit suites, `VideoFitRectTest`. - Swift: PunktfunkKit and PunktfunkClient on iOS, tvOS and macOS. - **Not on glass.** The four Phase 0 done-whens in the design's §9 are still open; T1's Android and Apple halves have no other check. Docs: `docs-site/content/docs/input.md` gains the fallback paragraph. CHANGELOG is written at release time.

enricobuehler added 6 commits 2026-08-29 21:29:16 +00:00

fix(presenter): scroll on exactly two fingers, never three ... ea04667f40

A three-finger drag scrolled on a Deck and summoned the keyboard on an
iPad. The published vocabulary is two-finger scroll; the desktop engine
scrolled on two OR MORE fingers.

Three or more fingers now anchor their own centroid, disqualify the tap
when they travel, and never scroll or move the cursor — the rule the
Android and Apple engines already apply.

Act::wire() moves to input.rs, the only place that needs InputKind, so
touch.rs has no platform-gated dependency and builds everywhere. Before
this, `cargo test -p pf-presenter` ran zero tests on macOS and passed.

fix(client): honour invert scroll on two-finger touch scrolling ... faa8c92cd0

Turning on "Invert scroll" changed nothing for two-finger scrolling on
an iPad or a Deck. The desktop applied the sign only in the wheel path
(Capture::on_wheel); the touch engine's notches went out raw. On Apple,
InputCapture.sendScroll applies it and its comment called itself the
one scroll sink, but TouchMouse sends straight to the connection.

Both engines now apply the sign where the notch is made, as Android
already did. The Swift comment names the second sink.

feat(client): long-press to drag on every touch engine ... 88213b1bf2

Press-and-hold — the touch idiom for "pick this up" — produced a click
and then a cursor move on all three clients, in trackpad and pointer
mode alike. The engines armed a held left button only from a tap
followed by a second touch within 250 ms. Dragging a window, selecting
text and dragging a file were unreachable from a touchscreen.

One finger held still for 500 ms now presses the left button and drags
until it lifts; the tap-then-drag path stays. A still finger raises no
event, so each engine gets its clock: a run-loop tick on the desktop,
withTimeoutOrNull on Android, a main-queue work item on Apple. The
Android gesture releases a held button on teardown, which it never did.

fix(android): touch on the letterbox bars reaches the gesture engine ... 3e9b6d123e

On a phone held against a 16:9 stream, a trackpad swipe that started on
a black bar did nothing: the gesture layer sat on the aspect-fitted
picture box, so most of the screen was inert and read as unresponsive.

The gesture layer now spans the whole container. Every absolute
mapping — direct pointer, passthrough, the pen lane — measures against
the picture rect (`videoFitRect`, the same centre-aligned fit the
surface is laid out with) and clamps into it, so a contact on a bar
lands on the nearest picture edge. Trackpad deltas need no rect.

feat(android): nativeRequestMode switches the live session's mode ... b96d727447

The core and the C ABI can renegotiate a session's resolution and
refresh without reconnecting, and the Apple and desktop clients use it.
Android had no binding, so the in-stream Resolution row the touch
overlay design adds could not work there.

One JNI on the nativeDisconnectQuit template, one external fun.

feat(core): HOST_CAP2_TOUCH says whether a host injects touch ... 5f5033bc28

Selecting the Touch (passthrough) model against a Hyprland, sway or
Omarchy host did nothing: the wlroots injector drops every contact, and
no capability bit let a client know. Windows below build 1809 fails the
same way. The client offered the mode, the host dropped the events, and
no layer said so.

The host now sets HOST_CAP2_TOUCH in the Welcome's second capability
byte when its live injector carries touch (libei, gamescope EIS, KWin;
the PT_TOUCH probe on Windows). The byte never reached a client before:
NativeClient::host_caps2, punktfunk_connection_host_caps2 and
PUNKTFUNK_HOST_CAP2_TOUCH expose it (C ABI 27, additive). Each client
runs the trackpad model for the session when the bit is absent and shows
a one-line notice at stream start.

enricobuehler added 1 commit 2026-08-29 21:35:44 +00:00

Merge origin/main into touch/phase-0 ... ea92f7148d

main took C ABI 27 for the Steam Controller 2 hidout widening, so
punktfunk_connection_host_caps2 and PUNKTFUNK_HOST_CAP2_TOUCH move to
ABI 28; the header is regenerated. The SC2 passthrough badge and the
touch-fallback badge now stack in the same bottom-centre slot.

enricobuehler marked the pull request as ready for review 2026-08-30 20:30:42 +00:00

enricobuehler merged commit fd050bbc67 into main 2026-08-30 20:38:48 +00:00

enricobuehler deleted branch touch/phase-0 2026-08-30 20:38:55 +00:00

enricobuehler referenced this pull request 2026-08-30 20:45:44 +00:00

fix(ci): the ABI pin, android dead code, windows gamescope compile #455

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

No items

No labels

Milestone

No items

No Milestone

Projects

Clear projects

No projects

Assignees

Clear assignees

FernandoMX FlipperDan HitFrostbite bluelightspecial enricobuehler (Enrico Bühler) kszlim luxus nannku neoney sonar

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: unom/punktfunk#447