Ruffle: Keyboard inputs unresponsive

[slamotas]

Ruffle version: nightly-2021-10-26
libTAS version: 1.4.2 interim c83a69f

Tested with this SWF: https://dagobah.net/flash/the_worlds_hardest_game.swf

Command-line options are -g gl /path/to/dagobah_the_worlds_hardest_game.swf. Time tracking with clock_gettime() is required.

Get into the first level and try to move around with the arrow keys (you are the red square). You usually don't move around, but sometimes the player will start moving in a direction and continue to move without any inputs. The game works properly outside of libTAS.

[MyNameIsTrez]

I was about to make the exact same issue!

[clementgallet]

Keyboard inputs work fine for me :/ I guess mouse inputs work fine if you can get ingame?
Here's an example of me pressing down with Keyboard log enabled, do you have the same?

[libTAS f:493] Thread 5323 (main) Generate XEvent KeyPress with keycode 116
[libTAS f:493] Thread 5323 (main) Generate XIEvent RawKeyPress with keycode 116
[libTAS f:493] Thread 5323 (main) XKeycodeToKeysym called with keycode 116
[libTAS f:493] Thread 5323 (main)    returning 65364
[libTAS f:493] Thread 5323 (main) XLookupString called with keycode 116
[libTAS f:493] Thread 5323 (main) XLookupString called with keycode 116
[libTAS f:493] Thread 5323 (main) Xutf8LookupString called with keycode 116

[slamotas]

Here is me pressing and then releasing it. Some of the lines are missing?

[libTAS f:5643] Thread 18268 (main) Generate XEvent KeyPress with keycode 116
[libTAS f:5643] Thread 18268 (main) Generate XIEvent RawKeyPress with keycode 116
[libTAS f:5643] Thread 18268 (main) XKeycodeToKeysym called with keycode 116
[libTAS f:5643] Thread 18268 (main) returning 65364
[libTAS f:5661] Thread 18268 (main) Generate XEvent KeyRelease with keycode 116
[libTAS f:5661] Thread 18268 (main) Generate XIEvent RawKeyRelease with keycode 116
[libTAS f:5661] Thread 18268 (main) XKeycodeToKeysym called with keycode 116
[libTAS f:5661] Thread 18268 (main) returning 65364
[libTAS f:5661] Thread 18268 (main) XLookupString called with keycode 116

[MyNameIsTrez]

I'm getting exactly the same console output as slamotas is getting:

[libTAS f:1823] Thread 39782 (main) Generate XEvent KeyPress with keycode 116
[libTAS f:1823] Thread 39782 (main) Generate XIEvent RawKeyPress with keycode 116
[libTAS f:1823] Thread 39782 (main) XKeycodeToKeysym called with keycode 116
[libTAS f:1823] Thread 39782 (main)    returning 65364
[libTAS f:1958] Thread 39782 (main) Generate XEvent KeyRelease with keycode 116
[libTAS f:1958] Thread 39782 (main) Generate XIEvent RawKeyRelease with keycode 116
[libTAS f:1958] Thread 39782 (main) XKeycodeToKeysym called with keycode 116
[libTAS f:1958] Thread 39782 (main)    returning 65364
[libTAS f:1958] Thread 39782 (main) XLookupString called with keycode 116

[MyNameIsTrez]

The console does seem to print every time the down arrow key is being pressed, even though the game doesn't register the key press. I'm thinking the dropped inputs may only be happening when the game is laggy, which mine definitely is. The top of the window reports it's 30 FPS, but I think that's wrong as it has a lot of small lag spikes.

[MyNameIsTrez]

In this video you can see me mashing the up/down/left/right arrow keys like a madman and how the game rarely registers any of those key presses:
https://user-images.githubusercontent.com/32989873/139593632-e21cd69b-cc2a-4fe9-a9cc-ad253d3a9210.mp4

[clementgallet]

There's one thing I'm concerned in the video: there's no updated mouse coordinates in the OSD. Can you confirm that mouse inputs always show 0,0 ?

[MyNameIsTrez]

@clementgallet I'll try to reproduce it with the latest version of libTAS + Ruffle + OpenGL once I get home.

[MyNameIsTrez]

@clementgallet This video showcases two things on the latest version of everything:

1. Moving the mouse freezes the game, and the faster the mouse moves the more it freezes.
2. The WASD controls still don't work, cause at the end of the video I'm furiously pressing the A and S keys to no avail.

foo.mp4

[jakobhellermann]

I looked into this (because I was trying to TAS another rust-based game engine) and found the reason for the unresponsive input events.

ruffle uses winit for window and input event handling. Commenting out this code
https://github.com/rust-windowing/winit/blob/0e52672f4aea376c3ef8a06b64aedca46a40e9b8/src/platform_impl/linux/x11/event_processor.rs#L120-L133
makes the inputs work as expected.

The code is basically

// XFilterEvent tells us when an event has been discarded by the input method.
// Specifically, this involves all of the KeyPress events in compose/pre-edit sequences,
// along with an extra copy of the KeyRelease events. This also prevents backspace and
// arrow keys from being detected twice.
if(XFilterEvent(event, window)) {
  // drop event
}

where XFilterEvent comes from the xlib library.

Why XFilterEvent returns true for some of the events (I think it returns true for the release events if an even number of keys is pressed and for the pressed events if an odd number of keys is pressed or something like that) I don't know.

[jakobhellermann]

I can confirm that c15b9c8 fixes the issue for me

[MyNameIsTrez]

I can confirm that the game in question works for me now, thank you @clementgallet! 👍