Improve soft-keyboard handling on mobile

[jasmussen]

There are issues with typing on mobile, both on Android and iOS. GIFs:

iOS

Problems:

1. Screen jumps on every linebreak as if resetting
2. You lose your place (this is an issue on every other textfield that auto-resizes, including the classic editor)

Android

Problems:

• Soft-keyboard sometimes hides itself and quickly opens again, causing a jarring effect

Causes

It is all but impossible to control when the soft-keyboard shows and hides on mobile devices. It seems to be controlled largely around the Javascript focus and blur events, whether the keyboard shows or hides.

In the case of Gutenberg, those two events fire when a new paragraph is created. We do this fast enough that sometimes the keyboard doesn't hide on Android, and it never hides on iOS. But we don't do it fast enough to prevent all side-effects:

• on iOS, the blur event for some reason causes the page to scroll to the top.
• on Android, the blur event sometimes causes the keyboard to hide and then quickly show again

The screen jump could potentially be mitigated by #353, but it seems like there could be a larger effort that could solve both. https://notion.so has a block editor that's similar to Gutenberg in many ways, and works paragraphs as individual contenteditable blocks. The problem described in this ticket is not an issue with the Notion editor, neither in Android or iOS:

Could it be that if we set focus on the new block fast enough, and employ a Typewriter-like scroll behavior like described in #353, is enough to address both issues? Or is there a better solution?

CC: @mtias @mcsf @aduth @youknowriad would appreciate your thoughts on this.

[jasmussen]

Could a solution be to create the new paragraph first, without moving focus, wait a tiny bit until the new paragraph is confirmed to exist, and then set focus on the new paragraph?

[jasmussen]

Tinkered a bit, and learned some new things

If you shift focus from one input field to another, the soft keyboard doesn't close or jump. It seems like the key is that the next input field has to exist before focus is set there, and that it's key the blur event is never fired.

I put up a simple codepen — try this one on your phone: https://codepen.io/joen/full/LQEEEz/

On both Android and iOS, the soft-keyboard stays there, with no weirdness.

In Gutenberg, if the Paragraph blocks/inputfields are already present, then there are no issues either. Or rather, it's still a little jumpy, but this is something we can address separately — the page doesn't reset its scroll position, at least, which is a start:

Both of these observations seem to suggest that the key issue here is that when you create a new paragraph, the focus is unset from the previous block, before it can be set on the new block.

If that is the case, then a fix could be to make sure the new paragraph block is instantiated before focus is set on it.

[hedgefield]

I've noticed this too, and it's the cause of most of what feels janky about the mobile experience. I hope your findings can fix this!

Another thing related to keyboard showing/hiding I noticed is that after you start typing (and the chrome disappears) it's hard to get it back.

[jasmussen]

Another thing related to keyboard showing/hiding I noticed is that after you start typing (and the chrome disappears) it's hard to get it back.

Hard to get the chrome back? Can you open this as a separate ticket?

[hedgefield]

Oh excuse me, I meant the toolbar and misremembered 🙃 That one is being taken care of (I think).

One more thing related to the above: could it also be that the block 'jumps' because the ellipsis options appear under a block when you focus it?

[ellatrix]

Mellemrum sounds like a good drink.

[jasmussen]

Mellemrum sounds like a good drink.

😂

It's danish for "space", literally "between space" :D :D :D

[ellatrix]

Been looking into this... I made some changes, but they don't really help on iOS as far as I see.

https://github.com/WordPress/gutenberg/compare/fix/ios-editable-keyboard

This branch will:

1. Move focus to editable earlier, before TinyMCE is initialised. All this now happens in the same call stack.
2. Only manually blur if focus has not been moved elsewhere in the next call stack.

But this all does not seem to help:

In this image you can see the order of events.

• prefocus and postfocus is logged right before and after calling element.focus(). Apparently moving focus manually will set the focus shortly on the document body anyway??

• The editor is blurred again before we attempt to set focus on init. I'm guessing this is done somewhere in the mce init process. It is blurred again when we adjust focus, but there it seems to stick in the correct element.

I'll be looking a bit more...

[jasmussen]

Thanks so much for working on this, I think it's important that we get this right sooner rather than later.

Move focus to editable earlier, before TinyMCE is initialised. All this now happens in the same call stack.

Can we set focus to the editable later? Can we avoid the blur entirely? I imagine that if the next editable exists before we set focus on it, that might improve things?

[ellatrix]

Can we set focus to the editable later? Can we avoid the blur entirely? I imagine that if the next editable exists before we set focus on it, that might improve things?

That's what I tried to do, but it doesn't seem to change anything. It seems to blur just by moving focus, even if the editable already exists. Maybe it's a TinyMCE initialisation issue.

[jasmussen]

Maybe it's a TinyMCE initialisation issue.

Is it possible to temporarily remove tinymce, or test this with blocks that don't utillize tinymce, just to narrow it down?

[ellatrix]

Our enter key behaviour is TinyMCE specific, so I'm not sure how to test without initialising it.

[ellatrix]

@jasmussen Does https://github.com/WordPress/gutenberg/compare/try/fix-ios-keyboard-hide fix this for you at all. Seems to fix it here on an old iPad. Only fixes the focus issue, not the scroll issue, but I guess that's one step in the right direction.

[ellatrix]

What I also notice is that it takes quite long for MCE to initialise... There's quite a delay on enter press.

[jasmussen]

@jasmussen Does https://github.com/WordPress/gutenberg/compare/try/fix-ios-keyboard-hide fix this for you at all. Seems to fix it here on an old iPad. Only fixes the focus issue, not the scroll issue, but I guess that's one step in the right direction.

I'm afraid it doesn't seem to have made any difference when I run in the Xcode simulator. That is to say, it may have fixed a focus issue, but I'm still seeing the big jumps.

[jasmussen]

Permit me to ask a dumb question. Here goes:

Can we intercept/change the behavior of the Enter button, so that it doesn't do anything until some time has passed, that amount of time being the time it takes for a new paragraph block below it to be created, and TinyMCE instantiated? Even if just as a prototype/hacky version, that would let us know if this was the way to solve it.

[ellatrix]

@jasmussen That's what #4775 does :)

[ellatrix]

The only issue left now is making sure the new block comes into view and have this strange scroll behaviour. I'm not sure what is causing the scroll jumps to the top at this time.

[jasmussen]

The only issue left now is making sure the new block comes into view and have this strange scroll behaviour. I'm not sure what is causing the scroll jumps to the top at this time.

I think this may be due to the side UI showing up underneath the content itself. You can, for debugging purposes, try to disable the mobile side UI with this:

.editor-block-list__block-mobile-toolbar {
display:none;
}

If you try that and it makes the experience much better, then I will rethink how we show the side UI on mobile.

[ellatrix]

No, that doesn't change anything. I've also tried scrolling the editable field into view when focus is set, but that doesn't help either.

[ellatrix]

While the branch fixes the focus/keyboard issue, I can't figure out exactly why the screen is jumping when moving focus. Chrome on iOS has the same issue.

[ellatrix]

Okay, #4775 has now a commit that is a terrible hack, but scrolls to the position of the block from which enter originates before TinyMCE initialises. This fixes the jumping as far as I see. This is also a good start to implement the typewriter experience. It seems smooth on desktop too.

[ellatrix]

Note: It is currently block based position adjustment, not line based. We might want to change that for the typewriting experience, but calculating exactly when a character flows to the next line is difficult.