feat(developer): Add more non-printing characters #9846

darcywong00 · 2023-10-25T08:08:30Z

Continues #9547

This adds special characters to Developer that were added in the new KeymanWeb font.

User Testing

Setup - Install the PR build of Keyman Developer

TEST_SPECIAL_CHARACTERS - verifies IDE can assign new characters

Launch Keyman Developer
Create a new keyboard that includes a touch layout keyboard
Go to the touch layout keyboard editor tab and select a key

In the Key properties, select the text dropdown and check for new strings
(will be in this list)

keyman/web/src/engine/osk/src/specialCharacters.ts

Lines 46 to 67 in a4e081a

    
           // Added in Keyman 17.0. 
        
           // Reference: https://github.com/silnrsi/font-symchar/blob/v4.000/documentation/encoding.md 
        
           '*ZWNJGeneric*':    0x79, // Generic version of ZWNJ (no override) 
        
           '*Sp*':             0x80, // Space 
        
           '*NBSp*':           0x82, // No-break Space 
        
           '*NarNBSp*':        0x83, // Narrow No-break Space 
        
           '*EnQ*':            0x84, // En Quad 
        
           '*EmQ*':            0x85, // Em Quad 
        
           '*EnSp*':           0x86, // En Space 
        
           '*EmSp*':           0x87, // Em Space 
        
           // TODO: Skipping #-per-em-space 
        
           '*PunctSp*':        0x8c, // Punctuation Space 
        
           '*ThSp*':           0x8d, // Thin Space 
        
           '*HSp*':            0x8e, // Hair Space 
        
           '*ZWSp*':           0x81, // Zero Width Space 
        
           '*ZWJ*':            0x77, // Zero Width Joiner 
        
           '*WJ*':             0x78, // Word Joiner 
        
           '*CGJ*':            0x7a, // Combining Grapheme Joiner 
        
           '*LTRM*':           0x90, // Left-to-right Mark 
        
           '*RTLM*':           0x91, // Right-to-left Mark 
        
           '*SH*':             0xa1, // Soft Hyphen 
        
           '*HTab*':           0xa2, // Horizontal Tabulation

keymanapp-test-bot · 2023-10-25T08:08:36Z

User Test Results

Test specification and instructions

✅ TEST_SPECIAL_CHARACTERS (PASSED): Tested with the attached PR build (Keyman Developer 17.0.202-alpha-test-9846) on Windows 11 OS (VM) and here is my observation: 1. Created a new keyboard and added Touch Layout feature in the Keyboard editor view. 2. Clicked the Touch Layout tab, then selected a key from the keyboard layout view. 3. In the Keycap properties, it seems that the new strings are available in the text dropdown list box. (notes)

Test Artifacts

Developer
Keyboards
- Test Keyboards

darcywong00 · 2023-10-26T00:05:04Z

developer/src/kmc-kmn/src/kmw-compiler/validate-layout-file.ts

@@ -199,6 +201,22 @@ function TransformSpecialKeys14(FDebug: boolean, sLayoutFile: string): string {
  return sLayoutFile;
 }

+function TransformSpecialKeys17(FDebug: boolean, sLayoutFile: string): string {


A bit WET, based on TransformSpecialKeys14 (ll 188-202)

developer/src/kmc-kmn/src/kmw-compiler/validate-layout-file.ts

bharanidharanj · 2023-10-26T06:47:12Z

Test Results

TEST_SPECIAL_CHARACTERS (PASSED): Tested with the attached PR build (Keyman Developer 17.0.198-alpha-test-9846) on Windows 11 OS (VM) and here is my observation: 1. Verified that the the newly added special characters are available in the Keycap Value drop-down list.

darcywong00 · 2023-10-26T06:52:24Z

Hm, from @bharanidharanj 's screenshot, the new characters show tofu. I thought Developer would be using the same keymanweb-osk.ttf font.

…hore/developer/non-printing-chars

mcdurdin

Changes look good, except we may still have the old font?

developer/src/kmc-kmn/src/kmw-compiler/constants.ts

developer/src/kmc-kmn/src/kmw-compiler/validate-layout-file.ts

Co-authored-by: Marc Durdin <marc@durdin.net>

darcywong00 · 2023-11-02T08:56:02Z

Changes look good, except we may still have the old font?

I copied keymanweb-osk.ttf to developer. (didn't realize it's a separate copy)

@keymanapp-test-bot retest

bharanidharanj · 2023-11-03T08:38:14Z

Test Results

TEST_SPECIAL_CHARACTERS (PASSED): Tested with the attached PR build (Keyman Developer 17.0.202-alpha-test-9846) on Windows 11 OS (VM) and here is my observation: 1. Created a new keyboard and added Touch Layout feature in the Keyboard editor view. 2. Clicked the Touch Layout tab, then selected a key from the keyboard layout view. 3. In the Keycap properties, it seems that the new strings are available in the text dropdown list box.

developer/src/test/auto/kmcomp/test_touchlayout_14_1.keyman-touch-layout

mcdurdin

Need to revert the three test_touchlayout_14 files and then I think it should be okay with the proposed changes

mcdurdin · 2023-11-06T06:33:00Z

common/web/types/src/kmx/kmx.ts

+  public static readonly VERSION_110 = 0x00000B00;
+  public static readonly VERSION_120 = 0x00000C00;
+  public static readonly VERSION_130 = 0x00000D00;


We shouldn't add these because we don't have files with these version numbers

developer/src/kmc-kmn/src/kmw-compiler/constants.ts

developer/src/kmc-kmn/src/kmw-compiler/validate-layout-file.ts

mcdurdin · 2023-11-06T06:39:22Z

developer/src/kmc-kmn/src/kmw-compiler/validate-layout-file.ts

+    for(let i = 0; i < CSpecialText17Map.length; i++) {
+      // Assumes the JSON output format will not change
+      if(FDebug) {
+        sLayoutFile = sLayoutFile.replaceAll('"text": "'+CSpecialText17Map[i][0]+'"', '"text": this._v>16 ? "'+CSpecialText17Map[i][0]+'" : "'+CSpecialText17Map[i][1]+'"');
+      } else {
+        sLayoutFile = sLayoutFile.replaceAll('"text":"'+CSpecialText17Map[i][0]+'"', '"text":this._v>16?"'+CSpecialText17Map[i][0]+'":"'+CSpecialText17Map[i][1]+'"');
+      }
+    }


We are using a multidimensional array here instead of a map, due to the conversion from Delphi. One day should clean this up

Co-authored-by: Marc Durdin <marc@durdin.net>

developer/src/kmc-kmn/src/kmw-compiler/validate-layout-file.ts

Co-authored-by: Marc Durdin <marc@durdin.net>

darcywong00 · 2023-11-07T08:05:00Z

@mcdurdin - I think all the test files got reverted. Is this ready to merge now?

mcdurdin · 2023-11-07T08:10:17Z

@mcdurdin - I think all the test files got reverted. Is this ready to merge now?

LGTM

keyman-server · 2023-11-07T18:02:36Z

Changes in this pull request will be available for download in Keyman version 17.0.207-alpha

darcywong00 added 2 commits October 25, 2023 14:26

chore(developer): Add non-printing characters

cd27628

chore(developer): Add handling for new characters

bf0a093

keymanapp-test-bot bot added has-user-test user-test-required User tests have not been completed labels Oct 25, 2023

keymanapp-test-bot bot added this to the A17S24 milestone Oct 25, 2023

github-actions bot added developer/ developer/compilers/ developer/ide/ feat labels Oct 25, 2023

fix(developer): Add check for CSpecialText17

7b00306

darcywong00 marked this pull request as ready for review October 25, 2023 23:51

darcywong00 requested a review from mcdurdin as a code owner October 25, 2023 23:51

darcywong00 commented Oct 26, 2023

View reviewed changes

developer/src/kmc-kmn/src/kmw-compiler/validate-layout-file.ts Outdated Show resolved Hide resolved

mcdurdin requested changes Oct 26, 2023

View reviewed changes

developer/src/kmc-kmn/src/kmw-compiler/validate-layout-file.ts Outdated Show resolved Hide resolved

keymanapp-test-bot bot removed the user-test-required User tests have not been completed label Oct 26, 2023

chore(developer): Merge remote-tracking branch 'origin/master' into c…

7bacbb2

…hore/developer/non-printing-chars

mcdurdin modified the milestones: A17S24, A17S25 Oct 27, 2023

darcywong00 added 3 commits October 30, 2023 09:25

chore(developer): Merge remote-tracking branch 'origin/master' into c…

1727278

…hore/developer/non-printing-chars

fix(developer): Revise check on special characters

4085612

chore(developer): Cleanup unused ZWNJ

2064ac4

mcdurdin approved these changes Nov 2, 2023

View reviewed changes

developer/src/kmc-kmn/src/kmw-compiler/constants.ts Outdated Show resolved Hide resolved

developer/src/kmc-kmn/src/kmw-compiler/validate-layout-file.ts Outdated Show resolved Hide resolved

darcywong00 and others added 2 commits November 2, 2023 13:22

Apply suggestions from code review

06cf276

Co-authored-by: Marc Durdin <marc@durdin.net>

fix(developer): Update keymanweb-osk.ttf font

ecae590

keymanapp-test-bot bot added the user-test-required User tests have not been completed label Nov 2, 2023

keymanapp-test-bot bot removed the user-test-required User tests have not been completed label Nov 3, 2023

mcdurdin reviewed Nov 6, 2023

View reviewed changes

developer/src/test/auto/kmcomp/test_touchlayout_14_1.keyman-touch-layout Outdated Show resolved Hide resolved

mcdurdin reviewed Nov 6, 2023

View reviewed changes

darcywong00 and others added 3 commits November 6, 2023 13:42

Apply suggestions from code review

56a86a8

Co-authored-by: Marc Durdin <marc@durdin.net>

fix(developer): Revert test changes

132c8c7

chore(common): revert VERSIONs

7ff8d7a

darcywong00 mentioned this pull request Nov 6, 2023

chore(developer): Cleanup multidimensional array to map #9945

Open

mcdurdin reviewed Nov 6, 2023

View reviewed changes

developer/src/kmc-kmn/src/kmw-compiler/validate-layout-file.ts Outdated Show resolved Hide resolved

chore(common): Revert other files

699e1f3

github-actions bot added common/ common/web/ and removed common/ common/web/ labels Nov 6, 2023

Apply suggestions from code review

5902c76

Co-authored-by: Marc Durdin <marc@durdin.net>

github-actions bot added common/ common/web/ and removed common/ common/web/ labels Nov 6, 2023

darcywong00 merged commit 71a9b9d into master Nov 7, 2023
4 checks passed

darcywong00 deleted the chore/developer/non-printing-chars branch November 7, 2023 08:20

This was referenced Nov 8, 2023

chore: Update "Key text" table for keymanweb-osk.ttf v4.0 keymanapp/help.keyman.com#867

Merged

chore(web,developer): Move keymanweb-osk.ttf to common/resources #9993

Merged

darcywong00 mentioned this pull request Dec 12, 2023

fix(web): Add BkSp1 special characer #10221

Closed

mcdurdin mentioned this pull request Jan 17, 2024

docs(developer): update with changes to v17 keymanapp/help.keyman.com#945

Closed

darcywong00 mentioned this pull request Mar 4, 2024

[khmer_angkor] Add non-printing characters to spacebar keymanapp/keyboards#2588

Merged

mcdurdin mentioned this pull request Apr 22, 2024

feat: osk should translate invisible controls to images #6031

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(developer): Add more non-printing characters #9846

feat(developer): Add more non-printing characters #9846

darcywong00 commented Oct 25, 2023 •

edited by mcdurdin

Loading

keymanapp-test-bot bot commented Oct 25, 2023 •

edited

Loading

darcywong00 Oct 26, 2023

bharanidharanj commented Oct 26, 2023

darcywong00 commented Oct 26, 2023

mcdurdin left a comment

darcywong00 commented Nov 2, 2023

bharanidharanj commented Nov 3, 2023

mcdurdin left a comment

mcdurdin Nov 6, 2023

mcdurdin Nov 6, 2023

darcywong00 commented Nov 7, 2023

mcdurdin commented Nov 7, 2023

keyman-server commented Nov 7, 2023

	// Added in Keyman 17.0.
	// Reference: https://github.com/silnrsi/font-symchar/blob/v4.000/documentation/encoding.md
	'ZWNJGeneric': 0x79, // Generic version of ZWNJ (no override)
	'Sp': 0x80, // Space
	'NBSp': 0x82, // No-break Space
	'NarNBSp': 0x83, // Narrow No-break Space
	'EnQ': 0x84, // En Quad
	'EmQ': 0x85, // Em Quad
	'EnSp': 0x86, // En Space
	'EmSp': 0x87, // Em Space
	// TODO: Skipping #-per-em-space
	'PunctSp': 0x8c, // Punctuation Space
	'ThSp': 0x8d, // Thin Space
	'HSp': 0x8e, // Hair Space
	'ZWSp': 0x81, // Zero Width Space
	'ZWJ': 0x77, // Zero Width Joiner
	'WJ': 0x78, // Word Joiner
	'CGJ': 0x7a, // Combining Grapheme Joiner
	'LTRM': 0x90, // Left-to-right Mark
	'RTLM': 0x91, // Right-to-left Mark
	'SH': 0xa1, // Soft Hyphen
	'HTab': 0xa2, // Horizontal Tabulation

feat(developer): Add more non-printing characters #9846

feat(developer): Add more non-printing characters #9846

Conversation

darcywong00 commented Oct 25, 2023 • edited by mcdurdin Loading

User Testing

keymanapp-test-bot bot commented Oct 25, 2023 • edited Loading

User Test Results

Test Artifacts

darcywong00 Oct 26, 2023

Choose a reason for hiding this comment

bharanidharanj commented Oct 26, 2023

Test Results

darcywong00 commented Oct 26, 2023

mcdurdin left a comment

Choose a reason for hiding this comment

darcywong00 commented Nov 2, 2023

bharanidharanj commented Nov 3, 2023

Test Results

mcdurdin left a comment

Choose a reason for hiding this comment

mcdurdin Nov 6, 2023

Choose a reason for hiding this comment

mcdurdin Nov 6, 2023

Choose a reason for hiding this comment

darcywong00 commented Nov 7, 2023

mcdurdin commented Nov 7, 2023

keyman-server commented Nov 7, 2023

darcywong00 commented Oct 25, 2023 •

edited by mcdurdin

Loading

keymanapp-test-bot bot commented Oct 25, 2023 •

edited

Loading