feat(core): infrastructure for normalization of output 🌱 #10403
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Relates to #9999. Fixes #10384. The context API endpoints should no longer be considered as part of the standard Core API. The only consumers that have a need to access these APIs are the IMX integration in Engine for Windows, and the Keyman Developer Debugger. These symbols are currently used by Developer: * `km_core_context` struct * `km_core_context_type` enum * `km_core_context_item` struct * `KM_CORE_CONTEXT_ITEM_END` macro * `km_core_state_context()` * `km_core_context_set()` * `km_core_context_clear()` These symbols are currently used by Windows IMX: * `km_core_context` struct * `km_core_context_type` enum * `km_core_context_item` struct * `KM_CORE_CONTEXT_ITEM_END` macro * `km_core_context_items_dispose()` * `km_core_context_item_list_size()` * `km_core_state_get_intermediate_context()` The following functions and symbols are moving to keyman_core_api_context.h: * `km_core_context` struct * `km_core_context_type` enum * `km_core_context_item` struct * `KM_CORE_CONTEXT_ITEM_END` macro * `km_core_state_context()` function * `km_core_state_get_intermediate_context()` function * `km_core_context_set()` function * `km_core_context_clear()` function * `km_core_context_get()` function * `km_core_context_items_from_utf16()` function * `km_core_context_items_from_utf8()` function * `km_core_context_items_to_utf8()` function * `km_core_context_items_to_utf16()` function * `km_core_context_items_to_utf32()` function * `km_core_context_items_dispose()` function * `km_core_context_length()` function * `km_core_context_append()` function * `km_core_context_shrink()` function * `km_core_context_item_list_size()` function
Relates to #9999. Establishes functions, unit test sources, normalization entry point and an effectively no-op unit test for normalization support.
keymanapp-test-bot
bot
added
the
user-test-missing
User Test ResultsTest specification and instructions User tests are not required |
keymanapp-test-bot
bot
changed the title
Fixes #9999. Note TODO items: - [ ] Renormalize cached_context across action boundary. Blocked by #10369. - [ ] Add extra tests for surrogate pairs - [ ] Move set_context_from_string into helper module - [ ] if we don't apply normalization, we still need to fixup the app_context, to keep it coherent with cached_context (or at least we need to verify that app_context is never used in this situation)
mcdurdin
force-pushed
the
chore/mac/9999-remove-legacy-action-items-references
branch
from
ba1f592
to
cf2d6c4
Compare
…nto feat/core/9999-normalize-output
keymanapp-test-bot
bot
removed
the
user-test-missing
This was referenced Jan 17, 2024
Closed
|
Returning to draft to rework part of this per #10422 (comment) |
Per discussion in #10422, we can assume that input cached_context is always NFD. However input actions->output may not start at a normalization boundary, so we still need to backtrack to a normalization boundary in order to get our NFC output. But cached_context never need change. This makes no change to the algorithm, but tweaks some of the unit tests to adhere to this input assumption. Note: we could consider adding a debug assertion that cached_context is NFD.
mcdurdin
commented
Jan 18, 2024
This was
linked to
issues
Jan 18, 2024
Base automatically changed from
chore/mac/9999-remove-legacy-action-items-references
to
epic/core/9999-normalization
January 18, 2024 03:46
ermshiperete
approved these changes
Jan 18, 2024
srl295
approved these changes
Jan 18, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #9999.
Fixes #10421.
Fixes #10422.
Note TODO items for follow-up PRs:
@keymanapp-test-bot skip