Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(core): infrastructure for normalization of output 🌱 #10403

Merged
merged 14 commits into from
Jan 19, 2024

Commits on Jan 16, 2024

  1. Configuration menu
    Copy the full SHA
    7b44175 View commit details
    Browse the repository at this point in the history
  2. refactor(core): split context API from Core primary API

    Relates to #9999.
    Fixes #10384.
    
    The context API endpoints should no longer be considered as part of the
    standard Core API. The only consumers that have a need to access these
    APIs are the IMX integration in Engine for Windows, and the Keyman
    Developer Debugger.
    
    These symbols are currently used by Developer:
    * `km_core_context` struct
    * `km_core_context_type` enum
    * `km_core_context_item` struct
    * `KM_CORE_CONTEXT_ITEM_END` macro
    * `km_core_state_context()`
    * `km_core_context_set()`
    * `km_core_context_clear()`
    
    These symbols are currently used by Windows IMX:
    * `km_core_context` struct
    * `km_core_context_type` enum
    * `km_core_context_item` struct
    * `KM_CORE_CONTEXT_ITEM_END` macro
    * `km_core_context_items_dispose()`
    * `km_core_context_item_list_size()`
    * `km_core_state_get_intermediate_context()`
    
    The following functions and symbols are moving to
    keyman_core_api_context.h:
    * `km_core_context` struct
    * `km_core_context_type` enum
    * `km_core_context_item` struct
    * `KM_CORE_CONTEXT_ITEM_END` macro
    * `km_core_state_context()` function
    * `km_core_state_get_intermediate_context()` function
    * `km_core_context_set()` function
    * `km_core_context_clear()` function
    * `km_core_context_get()` function
    * `km_core_context_items_from_utf16()` function
    * `km_core_context_items_from_utf8()` function
    * `km_core_context_items_to_utf8()` function
    * `km_core_context_items_to_utf16()` function
    * `km_core_context_items_to_utf32()` function
    * `km_core_context_items_dispose()` function
    * `km_core_context_length()` function
    * `km_core_context_append()` function
    * `km_core_context_shrink()` function
    * `km_core_context_item_list_size()` function
    mcdurdin committed Jan 16, 2024
    Configuration menu
    Copy the full SHA
    e5c2525 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    ba1f592 View commit details
    Browse the repository at this point in the history
  4. feat(core): infrastructure for normalization of output

    Relates to #9999.
    
    Establishes functions, unit test sources, normalization entry point and
    an effectively no-op unit test for normalization support.
    mcdurdin committed Jan 16, 2024
    Configuration menu
    Copy the full SHA
    e39affe View commit details
    Browse the repository at this point in the history

Commits on Jan 17, 2024

  1. feat(core): action output normalization

    Fixes #9999.
    
    Note TODO items:
    - [ ] Renormalize cached_context across action boundary. Blocked by #10369.
    - [ ] Add extra tests for surrogate pairs
    - [ ] Move set_context_from_string into helper module
    - [ ] if we don't apply normalization, we still need to fixup the
          app_context, to keep it coherent with cached_context (or at least
          we need to verify that app_context is never used in this
          situation)
    mcdurdin committed Jan 17, 2024
    Configuration menu
    Copy the full SHA
    e6096a2 View commit details
    Browse the repository at this point in the history
  2. Merge branch 'chore/mac/9999-remove-legacy-action-items-references' i…

    …nto feat/core/9999-normalize-output
    mcdurdin authored Jan 17, 2024
    Configuration menu
    Copy the full SHA
    c2f2816 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    40f0839 View commit details
    Browse the repository at this point in the history
  4. chore(core): test typos

    mcdurdin committed Jan 17, 2024
    Configuration menu
    Copy the full SHA
    7218f93 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    39f0813 View commit details
    Browse the repository at this point in the history

Commits on Jan 18, 2024

  1. chore(core): simplify input assumptions for actions_normalize

    Per discussion in #10422, we can assume that input cached_context is
    always NFD. However input actions->output may not start at a
    normalization boundary, so we still need to backtrack to a normalization
    boundary in order to get our NFC output. But cached_context never need
    change.
    
    This makes no change to the algorithm, but tweaks some of the unit tests
    to adhere to this input assumption.
    
    Note: we could consider adding a debug assertion that cached_context is
    NFD.
    mcdurdin committed Jan 18, 2024
    Configuration menu
    Copy the full SHA
    823120b View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    4ab47ea View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    425ebe3 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    3878394 View commit details
    Browse the repository at this point in the history

Commits on Jan 19, 2024

  1. Configuration menu
    Copy the full SHA
    ba46d21 View commit details
    Browse the repository at this point in the history