Transliterator runtime #3946
Transliterator runtime #3946
Conversation
commit b62edfe Author: Niels Saurer <nielssaurer@google.com> Date: Sat Aug 26 02:30:56 2023 +0200 regenerate test+bakeddata commit ba7a333 Author: Niels Saurer <nielssaurer@google.com> Date: Sat Aug 26 02:23:33 2023 +0200 add idx to segments commit bddf819 Author: Robert Bastian <robertbastian@users.noreply.github.com> Date: Fri Aug 25 21:45:28 2023 +0200 pen Cleaning up the workspace (unicode-org#3938) commit beae962 Author: Shane F. Carr <shane@unicode.org> Date: Fri Aug 25 12:22:25 2023 -0700 Add some scripts and regions to HelloWorldProvider (unicode-org#3936)
| /// The used indices in method parameters are all compatible with each other. | ||
| // Thought: I don't think this needs to be called *Utf8* matcher, maybe just Matcheable | ||
|
|
||
| pub(super) trait Utf8Matcher<D: MatchDirection>: Debug { |
There was a problem hiding this comment.
Question:
Looking for advice for a different API than implementing two traits (Utf8M<Forward> and <Reverse>) with the same method API surface for the same type. Leads to conflicts that need to be fully qualified such as here (in a follow-up PR): https://github.com/unicode-org/icu4x/pull/3965/files#diff-6dd89dfb5721915aafe24a449bc4d2d499df79d63e2f7687e2eccd9384307cddR44
Maybe just explicit method names? match_str_forward, match_str_reverse, etc?
experimental/transliteration/data/data/macros/transliterator_rules_v1.data.rs
Outdated
Show resolved
Hide resolved
| if idx < next_base { | ||
| return Some(VarTableElement::Quantifier( | ||
| QuantifierKind::ZeroOrOne, | ||
| &self.quantifiers_opt[idx], |
There was a problem hiding this comment.
clippy will want to know that these indexing operations are non-panicking
|
@Manishearth if you want to, I'd appreciate your comments on the use of |
|
I have a rough understanding of what's going on in |
Depends on #3943
Part of #3736
Implements the runtime for transliterators:
Has tests:
Has a setup for benchmarks. This is very much just the structure, the benchmarks are not well-thought out yet. #3957
I will turn the commented-out
eprintln's intolog::info!'s in a follow-up: #3958Note to reviewers: The commit history contains a lot of old designs, starting from the main module being unsafe + allocate-y to now the main module being safe + less allocate-y. Probably best to review the files individually instead of going through the commits.