Convert between predicate options and collator expressions #12329

1ec5 · 2018-07-05T22:14:15Z

The [c] and [d] options are now supported within comparison predicates for case and diacritic insensitivity, respectively. Along the way, the mbgl code that detects expressions in filters had to be updated to allow a collator as the third argument to a comparison expression.

Fixes #12269. Working towards #11786.

/cc @ChrisLoer @fabian-guerra @anandthakker

1ec5 · 2018-07-05T22:15:23Z

src/mbgl/style/conversion/filter.cpp

@@ -55,7 +55,8 @@ bool isExpression(const Convertible& filter) {
        return false;

    } else if (*op == "==" || *op == "!=" || *op == ">" || *op == ">=" || *op == "<" || *op == "<=") {
-        return arrayLength(filter) == 3 && (isArray(arrayMember(filter, 1)) || isArray(arrayMember(filter, 2)));
+        auto length = arrayLength(filter);
+        return (length == 3 || length == 4) && (isArray(arrayMember(filter, 1)) || isArray(arrayMember(filter, 2)));


Apparently this code was ported from GL JS code that will also need to be updated.

Filed as mapbox/mapbox-gl-js#6920

Maybe slightly better would be:

length != 3 || (isArray(arrayMember(filter, 1)) || isArray(arrayMember(filter, 2)))

The reasoning being that the default should be "this is an expression", and we should scope the fallback to legacy filters as tightly as possible?

Makes sense. Done.

The reasoning being that the default should be "this is an expression", and we should scope the fallback to legacy filters as tightly as possible?

Yeah, pretty much: sending as many inputs as possible through the expression parsing pipeline, since that will be more and more likely what users will be trying to write, and so expression-related validation messages are more likely to be useful

ChrisLoer · 2018-07-05T23:15:54Z

platform/darwin/src/NSComparisonPredicate+MGLAdditions.mm

+            @"case-sensitive": @(!(self.options & NSCaseInsensitivePredicateOption)),
+            @"diacritic-sensitive": @(!(self.options & NSDiacriticInsensitivePredicateOption)),
+        };
+        return [comparisonArray arrayByAddingObject:@[@"collator", collatorObject]];


Does this means 'Québec' =[] 'Quebec' will use a collator (and thus the current locale) vs. 'Québec' = 'Quebec' doing non-locale-aware comparison?

As long as an option is specified in brackets, it’ll use a collator. To match Core Data behavior, I suppose we could default the locale to en-posix when c or d is specified unless l is also specified.

A literal =[] is invalid syntax; there has to be an option. NSPredicate will raise an exception immediately on initialization.

I think the current behavior sounds right. 👍

ChrisLoer · 2018-07-05T23:17:04Z

platform/ios/CHANGELOG.md

@@ -12,6 +12,7 @@ Mapbox welcomes participation and contributions from everyone. Please read [CONT

 * Added an `MGLRasterStyleLayer.rasterResamplingMode` property for configuring how raster style layers are overscaled. ([#12176](https://github.com/mapbox/mapbox-gl-native/pull/12176))
 * `-[MGLStyle localizeLabelsIntoLocale:]` and `-[NSExpression mgl_expressionLocalizedIntoLocale:]` can automatically localize labels into Japanese or Korean based on the system’s language settings. ([#12286](https://github.com/mapbox/mapbox-gl-native/pull/12286))
+* The `c` and `d` options are supported within comparison predicates for case and diacritic insensitivity, respectively. ([#12329](https://github.com/mapbox/mapbox-gl-native/pull/12329))


🤔 Now that I see this, I wonder if it makes sense to treat the core feature ("support styles using the collator expression") as a separate changelog entry from the platform-specific wrapper...

That’s a good point; we normally mention the JSON equivalent when that’s also new.

Updated the changelog entry.

ChrisLoer · 2018-07-05T23:20:02Z

platform/darwin/docs/guides/Predicates and Expressions.md

+`NSCaseInsensitivePredicateOption`      | `'QUEBEC' =[c] 'Quebec'`
+`NSDiacriticInsensitivePredicateOption` | `'Québec' =[d] 'Quebec'`
+
+Other comparison predicate options are unsupported, namely `l`


To make sure I'm understanding: if you wanted to explicitly do locale-sensitive comparison with something other than the current locale, you'd have to manually craft your expression, because NSPredicate only supports a binary "locale-sensitive"?

That’s correct. The syntax would look something like this:

NSPredicate(format: "MGL_FUNCTION('==', 'QUEBEC', 'Québec', MGL_FUNCTION('collator', %@))", ["case-sensitive": false, "locale": "fr-CA"])

There’s a couple things we could adjust at the SDK level, like allowing locale to be set to an NSLocale instead of a raw locale identifier. But my understanding is that we normally want developers to keep using the default collator for better performance, so maybe it doesn’t help much to simplify the locale-specific collator syntax.

I made some last-minute changes so that this predicate would work:

The collator options object is no longer wrapped in a literal expression when converting from NSExpression to JSON.

The MGL_FUNCTION() syntax is used instead of the ==[…] syntax when a locale is specified.

fabian-guerra

👍🏼

fabian-guerra · 2018-07-06T18:14:23Z

platform/darwin/src/NSPredicate+MGLAdditions.mm

-    if ([op isEqualToString:@">="]) {
-        NSArray *subexpressions = MGLSubexpressionsWithJSONObjects([objects subarrayWithRange:NSMakeRange(1, objects.count - 1)]);
-        return [NSPredicate predicateWithFormat:@"%@ >= %@" argumentArray:subexpressions];
+        return [NSComparisonPredicate predicateWithLeftExpression:subexpressions[0]


When detecting an expression used as a filter, relax the check for expressions to allow comparison expressions to contain a fourth member for the collator.

1ec5 added iOS Mapbox Maps SDK for iOS macOS Mapbox Maps SDK for macOS Core The cross-platform C++ core, aka mbgl runtime styling labels Jul 5, 2018

1ec5 self-assigned this Jul 5, 2018

1ec5 requested review from ChrisLoer, fabian-guerra and anandthakker July 5, 2018 22:14

1ec5 added this to the ios-v4.2.0 milestone Jul 5, 2018

1ec5 commented Jul 5, 2018

View reviewed changes

1ec5 added the GL JS parity For feature parity with Mapbox GL JS label Jul 5, 2018

ChrisLoer reviewed Jul 5, 2018

View reviewed changes

fabian-guerra approved these changes Jul 6, 2018

View reviewed changes

1ec5 force-pushed the 1ec5-darwin-collator-12269 branch 3 times, most recently from 421db04 to e992355 Compare July 7, 2018 01:56

1ec5 added 2 commits July 9, 2018 14:07

[core] Comparison expressions can have 4th member

93a3029

When detecting an expression used as a filter, relax the check for expressions to allow comparison expressions to contain a fourth member for the collator.

[ios, macos] Convert predicate options to collators

aaac2a5

1ec5 force-pushed the 1ec5-darwin-collator-12269 branch from e992355 to aaac2a5 Compare July 9, 2018 21:07

1ec5 merged commit f11ab91 into master Jul 9, 2018

1ec5 deleted the 1ec5-darwin-collator-12269 branch July 9, 2018 23:02

This was referenced Jul 18, 2018

Many NSPredicate and NSExpression operators have no equivalent in style JSON #11786

Open

android-v6.3.0, ios-v4.2.0, macos-v0.9.0 in style spec mapbox/mapbox-gl-js#6988

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Convert between predicate options and collator expressions #12329

Convert between predicate options and collator expressions #12329

1ec5 commented Jul 5, 2018

1ec5 Jul 5, 2018

ChrisLoer Jul 5, 2018

anandthakker Jul 6, 2018

ChrisLoer Jul 6, 2018

1ec5 Jul 6, 2018

anandthakker Jul 9, 2018

ChrisLoer Jul 5, 2018

1ec5 Jul 5, 2018

ChrisLoer Jul 5, 2018

ChrisLoer Jul 5, 2018

1ec5 Jul 5, 2018

1ec5 Jul 6, 2018

ChrisLoer Jul 5, 2018

1ec5 Jul 5, 2018

1ec5 Jul 6, 2018

fabian-guerra left a comment

fabian-guerra Jul 6, 2018

Convert between predicate options and collator expressions #12329

Convert between predicate options and collator expressions #12329

Conversation

1ec5 commented Jul 5, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fabian-guerra left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment