make featuresAt return symbols and use style properties

[ansis]

This adds currently visible text and icons to featuresAt results.

Style properties are used to return features whose rendered representation matches the query.

query test pr: mapbox/mapbox-gl-test-suite#77

fix #303
fix #316
fix #862
fix #902
fix #1822
fix #2100
fix #2103

This increases worker memory usage by around 10% (a couple of MB).
1/3 of the increase comes from retaining the CollisionTile.
2/3 of the increase comes from retaining VectorTileFeatures.

SymbolBucket merges lines to improve placement. This means that sometimes there is more than one VectorTileFeature responsible for that symbol. It currently returns the first VectorTileFeature and ignores the others that were merged into it. Should it do something else?

👀 @jfirebaugh

[jfirebaugh]

Query tests? 😉

cc @mourner

[ansis]

@lucaswoj I just added these style layer creations and recalculations here as part of the style-property-aware featuresAt but I don't think they should stay here. Some of this is a duplication of what Style does but we don't need a full Style here. Any ideas on how this should be split out? Maybe a StyleLayers?

[lucaswoj]

Can you read the Style from tile.symbolBuckets[i].style? We could even add a symbolBucketsById property, if you think that'd have better perf.

EDIT: I see. These are not necessarily symbol layers.

[lucaswoj]

We currently create an instance of Style per Bucket. That now strikes me as silly. We should create a single persistent Style per Worker and update it whenever we receive set layers

EDIT: Ugh. Sorry. Buckets just have StyleLayers.

[lucaswoj]

We should create a single persistent Style per Worker and update it whenever we receive set layers. We can add flags to disable any extra overhead on Style (i.e. fetching TileJSON).

[averas]

Will this addition allow me to distinguish "geographical" hits from text/icon hits? If you look at my example in #1882 where I have three icons (i.e. three layers where two have icon-offset) based on a single geographical point; would I be able to sort out a click on the middle icon (where the point actually is) or will it return all three layers with no way to tell that it was one icon hit and three "geo" hits? Ideally I would want to be able to distinguish clicks on all three icons..

[ansis]

Will this addition allow me to distinguish "geographical" hits from text/icon hits?

After this and #316 there will be no more "geo" hits if I'm understanding you correctly. All hits would be based on the rendered representation of features, not the source geometry. In your case, if you click in the center you would get a hit only for the center icon.

[averas]

@ansis Exactly what I was hoping for! Thanks!

[ansis]

@jfirebaugh added query tests: https://github.com/mapbox/mapbox-gl-test-suite/compare/query-tests?expand=1

This is ready for review

[lucaswoj]

I'm generally feeling good about this code but I'm not sure we can afford to suddenly switch from "geographical" hits to "symbol" hits. Have you thought any more about making that a separate API @ansis?

[lucaswoj]

Can we call the APIs featuresAt and dataFeaturesAt to emphasize their similarity while highlighting their differences?

If I could do a clean-slate rename, I might put a verb in the name (and remove the preposition at the end of the name) like findFeatures and findDataFeatures but I think backwards compatibility is more important here.

[ansis]

Can we call the APIs featuresAt and dataFeaturesAt to emphasize their similarity while highlighting their differences?

The current implementation doesn't have very many similarities. They accept different parameters and return different things. I think the names should be very different.

If I could do a clean-slate rename, I might put a verb in the name (and remove the preposition at the end of the name) like findFeatures and findDataFeatures but I think backwards compatibility is more important here.

I think it's ok to rename it since we're already breaking it's behavior heavily. I don't like featuresAt as a name either. I'm also wondering if featuresAt and featuresIn should be merged.

[lucaswoj]

We need to take a moment to think about the linguistic semantics of these API methods

• featuresAt
• featuresIn
• getSourceTileData

In particular, we need to

• emphasize the ways these methods are similar (they introspect the map's features)
• highlight the ways these methods are different (query geometry shape, feature type)
• keep these method names as short as possible
• follow conventions in API method naming (use verb phrases for method names)
• follow conventions in English grammar (ending a phrase with a preposition is bad practice)

If we can tolerate a clean-slate API redesign (preserving the existing methods as "deprecated" for a few releases) then I propose the following method names, each of which can handle both bbox and point-and-radius queries.

• findStyledFeatures(options)
• findFeatures(options)

[lucaswoj]

#2125 (comment) makes me think we might want to drop featuresAt entirely

[jfirebaugh]

For rendering queries, I think we'd keep featuresAt and featuresIn equivalents, but drop the radius option. The goal is that they both become exact queries of the rendered representation.

You could implement featuresAt in terms of featuresIn with a degenerate bounding box consisting of a single point, but I think we'd still want it for API clarity.

[ansis]

If we can tolerate a clean-slate API redesign (preserving the existing methods as "deprecated" for a few releases) then I propose the following method names

I'm fine with a breaking API redesign without deprecation. We don't want to support the old version for any meaningful length of time so we might as well drop it now.

each of which can handle both bbox and point-and-radius queries.

I think we should leave getSourceTileData (or whatever it's called) as a low level api that just returns all the data and doesn't do any geometric filtering (6c1b03c):

• we aren't that clear on what the use cases for this could be so it's better to provide the low level access and add more once we learn about how it's used
• the source data is tiled which means there are a lot of gotcha's with clipped and duplicate features. Returning each tile as a complete and separate feature collection is upfront about this.

findStyledFeatures(options)
findFeatures(options)

I'm a bit afraid of overloading the name Features with multiple meanings, but if we need to use it we should at least make findFeatures into something more specific like findSourceFeatures.

but drop the radius option

Could the radius option still be useful for less precise selection like touch?

You could implement featuresAt in terms of featuresIn with a degenerate bounding box consisting of a single point, but I think we'd still want it for API clarity.

Sounds good to me.

[jfirebaugh]

Could the radius option still be useful for less precise selection like touch?

For that use case you want to use featuresIn so that you don't run into #2125.

[averas]

Perhaps there is some inspiration and experience that could be drawn from OpenLayers 3 here. They have been doing pixel perfect hit detection of vector data since their early releases:

HIT DETECTION
The new vector implementation includes a technique for detecting features at a given pixel. This technique involves re-rendering features, one by one, in a 1 pixel by 1 pixel canvas, and, for each feature, testing the presence of a color using Canvas’ getImageData function. This technique is very robust (pixel-perfect), and it is proven to be very efficient as well.

[ansis]

@averas thanks for the suggestion. We considered that approach but some experiments showed that rendering each feature one-by-one would be too slow and that reading back the pixels with gl.readPixels is slow and blocks the main thread . It could work well with a limited number of features but our goal right now is to make every map feature interactive. Calculating intersections on the cpu has other downsides but hopefully it will work out.

[ansis]

I dropped radius, combined featuresAt and featuresIn internally, and fixed featuresIn when when the map is rotated or tilted. The main remaining things are:

• review
• decided on external api and naming

[mourner]

@ansis reviewed the feature tree geometry code. It generally looks good, the only thing that caught my attention is that you rebuild full geometries on every query to account for offsets/etc., which is expensive. I'd prefer the intersection code to account for offsets during the checks, so that we only touch a fraction of all geometries due to early rejects/accepts.

[mourner]

@ansis yeah, I would also suggest to settle on worker-based featuresAt for now and revisit later while keeping compatibility. The PR is already pretty big.

[lucaswoj]

fixes #1785

[averas]

I thought I'd raise an issue I've ran into a couple of times before this PR is merged, since a solution might also involve breaking changes...

When calling featureIn on multiple layers, such as {layers: ['layer1', 'layer2']} on a large data set (say a couple of hundred thousand points) there is a certain cost and time associated with such as call, which is natural.

In many of my use cases, however, I want to pass on the results, or an aggregation of the results (for example a count), to a visualization (say d3), but before I can do that I have to filter/aggregate the data set again! Why? Because the result returned by featuresIn is flattened to a single array with the layer information present on each object, such as:

[
{layer: {...}, properties: {...} ... },
{layer: {...}, properties: {...} ... },
{layer: {...}, properties: {...} ... }
]

To pass this data on I first have to do a features.filter(... layer.id === "..."). Normally, for small data sets, this isn't really a problem. But as size increases it becomes more and more problematic to filter the data set twice, so to speak.

Would it make sense to evaluate whether it would actually be better to return a hierarchical representation of the results where features are nested under their respective layer?

[ansis]

rebased on master. 8bdf3c7 was the latest before the rebase.