Fix annotation_json endpoint #5621

hughhhh · 2018-08-13T23:19:30Z

Current implementation of /annotations_json/{layerId} is broken. Simplified the endpoint to filter based upon the layer.

@betodealmeida @mistercrunch

…x_annotations_json

betodealmeida · 2018-08-14T02:38:44Z

superset/connectors/sqla/models.py

@@ -47,8 +47,6 @@ def query(self, query_obj):
        error_message = None
        qry = db.session.query(Annotation)
        qry = qry.filter(Annotation.layer_id == query_obj['filter'][0]['val'])
-        qry = qry.filter(Annotation.start_dttm >= query_obj['from_dttm'])
-        qry = qry.filter(Annotation.end_dttm <= query_obj['to_dttm'])


@hughhhh, why do you need to remove this? I'm worried it might bring too many annotations to the browser if we remove this — it would bring all holidays, instead of only the ones being displayed, eg.

how about:

if query_obj['from_dttm']: qry = qry.filter(Annotation.start_dttm >= query_obj['from_dttm'])

looking at the annotations_json endpoint we don't set the from_dttm or to_dttm at all. I just wanted to fix this endpoint to actually work because on master its not working at all.

https://github.com/apache/incubator-superset/blob/master/superset/views/core.py#L1118

betodealmeida · 2018-08-14T02:39:21Z

superset/viz.py

-                    df = db_engine_spec.adjust_df_column_names(df, self.form_data)
-                if self.status != utils.QueryStatus.FAILED:
-                    stats_logger.incr('loaded_from_source')
-                    is_loaded = True


Same here: why is this broken? I'm worried changing this will break existing code.

Removing this looks dangerous, the hasattr condition seems to be for preventing druid-related exception (a condition on that would be more explicit here). I'm pretty sure it's there for a reason.

villebro · 2018-08-14T17:11:24Z

I added that. It's there to check whether the datasource is sqla or not. Apparently the check needs to be for datasource.database, not datasource.database.db_engine_spec.

mistercrunch · 2018-08-14T17:39:22Z

superset/connectors/sqla/models.py

@@ -47,7 +47,8 @@ def query(self, query_obj):
        error_message = None
        qry = db.session.query(Annotation)
        qry = qry.filter(Annotation.layer_id == query_obj['filter'][0]['val'])
-        qry = qry.filter(Annotation.start_dttm >= query_obj['from_dttm'])
+        if query_obj['from_dttm']:
+            qry = qry.filter(Annotation.start_dttm >= query_obj['from_dttm'])


oh we need to same for to_dttm I think

to_dttm is set to the current date in the query object. I can add the guard though 👍

villebro · 2018-08-14T17:42:34Z

@hughhhh To add some context, db_engine_spec.adjust_df_column_names(df, self.form_data) is supposed to be called for sqla datasources. Since Druid doesn't have a db_engine_spec but does have a database attribute, I assumed that I only needed to do check if database has a db_engine_spec attribute. Changing it to if hasattr(self.datasource, 'database') and hasattr(self.datasource.database, 'db_engine_spec'): probably should do the trick.

hughhhh · 2018-08-14T17:44:52Z

@villebro I think the check is fine. The problem is that the AnnotationsDatasource object is missing fields and is basically buffer to leverage the viz.py file. I set the database field to None for now so bypass this condition.

hughhhh · 2018-08-14T17:45:23Z

I'll go with your approach

codecov-io · 2018-08-14T18:04:21Z

Codecov Report

Merging #5621 into master will increase coverage by 0.24%.
The diff coverage is 20%.

@@            Coverage Diff             @@
##           master    #5621      +/-   ##
==========================================
+ Coverage   63.48%   63.73%   +0.24%     
==========================================
  Files         360      360              
  Lines       22886    23113     +227     
  Branches     2547     2548       +1     
==========================================
+ Hits        14529    14730     +201     
- Misses       8342     8368      +26     
  Partials       15       15

Impacted Files	Coverage Δ
superset/connectors/sqla/models.py	`78.66% <0%> (-0.31%)`	⬇️
superset/viz.py	`81.16% <100%> (-0.02%)`	⬇️
superset/models/core.py	`85.09% <0%> (-0.1%)`	⬇️
superset/assets/src/explore/visTypes.jsx	`57.14% <0%> (ø)`	⬆️
superset/assets/src/visualizations/line_multi.js	`0% <0%> (ø)`	⬆️
superset/assets/src/modules/colors.js	`77.55% <0%> (ø)`	⬆️
superset/assets/src/explore/controls.jsx	`45.92% <0%> (ø)`	⬆️
...uperset/assets/src/SqlLab/components/SqlEditor.jsx	`78.09% <0%> (ø)`	⬆️
superset/connectors/druid/models.py	`82.56% <0%> (+0.02%)`	⬆️
... and 2 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 536478e...bd320d8. Read the comment docs.

mistercrunch · 2018-08-14T18:35:26Z

LGTM

* fix annotation_json endpoint * add time back * set db to None * change if condition * remove prop * add guard for to_dttm * linting (cherry picked from commit 85a6da1)

* fix annotation_json endpoint * add time back * set db to None * change if condition * remove prop * add guard for to_dttm * linting

fix annotation_json endpoint

07b12b6

hughhhh added the lyft Related to Lyft label Aug 13, 2018

Merge branch 'master' of github.com:apache/incubator-superset into fi…

3874af6

…x_annotations_json

betodealmeida reviewed Aug 14, 2018

View reviewed changes

add time back

769d53c

mistercrunch reviewed Aug 14, 2018

View reviewed changes

set db to None

8d8a640

hughhhh added 4 commits August 14, 2018 10:47

change if condition

b94f1a6

remove prop

23ab179

add guard for to_dttm

b947a39

linting

bd320d8

hughhhh merged commit 85a6da1 into apache:master Aug 14, 2018

wenchma pushed a commit to wenchma/incubator-superset that referenced this pull request Nov 16, 2018

Fix annotation_json endpoint (apache#5621)

77663ef

* fix annotation_json endpoint * add time back * set db to None * change if condition * remove prop * add guard for to_dttm * linting

mistercrunch added 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 0.28.0 labels Feb 27, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix annotation_json endpoint #5621

Fix annotation_json endpoint #5621

hughhhh commented Aug 13, 2018 •

edited

Loading

betodealmeida Aug 14, 2018

mistercrunch Aug 14, 2018

hughhhh Aug 14, 2018

betodealmeida Aug 14, 2018

mistercrunch Aug 14, 2018

villebro commented Aug 14, 2018

mistercrunch Aug 14, 2018

hughhhh Aug 14, 2018

villebro commented Aug 14, 2018

hughhhh commented Aug 14, 2018

hughhhh commented Aug 14, 2018

codecov-io commented Aug 14, 2018 •

edited

Loading

mistercrunch commented Aug 14, 2018

Fix annotation_json endpoint #5621

Fix annotation_json endpoint #5621

Conversation

hughhhh commented Aug 13, 2018 • edited Loading

betodealmeida Aug 14, 2018

Choose a reason for hiding this comment

mistercrunch Aug 14, 2018

Choose a reason for hiding this comment

hughhhh Aug 14, 2018

Choose a reason for hiding this comment

betodealmeida Aug 14, 2018

Choose a reason for hiding this comment

mistercrunch Aug 14, 2018

Choose a reason for hiding this comment

villebro commented Aug 14, 2018

mistercrunch Aug 14, 2018

Choose a reason for hiding this comment

hughhhh Aug 14, 2018

Choose a reason for hiding this comment

villebro commented Aug 14, 2018

hughhhh commented Aug 14, 2018

hughhhh commented Aug 14, 2018

codecov-io commented Aug 14, 2018 • edited Loading

Codecov Report

mistercrunch commented Aug 14, 2018

hughhhh commented Aug 13, 2018 •

edited

Loading

codecov-io commented Aug 14, 2018 •

edited

Loading