Skip to content

Commit

Permalink
Update doc and sample dag for Quicksight (#23653)
Browse files Browse the repository at this point in the history
  • Loading branch information
vincbeck committed May 12, 2022
1 parent afdfece commit 1a8f1ee
Show file tree
Hide file tree
Showing 3 changed files with 30 additions and 23 deletions.
16 changes: 9 additions & 7 deletions airflow/providers/amazon/aws/example_dags/example_quicksight.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,11 +19,12 @@
from datetime import datetime

from airflow import DAG
from airflow.models.baseoperator import chain
from airflow.providers.amazon.aws.operators.quicksight import QuickSightCreateIngestionOperator
from airflow.providers.amazon.aws.sensors.quicksight import QuickSightSensor

DATA_SET_ID = os.getenv("DATA_SET_ID", "DemoDataSet_Test")
INGESTION_NO_WAITING_ID = os.getenv("INGESTION_NO_WAITING_ID", "DemoDataSet_Ingestion_No_Waiting_Test")
DATA_SET_ID = os.getenv("DATA_SET_ID", "data-set-id")
INGESTION_ID = os.getenv("INGESTION_ID", "ingestion-id")

with DAG(
dag_id="example_quicksight",
Expand All @@ -36,20 +37,21 @@
# and does not wait for its completion
# [START howto_operator_quicksight_create_ingestion]
quicksight_create_ingestion_no_waiting = QuickSightCreateIngestionOperator(
task_id="quicksight_create_ingestion_no_waiting",
data_set_id=DATA_SET_ID,
ingestion_id=INGESTION_NO_WAITING_ID,
ingestion_id=INGESTION_ID,
wait_for_completion=False,
task_id="sample_quicksight_no_waiting_dag",
)
# [END howto_operator_quicksight_create_ingestion]

# The following task checks the status of the QuickSight SPICE ingestion
# job until it succeeds.
# [START howto_sensor_quicksight]
quicksight_job_status = QuickSightSensor(
task_id="quicksight_job_status",
data_set_id=DATA_SET_ID,
ingestion_id=INGESTION_NO_WAITING_ID,
task_id="check_quicksight_job_status",
ingestion_id=INGESTION_ID,
)
# [END howto_sensor_quicksight]
quicksight_create_ingestion_no_waiting >> quicksight_job_status

chain(quicksight_create_ingestion_no_waiting, quicksight_job_status)
36 changes: 20 additions & 16 deletions docs/apache-airflow-providers-amazon/operators/quicksight.rst
Original file line number Diff line number Diff line change
Expand Up @@ -15,43 +15,47 @@
specific language governing permissions and limitations
under the License.
=================
Amazon QuickSight
=================

Amazon QuickSight Operators
========================================
`Amazon QuickSight <https://aws.amazon.com/quicksight/>`__ is a fast business analytics service to build visualizations,
perform ad hoc analysis, and quickly get business insights from your data.
Amazon QuickSight seamlessly discovers AWS data sources, enables organizations to scale to hundreds of thousands of
users, and delivers fast and responsive query performance by using the Amazon QuickSight Super-fast,
Parallel, In-Memory, Calculation Engine (SPICE).

Prerequisite Tasks
------------------

.. include:: _partials/prerequisite_tasks.rst

Overview
--------

Airflow to Amazon QuickSight integration allows users to create and start the SPICE ingestion for dataset.

- :class:`~airflow.providers.amazon.aws.operators.quicksight.QuickSightCreateIngestionOperator`
- :class:`~airflow.providers.amazon.aws.sensor.quicksight.QuickSightSensor`
Operators
---------

.. _howto/operator:QuickSightCreateIngestionOperator:

Amazon QuickSight CreateIngestion Operator
"""""""""""""""""""""""""""""""""""""""""""
Amazon QuickSight create ingestion
==================================

The QuickSightCreateIngestionOperator Creates and starts a new SPICE ingestion for a dataset.
The operator also refreshes existing SPICE datasets
The ``QuickSightCreateIngestionOperator`` creates and starts a new SPICE ingestion for a dataset.
The operator also refreshes existing SPICE datasets.

.. exampleinclude:: /../../airflow/providers/amazon/aws/example_dags/example_quicksight.py
:language: python
:dedent: 4
:start-after: [START howto_operator_quicksight_create_ingestion]
:end-before: [END howto_operator_quicksight_create_ingestion]

Sensors
-------

.. _howto/sensor:QuickSightSensor:

Amazon QuickSight Sensor
""""""""""""""""""""""""
Amazon QuickSight ingestion sensor
==================================

The QuickSightSensor wait for Amazon QuickSight CreateIngestion until it reaches a terminal state
The ``QuickSightSensor`` waits for an Amazon QuickSight create ingestion until it reaches a terminal state.

.. exampleinclude:: /../../airflow/providers/amazon/aws/example_dags/example_quicksight.py
:language: python
Expand Down
1 change: 1 addition & 0 deletions docs/spelling_wordlist.txt
Original file line number Diff line number Diff line change
Expand Up @@ -892,6 +892,7 @@ hiveconfs
hivemeta
hiveserver
hmsclient
hoc
homebrew
honoured
hookable
Expand Down

0 comments on commit 1a8f1ee

Please sign in to comment.