Skip to content

Commit

Permalink
(#2337) Handle array values in agate dataframe building
Browse files Browse the repository at this point in the history
  • Loading branch information
drewbanin committed May 2, 2020
1 parent 68babfb commit 0781cef
Show file tree
Hide file tree
Showing 4 changed files with 46 additions and 1 deletion.
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@
- Fix "Object of type Decimal is not JSON serializable" error when BigQuery queries returned numeric types in nested data structures ([#2336](https://github.com/fishtown-analytics/dbt/issues/2336), [#2348](https://github.com/fishtown-analytics/dbt/pull/2348))
- No longer query the information_schema.schemata view on bigquery ([#2320](https://github.com/fishtown-analytics/dbt/issues/2320), [#2382](https://github.com/fishtown-analytics/dbt/pull/2382))
- Add support for `sql_header` config in incremental models ([#2136](https://github.com/fishtown-analytics/dbt/issues/2136), [#2200](https://github.com/fishtown-analytics/dbt/pull/2200))
- Postgres array types can now be returned via `run_query` macro calls ([#2337](https://github.com/fishtown-analytics/dbt/issues/2337), [#2376](https://github.com/fishtown-analytics/dbt/pull/2376))

### Under the hood
- Added more tests for source inheritance ([#2264](https://github.com/fishtown-analytics/dbt/issues/2264), [#2291](https://github.com/fishtown-analytics/dbt/pull/2291))
Expand Down
5 changes: 4 additions & 1 deletion core/dbt/adapters/sql/connections.py
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,10 @@ def get_result_from_cursor(cls, cursor: Any) -> agate.Table:
rows = cursor.fetchall()
data = cls.process_results(column_names, rows)

return dbt.clients.agate_helper.table_from_data(data, column_names)
return dbt.clients.agate_helper.table_from_data_flat(
data,
column_names
)

def execute(
self, sql: str, auto_begin: bool = False, fetch: bool = False
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@

{% macro test_array_results() %}

{% set sql %}
select ARRAY[1, 2, 3, 4] as mydata
{% endset %}

{% set result = run_query(sql) %}
{% set value = result.columns['mydata'][0] %}

{# This will be json-stringified #}
{% if value != "[1, 2, 3, 4]" %}
{% do exceptions.raise_compiler_error("Value was " ~ value) %}
{% endif %}

{% endmacro %}
25 changes: 25 additions & 0 deletions test/integration/060_run_query_tests/test_pg_types.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@

from test.integration.base import DBTIntegrationTest, use_profile
import json

class TestPostgresTypes(DBTIntegrationTest):

@property
def schema(self):
return "pg_query_types_060"

@property
def models(self):
return "models"

@property
def project_config(self):
return {
'config-version': 2,
'macro-paths': ['macros'],
}

@use_profile('postgres')
def test__postgres_nested_types(self):
result = self.run_dbt(['run-operation', 'test_array_results'])
self.assertTrue(result.success)

0 comments on commit 0781cef

Please sign in to comment.