GT4Py v1.0.4

Summary of changes since v1.0.3

License

GT4Py is now licensed under the BSD license. SPDX-License-Identifier: BSD-3-Clause

Cartesian

• Introduced a GlobalTable which is a data dimensions only Field. Access is provided via .A which also becomes a way to access data dimensions on regular Field.
• Added an error message if a non existing backend is selected.
• Allow setting compiler optimization level and flags on a per stencil basis
• Added GT4PY_EXTRA_COMPILE_ARGS, GT4PY_EXTRA_LINK_ARGS and DACE_DEFAULT_BLOCK_SIZE environment variables
• Fixes for the DaCe backend
• Various style modernizations

Deprecation

• The cuda backend is deprecated (enable by setting environment variable GT4PY_GTC_CUDA_USE=1), use gt:gpu or dace:gpu instead.

Development

• Replaced flake8, black with ruff
• Added CI plan with GH200 GPUs

All changes

• feature[next]: Temporary extraction heuristics by @tehrengruber in #1341
• fix[next]: Detection of ROCM vs CUDA gpu runtime by @edopao in #1448
• ci: Update CI config to use new template for container build by @edopao in #1450
• feat[next]: Slicing field to 0d to return field not scalar by @nfarabullini in #1427
• feature[next]: Improve CollapseTuple pass by @tehrengruber in #1350
• fix[next][dace]: Bugfix in neighbor reduction by @edopao in #1456
• refactor[next]: Change foast lowering from iterator of tuple to tuple of iterator by @tehrengruber in #1449
• fix[next]: Disable collapse tuple if propagation by @tehrengruber in #1454
• fix[next][dace]: Bugfix for nested neighbor reduction by @edopao in #1457
• feat[next][dace]: Use gt4py symbols for field size as dace array shape by @edopao in #1458
• feature[next]: toolchain configuration interfaces by @DropD in #1438
• feat[next][dace]: Remove offsets in connectivity arrays by @edopao in #1460
• build: Remove duplicated factory-boy dependency. by @egparedes in #1465
• feat[next][dace]: Remove array offsets by @edopao in #1459
• feat[next]: Embedded support for skip value connectivities by @havogt in #1441
• feat[next]: new domain slice syntax by @nfarabullini in #1453
• feat[next]: config improvements by @DropD in #1461
• fix[next][dace]: Fix translation of if statement from tasklet to inter-state condition by @edopao in #1469
• feat[cartesian]: Setting extra compile args and DaCe block size via env variables by @stubbiali in #1462
• bug[next]: Fix reverse operators in embedded by @havogt in #1467
• build: fix tox requirements task and hypothesis setting with updated version by @egparedes in #1472
• feature[next]: Fix caching stability by @tehrengruber in #1473
• feat[next][dace]: make canonical representation of field domain optional by @edopao in #1476
• refactor[next]: Refactor workflow based executors and move the backend class. by @DropD in #1470
• feat[next][dace]: Add gt4py workflow for the DaCe backend by @edopao in #1477
• fix[next][dace]: accept runtime lift_mode as argument to dace backend by @edopao in #1481
• feat[cartesian]: read-only data dims direct access & Fields by @FlorianDeconinck in #1451
• style: Fix leftovers of moving lint infrastructure to ruff by @egparedes in #1480
• feature[next]: Inline center deref lift vars by @tehrengruber in #1455
• bug[next]: Increase recursion limit in TraceShift pass by @tehrengruber in #1482
• bug[next]: Fix CSCS CI by pinning cupy to 12.3.0 by @tehrengruber in #1485
• bug[next]: Mark cupy tests as requires gpu by @tehrengruber in #1483
• feat[next]: variadic generic type by @nfarabullini in #1486
• refactor[next]: workflowify PAST -> ITIR toolchain step by @DropD in #1479
• style[next]: stricter typing in ffront by @havogt in #1492
• feat[next]: concat_where for boundary conditions by @havogt in #1468
• style[next]: improve typing in next.type_translation by @havogt in #1493
• bug[next]: fix _Dedup for tuples in type_inference by @havogt in #1495
• style[next]: more strict typing by @havogt in #1494
• fix[next][dace]: Fixes to DaCe backend to support latest ITIR by @edopao in #1499
• feature[next]: Validate no dynamic offsets by @tehrengruber in #1504
• build: Update mypy configuration by @egparedes in #1505
• refactor[next]: NamedRange/NamedIndex tuple to NamedTuple by @nfarabullini in #1490
• bug[eve]: Preserve annex None value in visitors by @tehrengruber in #1508
• feature[next]: Improve ITIR constant folding by @tehrengruber in #1510
• bug[next]: Improve ITIR Pretty Printer by @tehrengruber in #1509
• refactor[next]: make function definition -> PAST into a workflow step by @DropD in #1500
• bug[next]: Fix gtfn code generation for negative literals by @tehrengruber in #1511
• feat[next][dace]: Remove old dace backend (replaced by workflow infrastructure) by @edopao in #1503
• build: add ruff to the automated requirement generation workflow by @egparedes in #1497
• build: bump minimum version of typing_extensions requirement by @egparedes in #1512
• Migrate from discontinued bump2version to bump-my-version by @egparedes in #1514
• feat[next]: workshop material and docs field Dims type hint update by @nfarabullini in #1489
• feat[next][dace]: Filter unused connectivity tables by @edopao in #1502
• bug[next]: fix minimum version of bump-my-version by @havogt in #1515
• style: update requirements and change ruff-format style to stable by @egparedes in #1517
• style: minimize vertical space style by @egparedes in #1518
• fix[next]: replace extended_runtime_checkable workaround by upgrading typing_extensions by @egparedes in #1519
• build: bump required typing_extensions version to skip buggy versions by @egparedes in #1522
• fix[next]: Fix Program.itir by @DropD in #1523
• feat[next][dace]: Several features for icon4py-backend integration by @edopao in #1525
• refactor[next]: itir embedded: cleaner closure run by @havogt in #1521
• refactor[next]: Fencil to itir.Program for gtfn by @havogt in #1524
• refactor[next]: Use type specification for itir.Literal by @tehrengruber in #1529
• refactor[next]: Use is_call_to instead of equality comparison with itir.Ref. by @tehrengruber in #1532
• test[next]: delete cppbackend_tests by @havogt in #1534
• refactor[next]: move lift_mode itir test fixture into program_processor by @havogt in #1533
• feat[cartesian]: gtc cuda backend deprecation by @FlorianDeconinck in #1498
• refactor[next]: workflowify step3 by @DropD in #1516
• ci[cartesian]: env var on CSCS-CI to suppress cuda backend deprecation error by @DropD in #1541
• ci: define pytest addopts settings as env var in tox config by @egparedes in #1542
• feat[cartesian]: Feedback when a bad backend is selected by @FlorianDeconinck in #1544
• feat[next]: extend embedded implementation of premap() by @egparedes in #1501
• build: update frozen dependencies by @havogt in #1543
• refactor[next]: cleaner toolchain by @DropD in #1537
• feat[next][dace]: Modify name of stride symbols by @edopao in #1548
• ci: add manual trigger for daily...

GT4Py v1.0.3

Summary of changes since v1.0.2

General

• Support for Python 3.11 and updated dependencies

Testing

• Testing of Jupyter notebooks in CI

All changes

• fix[next][dace]: Use constant shape for neighbor tables in local dimension by @edopao in #1422
• feat[next][dace]: use new LoopRegion construct for scan operator by @edopao in #1424
• fix[next][dace]: Remove usage of deprecated API dace.Memlet.simple by @edopao in #1425
• ci: test jupyter notebooks by @havogt in #1426
• feat[next]: Enable tests for embedded with cupy by @havogt in #1372
• fix[next][dace]: Bugfix in deref (dynamic memory allocation) by @edopao in #1430
• build: update min requirements by @havogt in #1435
• test[next]: fix obsolete asarray by @havogt in #1436
• test[next]: Add unit test for embedded inverse_image and fix bugs by @egparedes in #1432
• build: Update gridtools-cpp version to 2.3.2 by @havogt in #1437
• feat[next]: add tests with mesh with skip values by @havogt in #1433
• fix[cartesian]: Add missing cstdint header in gtcpp codegen by @tehrengruber in #1439
• feat[next][dace]: Modified the file caching. by @philip-paul-mueller in #1434
• bug[next]: allow fields of different sizes in tuple in itir embedded by @havogt in #1442
• fix[next][dace]: Fix for neighbor reduction with skip values by @edopao in #1443
• feat[next][dace]: Add support for lift expressions in neighbor reductions (no unrolling) by @edopao in #1431
• feat[next][dace]: DaCe support for temporaries by @edopao in #1351
• bug[next]: fix field_operator caching by @havogt in #1445
• build: Update requirements versions and scripts to support 3.11 by @egparedes in #1444
• build: Update deployment action with trusted publisher by @havogt in #1423
• release: v1.0.3 by @havogt in #1446

Full Changelog: v1.0.2...v1.0.3

GT4Py v1.0.2

Summary of changes since v1.0.1

Cartesian

• Compatibility of gt4py.next Fields with gt4py.cartesian computations.
• Fixes for DaCe 0.15.1 compatibility.
• Added log10 as native function.
• Make scipy optional: get scipy by installing gt4py[full] for best performance with numpy backend.

Storage

• Refactored low-level storage allocation.

All changes

• Fix typing of cast_ builtin in ITIR type inference by @tehrengruber in #1182
• Fix codegen of dace:cpu by @gronerl in #1186
• Extend ITIR type inference to collect the types of all subexpressions by @tehrengruber in #1155
• Enhancements to CI and development tools configurations by @egparedes in #1163
• Eve: document behavior of annex by @havogt in #1137
• Update CI docs by @DropD in #1170
• Fix conditional test in frontend by @tehrengruber in #1190
• compiledb name: strip non-alphanumeric chars by @havogt in #1191
• Improve Gitpod pre-commit and remove obsolete files by @havogt in #1176
• GTFN: enable more tests by @havogt in #1192
• Implementations for as_offset by @nfarabullini in #1143
• Small fixes to FOAST -> ITIR lowering by @tehrengruber in #1196
• Implementation for min and max in domain by @nfarabullini in #1184
• config: Update development tools configuration and development documentation by @egparedes in #1189
• ITIR type inference fix stencil closure location constraints by @tehrengruber in #1185
• Fix lambda inliner if parameter is not referenced by @tehrengruber in #1206
• Onboarding checklist by @DropD in #1207
• Experimental GTFN Executor Caching by @tehrengruber in #1197
• Small improvements to ITIR type inference by @tehrengruber in #1205
• Fix GTFN Executor Caching by @tehrengruber in #1214
• ci: Fix 'no_impliit_reexport' typo in mypy config by @havogt in #1213
• Enhance CSE to support lambdas by @tehrengruber in #1175
• Replace partial shifts by neighbor lists by @havogt in #1179
• bug[next]: Support type inference for math builtins by @havogt in #1217
• fix[next]: Fix cpp_backend tests by @havogt in #1222
• refactor[next]: Typing for bindings by @havogt in #1218
• fix[next]: Cleaner SID bindings by @havogt in #1225
• tests[all]: update the test file structure by @DropD in #1226
• Add proper test condition to tests in test_execution.py by @gronerl in #1208
• Simplify ITIR shift collector (TraceShifts) by @tehrengruber in #1216
• feature[next]: Tuple in bindings by @havogt in #1221
• refactor[next]: interface for creating workflow variants. by @DropD in #1224
• fix[next]: ITIR type inference for dynamic offsets by @tehrengruber in #1228
• test[next]: Add todo for improving foast lowering by @tehrengruber in #1230
• feature[eve]: Add functionality to preserve parts of the annex in NodeTranslators by @tehrengruber in #1227
• Propagate foast & past typing information to ITIR by @tehrengruber in #1199
• build: enhance dependency management and make scipy optional. by @egparedes in #1215
• test[next]: Move ffront laplacian tests by @tehrengruber in #1229
• feature[next]: Allow manually selecting the grid type of a field operator by @tehrengruber in #1239
• bug[next]: Fix shift / remap lowering by @tehrengruber in #1231
• feature[next]: bindings for sparse fields by @havogt in #1237
• refactor[next]: itir_makers -> iterator.makers by @DropD in #1241
• fix[next]: Yet another test parametrization cleanup by @havogt in #1244
• fix[next,gtfn]: fix too eager generation of integral_constants by @tehrengruber in #1204
• Disable CSE for an if with a condition calling can_deref by @tehrengruber in #1246
• feature[next]: Allow passing grid_type to field operator by @tehrengruber in #1247
• test[next]: simplify boilerplate for fieldview single feature tests by @DropD in #1236
• feat[next]: use context vars instead of global state in embedded iterator execution by @havogt in #1120
• feature[next]: Extend Single Static Assignment (SSA) pass to support if statements by @tehrengruber in #1250
• fix[next]: improved tuple collapse by @havogt in #1243
• feature[next]: Scalar if by @tehrengruber in #1079
• docs[cartesian]: fix paths in documentation by @edopao in #1253
• Ignore system paths for C++ libraries by @petiaccja in #1257
• build: freeze typing_extensions version by @egparedes in #1256
• feature[next]: Improve CSE pass by @tehrengruber in #1252
• feature[cartesian]: Add log10: as a native function by @oelbert in #1262
• fix[next]: Use dataclasses.replace for with_grid_type and with_backend by @tehrengruber in #1264
• feature[next]: Save type to annex in ITIR type inference by @tehrengruber in #1259
• refactor[next]: add public API to top level by @DropD in #1263
• Test execution conversion by @nfarabullini in #1265
• change of fieldview_backend by @nfarabullini in #1266
• Test refactoring 1 by @nfarabullini in #1268
• feature[next]: Remove int type from FOAST, PAST, ITIR by @tehrengruber in #1255
• refactor[next]: Frontend test refactoring part 2 by @nfarabullini in #1270
• fix[next]: Fix CSE extraction for if statements by @tehrengruber in #1273
• feature[next]: Add support for positional-or-keyword arguments by @tehrengruber in #1248
• refactor[next]: Frontend test refactoring part 3 by @nfarabullini in #1274
• feature[next]: Use global type information in CollapseTuple pass by @tehrengruber in #1260
• feature[next]: Pretty format Iterator roundtrip backend output by @tehrengruber in #1281
• bug[next]: itir.embedded: fix shift inside scan pass by @tehrengruber in #1280
• feature[next]: Fix function signature validation for tuple arguments by @tehrengruber in #1279
• feature[next] enable field origin in GTFN backend by @petiaccja in #1277
• [next] DaCe backend by @petiaccja in #1245
• ci: uplift version of nvidia base image by @edopao in #1285
• documentation[next]: CODING_GUIDELINES.md edits by @nfarabullini in #1272
• fix[next]: ensure scan operator init, carry, return type is consistent by @nfarabullini in #1286
• refactor[next]: use eve.codegen format utils instead of black in ITIR roundtrip backend by @egparedes in #1287
• docs: Move release documentation to docs/development by @egparedes in #1238
• refactor[next] error handling by @petiaccja in #1275
• feat[next]: DaCe scan operator by @edopao in #1284
• fix[next]: DaCe scalar argument in stencil closure by @edopao in #1293
• feat[next]: enable DaCe test macro by @edopao in #1288
• fix[next]: DaCe scan connectivities by @edopao in #1297
• deps: add nanobind and (optional) jax for future features by @havogt in #1306
• feature[cartesian]: Add support for HIP/ROCm by @stubbiali in #1278
• feature[next]: Impr...

GT4Py v1.0.1

First version including the experimental gt4py.next aka Declarative GT4Py. The gt4py.next package is excluded from semantic versioning.

Changes since v1.0.0

Cartesian

• Parametrized dtype: see option 4 of the gtscript concept workshop

GT4Py v1.0.0

First official release of the Cartesian GT4py.

Changes since the pre-release version v0.0.1

Added

• Remove the Storage classes and implement the new "No Storage" concept.
• Support interfaces for calling stencils with arbitrary buffer objects (for details see docs/arrays.rst and PR #868).
• Updated documentation, logo and license headers.

Changed

• Important gt4py package reorganization and modularization.
• Moved most of existing functionality into gt4py.cartesian.
• Moved gtc package into gt4py.cartesian.
• Moved eve package into gt4py.
• Refactored gt4py.storage to avoid dependencies on gt4py.cartesian.
• Tests split into sub-packages and reorganized as unit_tests and integration_tests.

Cartesian GT4Py v0.1.0

Last development version using old-style Storage class.