Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use nightly wheels where available to run tests and support NumPy 2.0 #3427

Merged
merged 7 commits into from
Jun 19, 2024

Conversation

MridulS
Copy link
Member

@MridulS MridulS commented Apr 18, 2024

I'm not sure if we are testing against nightly builds of packages (where available) of scipp dependencies. Numpy 2.0 will have major breaking changes (which may or may not affect us).

@YooSunYoung
Copy link
Member

I think it should be another nightly test, not part of the nightly build of the scipp.
Or maybe the last part of the build & test so that it doesn't block the uploading of the built package.

Can you add a nightly test action or move it to the end?

requirements/extra.in Outdated Show resolved Hide resolved
@MridulS
Copy link
Member Author

MridulS commented May 8, 2024

The tests here should fail as the code needs to be updated numpy2.0

The current run is with the nightly builds of the following deps:

numpy-2.1.0.dev0
scipy-1.14.0.dev0
pandas-3.0.0.dev0+906.g882b228a49
xarray-2024.3.1.dev55+g71661d5b
ipython-8.25.0.dev0

lib/python/dtype.cpp Outdated Show resolved Hide resolved
@@ -671,7 +671,7 @@ def test_arange_datetime_from_str_raises_if_step_has_no_unit():


def test_arange_datetime_from_str_raises_given_string_with_timezone():
with pytest.raises(ValueError, match='timezone'):
with pytest.raises(UserWarning, match='timezone'):
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MridulS
Copy link
Member Author

MridulS commented May 8, 2024

There are a couple of more tests failing, I'll get back to this later :)

@MridulS MridulS changed the title Use nightly wheels where available to run tests Use nightly wheels where available to run tests and support NumPy 2.0 May 8, 2024
@@ -897,7 +897,7 @@ def test_bin_with_explicit_lower_precision_drops_rows_outside_domain():
x = sc.linspace('x', 0.0, 1.0, 3, unit='m', dtype='float32')
da = table.bin(x=x)
size = da.bins.size().sum().value
table.coords['x'].values[0] = 2.0 * np.finfo(np.float32).max
table.coords['x'].values[0] = np.float64(2.0) * np.finfo(np.float32).max
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

@SimonHeybrock SimonHeybrock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes look good. Are we now just waiting for Numba support?

@MridulS
Copy link
Member Author

MridulS commented May 27, 2024

With the changes in this PR and a scipp built with numpy2, mac, linux and windows passes the test suite with python 3.10

Some of the changes here are backward incompatible, but I can update the code to make it work with numpy < 2 too.

To make sure we don't distribute broken scipp, we need to release scipp built with numpy2 once numpy makes a 2.0 release (16th June - numpy/numpy#24300 (comment)).

We can also cut a release of scipp with an upper pin on numpy if we do want to move to numpy 2 directly.

Copy link
Member

@jl-wynen jl-wynen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good so far. But some tests are still broken.

src/scipp/io/hdf5.py Show resolved Hide resolved
pyproject.toml Outdated Show resolved Hide resolved
@MridulS
Copy link
Member Author

MridulS commented Jun 13, 2024

This is ready to be reviewed. The PR does 2 things

@MridulS MridulS enabled auto-merge (squash) June 13, 2024 14:36
if cls.__module__ == 'numpy' and cls.__name__[:4] == 'bool':
type_name = 'numpy.bool'
else:
type_name = cls.__name__
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this break compatibility with old files? If so, please add a script to https://github.com/scipp/scipp/tree/main/tools/migration

@@ -17,3 +18,4 @@ dependencies:
- tbb=2020.2.*
- tbb-devel=2020.2.*
- tox=3.25.0
- numpy=2.0.0rc2
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can the rc2 be removed?

@MridulS MridulS merged commit 0682656 into scipp:main Jun 19, 2024
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

5 participants