PEP621 and `hatch` scripts #202

LecrisUT · 2023-08-09T09:50:32Z

Here are various modernizaitons to the pyproject

Switched to PEP621
~~Switched to src-layout structure.~~ Dropped as people are opposed to it
Changed the build backend to hatchling
Added some hatch environments and scripts to align with tmt a bit
Switched the bin/fmf file to a script entry-point
Functional change: Switched to dynamic version
Simplified the spec file
~~Removed .travis.yaml~~ Small cleanups #237
python-coveralls dependency was removed. Not sure where that one was used
Switch PyPI publishing to trusted-publishing. This eliminates the use of the secret token. Closes Missing sdist in PyPI #224

lukaszachy · 2023-08-09T15:32:22Z

We need to be able to build on epel-9 which fails now (very likely contains too old macros/utils).

@psss WHat about epel-8 though, can we stop providing updates as we did with tmt? epel-8 has even older macros ;)

lukaszachy · 2023-08-09T15:34:34Z

Makefile contains useful stuff as 'make (s)rpm', 'make clean'. IMO we need leave it (it can call hatch commands though).

lukaszachy · 2023-08-09T15:36:12Z

Switched to src-layout structure.

We moved AWAY from it in the past. Easier to use libraries locally when python path works out of the box.

LecrisUT · 2023-08-09T15:55:31Z

We need to be able to build on epel-9 which fails now (very likely contains too old macros/utils).

Indeed, I was just about to look into that. I was debugging locally and I hit an issue with it just now about dynamic version. It's weird that the same setup is used in packit. That needs a bit of debugging.

Makefile contains useful stuff as 'make (s)rpm', 'make clean'. IMO we need leave it (it can call hatch commands though).

Sure thing. I would still prefer the CI, packit and spec to use the direct commands though. Fine with that?

We moved AWAY from it in the past. Easier to use libraries locally when python path works out of the box.

Roger. It is a dev preference in the end

LecrisUT · 2023-08-09T18:34:09Z

I am not sure why Rhel9 build is failing. I can't seem to debug it to check if the tmp folder includes .git_archival.txt. Any ideas on how to debug that?

LecrisUT · 2023-08-09T19:13:23Z

Seems like the error message in the JsonSchemaValidator is different on the other distros:

[31mFailed to load step data for DiscoverFmfStepData: Field 'DiscoverFmfStepData:ref' must be a string, 'int' found.�[0m
:: [ 18:35:04 ] :: [   FAIL   ] :: File '/var/tmp/rlRun_LOG.VEO4lNoT' should contain 'Failed to load step data for DiscoverFmfStepData: Field 'DiscoverFmfStepData:ref' must be unset or a string, 'int' found.'

(note the lack of must be unset or...

And similar with tmt errors reported.

psss · 2023-08-10T07:46:47Z

@psss What about epel-8 though, can we stop providing updates as we did with tmt? epel-8 has even older macros ;)

Yes, I'd say we should keep fmf and tmt aligned, let's stop providing updates for el-8.

LecrisUT · 2023-08-10T08:27:52Z

Ok, this should be ok for this PR. @psss can someone look into the testing-farm failures? They are in the integration test and it should just need a more relaxed check of the fail message.

psss · 2023-08-10T12:06:36Z

@psss can someone look into the testing-farm failures? They are in the integration test and it should just need a more relaxed check of the fail message.

The testing farm failures were most probably caused by the fresh tmt not being in the stable repo. Should be good now.

psss · 2023-08-10T12:06:45Z

/packit test

LecrisUT · 2023-08-11T09:45:19Z

@lukaszachy , @psss Ok, with this Epel 9 is fixed. Thanks to Nikola from packit for debugging the issue. I've also expanded the github tests, can someone trigger them? I've also enabled codecov, but on my project it did not work without a secret token. Maybe that needs to be setup here as well?

LecrisUT · 2023-08-11T11:13:01Z

/packit test

LecrisUT · 2023-08-25T08:15:52Z

@lukaszachy, @psss, @happz Review for this one please?

psss · 2023-09-19T07:55:24Z

@psss What about epel-8 though, can we stop providing updates as we did with tmt? epel-8 has even older macros ;)

Yes, I'd say we should keep fmf and tmt aligned, let's stop providing updates for el-8.

Filed #212 to drop support for el8.

psss · 2023-09-19T07:56:22Z

@lukaszachy, @psss, @happz Review for this one please?

Sorry, had a bunch of stuff on the plate so I did no get to this. Will have a look, hopefully this week. Sorry for the delay.

psss · 2023-09-29T18:55:25Z

Let's finish this as one of the first things for the 1.4 release.

LecrisUT · 2023-09-29T19:15:40Z

Oh, I need to rebase (after #215), need anything else than that?

psss · 2023-10-02T16:23:05Z

Yes, please rebase. No other changes needed right now (at least from the quick look through the changes).

.github/workflows/release.yml

.packit.yaml

pyproject.toml

fmf.spec

LecrisUT · 2024-06-05T12:21:55Z

@psss @lukaszachy @martinhoyer @happz What do you think of getting this in for 1.4, now that there has been quite some simplification to this PR. This PR is decoupled from most changes that require further discussion (e.g. #241), see the top-level comment for all changes in this PR. Right now it should not have any functional changes, other than the dynamic versioning, but @martinhoyer can you double-check it?

The current status is:

1 unresolved discussion about tests optional dependency. @martinhoyer can we move this for another PR issue?
setting up trusted publishing

Addressing #224 (40023ee) would be very useful before pushing and update, maybe I can cherry-pick that one out if needed.

martinhoyer · 2024-06-05T13:40:22Z

@LecrisUT Apologies, I'm afk for the rest of the week. Fwiw, I think it's in great state, but it might be a good idea to not rush it into 1.4 and instead maybe make 1.5 a dedicated release just for this? (thinking aloud)

psss · 2024-06-05T13:56:31Z

Thanks for putting this all together and for the nice summary! Had a quick look, sounds good, but it's not that little change and (also from the tmt experience and the packaging changes) I'd expect there will be a couple of bumps on the road. So let's not block 1.4 any more to get it out finally and let's try to finish this pull request among the first ones in 1.5.

Signed-off-by: Cristian Le <cristian.le@mpsd.mpg.de>

martinhoyer

Added a few comments. One more thing I wonder about is how to generate manpage - if it make sense to have it same as https://github.com/teemtee/tmt/blob/54dfd29ede1e3a20bad98bb6bf8c79dc0da25e54/pyproject.toml#L183

.github/workflows/release.yml

.packit.yaml

martinhoyer · 2024-08-16T09:10:44Z

.packit.yaml

@@ -5,15 +5,18 @@ synced_files:
 upstream_package_name: fmf
 downstream_package_name: fmf

+# Epel9 fails to build with dynamic version. Need to create archive with PKG-INFO
+# F37 works with setuptools_scm 7.0
 actions:
  create-archive:


We might need to add - hatch run docs:man (or however we'll generate man) before build, same as in tmt.
Would be cool to drop man completely, but it didn't fly in tmt, so assuming it won't fly here either.

It is included in the spec file

cp docs/header.txt man.rst tail -n+7 README.rst >> man.rst rst2man man.rst > fmf.1

For me I think sdist should be as vanilla as close as possible to the git archive. But regarding the man generation, we should see about if we want to change it in teemtee/tmt#3088.

We still have quite a few discrepancies as tmt has evolved, so how about handling these one step at a time.

It's tricky, because then you have discrepancies between rpm source and pypi source. It's a "hack" either way. No strong opinion from me.

Makefile

martinhoyer · 2024-08-16T09:15:58Z

fmf.spec

+BuildRequires: python3-devel
+BuildRequires: python3dist(docutils)
+BuildRequires: git-core
+Requires:      python3-fmf == %{version}-%{release}


I would suggest to do the same as in https://github.com/teemtee/tmt/blob/main/tmt.spec, i.e. %py_provides python3-fmf

This was discussed in other, now resolved thread.
@psss Would you agree that we should build fmf package, which would also provide python3-fmf, in the same fashion as tmt?

Do you remember why there wasn't a Obsoletes: python3-tmt?

fmf.spec

pyproject.toml

Signed-off-by: Cristian Le <cristian.le@mpsd.mpg.de>

martinhoyer

Looks ready to me, except for the 'test' subpackage.

Signed-off-by: Cristian Le <cristian.le@mpsd.mpg.de>

LecrisUT force-pushed the pep621 branch 3 times, most recently from 059f6d9 to 4d79f9c Compare August 9, 2023 18:21

LecrisUT force-pushed the pep621 branch from 9ca1a6c to 1311985 Compare August 10, 2023 08:25

LecrisUT mentioned this pull request Aug 10, 2023

Use Click for cli #205

Open

LecrisUT force-pushed the pep621 branch 3 times, most recently from 8ccf9b6 to 8ca3f85 Compare August 11, 2023 09:41

LecrisUT force-pushed the pep621 branch from 8ca3f85 to d06a337 Compare August 11, 2023 09:56

LecrisUT mentioned this pull request Aug 22, 2023

Add more File api functions to Tree #209

Open

psss added this to the 1.4 milestone Sep 29, 2023

LecrisUT force-pushed the pep621 branch from d06a337 to 8771cee Compare October 2, 2023 19:57

LecrisUT mentioned this pull request May 21, 2024

Add more Github CI workflow #239

Draft

LecrisUT force-pushed the pep621 branch from 56b103f to 58405c8 Compare May 21, 2024 08:46

LecrisUT mentioned this pull request May 21, 2024

Add ruff #240

Draft

martinhoyer reviewed May 21, 2024

View reviewed changes

.github/workflows/release.yml Show resolved Hide resolved

.packit.yaml Outdated Show resolved Hide resolved

pyproject.toml Show resolved Hide resolved

pyproject.toml Outdated Show resolved Hide resolved

fmf.spec Outdated Show resolved Hide resolved

fmf.spec Outdated Show resolved Hide resolved

LecrisUT force-pushed the pep621 branch from 58405c8 to 26fc2d3 Compare May 23, 2024 12:54

LecrisUT commented May 23, 2024

View reviewed changes

fmf.spec Show resolved Hide resolved

LecrisUT force-pushed the pep621 branch from 26fc2d3 to 4f56271 Compare May 23, 2024 13:09

LecrisUT mentioned this pull request May 23, 2024

Merge fmf and python3-fmf packages #241

Draft

LecrisUT force-pushed the pep621 branch from 4f56271 to c652542 Compare June 4, 2024 07:17

LecrisUT force-pushed the pep621 branch from c652542 to 6af5881 Compare June 17, 2024 07:52

LecrisUT added 4 commits June 17, 2024 20:37

Switch to PEP621 and hatchling

c439d61

Signed-off-by: Cristian Le <cristian.le@mpsd.mpg.de>

Improve spec file

ddf379e

Signed-off-by: Cristian Le <cristian.le@mpsd.mpg.de>

Improve release workflow

5c3bec3

Signed-off-by: Cristian Le <cristian.le@mpsd.mpg.de>

Temporary fix for cli entry_point + test_suite

d7f40ad

Signed-off-by: Cristian Le <cristian.le@mpsd.mpg.de>

LecrisUT force-pushed the pep621 branch from 6af5881 to c8cdf1a Compare June 17, 2024 18:38

martinhoyer reviewed Aug 16, 2024

View reviewed changes

LecrisUT and others added 5 commits August 18, 2024 22:06

Use hatch scripts

914de46

Signed-off-by: Cristian Le <cristian.le@mpsd.mpg.de>

Switch to dynamic version

1e433dc

Signed-off-by: Cristian Le <cristian.le@mpsd.mpg.de>

Remove pytest from fmf.spec

991d44d

Remove workflow_dispatch from release workflow

2ddc36f

Merge python-fmf with fmf

acd1415

LecrisUT force-pushed the pep621 branch 2 times, most recently from 5f38cb8 to b3bed5f Compare August 18, 2024 21:20

martinhoyer reviewed Aug 19, 2024

View reviewed changes

LecrisUT force-pushed the pep621 branch from b3bed5f to db72b11 Compare August 20, 2024 13:53

Drop epel9 workaround for packit

078c07a

Signed-off-by: Cristian Le <cristian.le@mpsd.mpg.de>

LecrisUT force-pushed the pep621 branch from db72b11 to 078c07a Compare August 20, 2024 16:48

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

PEP621 and `hatch` scripts #202

PEP621 and `hatch` scripts #202

LecrisUT commented Aug 9, 2023 •

edited

Loading

lukaszachy commented Aug 9, 2023

lukaszachy commented Aug 9, 2023

lukaszachy commented Aug 9, 2023

LecrisUT commented Aug 9, 2023

LecrisUT commented Aug 9, 2023

LecrisUT commented Aug 9, 2023

psss commented Aug 10, 2023

LecrisUT commented Aug 10, 2023

psss commented Aug 10, 2023

psss commented Aug 10, 2023

LecrisUT commented Aug 11, 2023 •

edited

Loading

LecrisUT commented Aug 11, 2023 •

edited

Loading

LecrisUT commented Aug 25, 2023

psss commented Sep 19, 2023

psss commented Sep 19, 2023

psss commented Sep 29, 2023

LecrisUT commented Sep 29, 2023 •

edited

Loading

psss commented Oct 2, 2023

LecrisUT commented Jun 5, 2024 •

edited

Loading

martinhoyer commented Jun 5, 2024

psss commented Jun 5, 2024

martinhoyer left a comment

martinhoyer Aug 16, 2024

LecrisUT Aug 16, 2024

martinhoyer Aug 16, 2024

martinhoyer Aug 16, 2024

martinhoyer Aug 16, 2024

LecrisUT Aug 18, 2024

martinhoyer left a comment

PEP621 and hatch scripts #202

Are you sure you want to change the base?

PEP621 and hatch scripts #202

Conversation

LecrisUT commented Aug 9, 2023 • edited Loading

lukaszachy commented Aug 9, 2023

lukaszachy commented Aug 9, 2023

lukaszachy commented Aug 9, 2023

LecrisUT commented Aug 9, 2023

LecrisUT commented Aug 9, 2023

LecrisUT commented Aug 9, 2023

psss commented Aug 10, 2023

LecrisUT commented Aug 10, 2023

psss commented Aug 10, 2023

psss commented Aug 10, 2023

LecrisUT commented Aug 11, 2023 • edited Loading

LecrisUT commented Aug 11, 2023 • edited Loading

LecrisUT commented Aug 25, 2023

psss commented Sep 19, 2023

psss commented Sep 19, 2023

psss commented Sep 29, 2023

LecrisUT commented Sep 29, 2023 • edited Loading

psss commented Oct 2, 2023

LecrisUT commented Jun 5, 2024 • edited Loading

martinhoyer commented Jun 5, 2024

psss commented Jun 5, 2024

martinhoyer left a comment

Choose a reason for hiding this comment

martinhoyer Aug 16, 2024

Choose a reason for hiding this comment

LecrisUT Aug 16, 2024

Choose a reason for hiding this comment

martinhoyer Aug 16, 2024

Choose a reason for hiding this comment

martinhoyer Aug 16, 2024

Choose a reason for hiding this comment

martinhoyer Aug 16, 2024

Choose a reason for hiding this comment

LecrisUT Aug 18, 2024

Choose a reason for hiding this comment

martinhoyer left a comment

Choose a reason for hiding this comment

PEP621 and `hatch` scripts #202

PEP621 and `hatch` scripts #202

LecrisUT commented Aug 9, 2023 •

edited

Loading

LecrisUT commented Aug 11, 2023 •

edited

Loading

LecrisUT commented Aug 11, 2023 •

edited

Loading

LecrisUT commented Sep 29, 2023 •

edited

Loading

LecrisUT commented Jun 5, 2024 •

edited

Loading