Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kwxm/metatheory/fix test2 #6414

Merged
merged 31 commits into from
Aug 16, 2024
Merged

Kwxm/metatheory/fix test2 #6414

merged 31 commits into from
Aug 16, 2024

Conversation

kwxm
Copy link
Contributor

@kwxm kwxm commented Aug 14, 2024

plutus-metatheory:test2 was disabled in CI and would fail if you ran it locally. This should fix the tests, after a fashion. Those tests (now renamed to test-detailed) run various Agda and Haskell evaluators on some examples and check that the evaluators agree. The problem was that the test was attempting to re-parse the output and compare ASTs, but the Agda evaluators can only output textual PLC/UPLC with names (often just A or something similar) annotated with de Bruijn indices, and the parsers can't handle those. To work around this I've modified the plc and uplc evaluators to be able to output textual code with de Bruijn indices as well and modified the tests to compare canonicalised versions of the textual output (squashing up all the whitespace and replacing variables with just their de Bruijn indices). This is kind of horrible, but it does work and saves us from having to worry about alpha equivalence.To fix it properly I think we'd either have to make the Haskell parser accept de Bruijn indices or get the Agda code to output unique names, and neither of these seem especially easy.

I've also rewritten the tests to use Tasty.HUnit, which we use quite a lot elsewhere: previously they were using something called detailed-0.9, which seems to be an obscure type of test suite that cabal knows about. I don't really know what that is, but I've been able to replace it with exitcode-stdio-1.0, which we use everywhere else.

I messed about with project.nix to try to re-enable the tests in CI, but I'm not sure if I've done that correctly.

I also deleted some files in the top level of plutus-metatheory/test that appear to be unused.

@kwxm kwxm added Test Metatheory No Changelog Required Add this to skip the Changelog Check labels Aug 14, 2024
deps = cmp.executableToolDepends;
in
''PATH=${lib.makeBinPath deps}:$PATH'';

# FIXME: Somehow this is broken even with setting the path up as above
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zeme-wana I think this is what I have to remove to re-enable these tests in CI, but that's a guess. I'm also not sure what the stuff above about preCheck is for or whether we still need it.

Copy link
Contributor

@zeme-wana zeme-wana Aug 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, the test2->test-detailed came back and is passing.
As for the preCheck, I removed it and ci still passes, so it was probably obsolete.

@kwxm
Copy link
Contributor Author

kwxm commented Aug 14, 2024

This is issue #6406.

@kwxm kwxm mentioned this pull request Aug 14, 2024
@ana-pantilie
Copy link
Contributor

ana-pantilie commented Aug 15, 2024

This is issue #6406.

@kwxm you can write "Fixes <link-to-issue/#issueno>" to automatically close the issue when the PR gets merged

@kwxm
Copy link
Contributor Author

kwxm commented Aug 15, 2024

This is issue #6406.

@kwxm you can write "Fixes <link-to-issue/#issueno>" to automatically close the issue when the PR gets merged

Ah, thanks. I wondered if there was some way to make a PR and an issue interact sensibly.

Copy link
Contributor

@ana-pantilie ana-pantilie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool, thank you for fixing this! I only have some minor comments, feel free to ignore.

Comment on lines 187 to 191
if printModeDeBruijn
then
let w = toDeBruijnTypePLC ty
in print (getPrintMethod printMode w) >> exitSuccess
else print (getPrintMethod printMode ty) >> exitSuccess
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does something like this make sense? Makes it a bit cleaner imo, but feel free to ignore if you don't agree/it doesn't work.

Suggested change
if printModeDeBruijn
then
let w = toDeBruijnTypePLC ty
in print (getPrintMethod printMode w) >> exitSuccess
else print (getPrintMethod printMode ty) >> exitSuccess
let transformNames = if printModeDeBruijn then toDeBruijnTypePLC else id
in print (getPrintMethod printMode (transformNames ty)) >> exitSuccess

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately that doesn't work because ty and toDeBruijnTy ty have different types (the type of PLC types is parametric over the type of names, and toDebruijnType changes that). I think we'd have to introduce a new type class to get round that.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have tried to tidy it up a bit though.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, thank you!

plutus-core/executables/plc/Main.hs Outdated Show resolved Hide resolved
@@ -104,7 +104,7 @@ printmode = option auto
<> showDefault
<> help
("Print mode for textual output (ignored elsewhere): Classic -> plcPrettyClassicDef, "
<> "Debug -> plcPrettyClassicDebug, "
<> "Simple -> plcPrettyClassicSimple, "
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why the renaming?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The names changed in the prettyprinting code a while ago and plcPrettyClassicDebug no longer exists but the help message didn't get updated.

@@ -67,6 +68,7 @@ data EvalOptions =
Input
Format
PrintMode
Bool -- Use de Bruijn indices in the output?
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I personally think it's better to write a new datatype with 2 constructors something like ShowNames = UseDeBruijn | UseNames to avoid "boolean blindness"; it makes it possible to figure out from just the type/constructors what semantics they have, if you use booleans you have to add comments to maintain readability.

Copy link
Contributor Author

@kwxm kwxm Aug 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You caught me! I did feel guilty about not doing that but I was too lazy to change it, partly because I would have had to go and look up the right way to deal with the command-line options. I'll change it.

plutus-core/executables/uplc/Main.hs Outdated Show resolved Hide resolved
Copy link
Contributor

@ramsay-t ramsay-t left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me - handling the parsing of the weird de brujin variables seems ok now, but we might want to figure out where they are coming from eventually.

@kwxm kwxm merged commit 6b6877f into master Aug 16, 2024
6 checks passed
@kwxm kwxm deleted the kwxm/metatheory/fix-test2 branch August 16, 2024 19:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Metatheory No Changelog Required Add this to skip the Changelog Check Test
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants