Add a page about different notions of version

[zliu41]

This replaces the outdated "Plutus language versions" page, which was written 2 years ago.

Reviewers: ignore the diff outside of versions.md.

[bezirg]

This slightly contradicts the earlier:

There is no need (and nowhere) to specify whether you are writing a Plutus V1, or Plutus V2, or Plutus V3 program

Since indirectly the ledger language version is linked to a builtin-semantics-variant, which is observable even when you write/run offline programs.

[bezirg]

Suggestion to sth like: "omitting the flag defaults to latest/newest builtin semantics variant for each builtin"

[zliu41]

I don't see a contradiction. When evaluating a standlone program, you don't specify the ledger language version (since there's no ledger), but you can specify the builtin semantics variant. The fact that a particular ledger language version uses a particular builtin semantics variant doesn't contradict anything; it just happens to be that way. It's just a random correlation.

[effectfully]

It's just a random correlation.

That's a weird thing to say. The builtin semantics variant is solely determined by the protocol version and the ledger language version, it's basically like a unique hash of the two. I'm against viewing it as some abstract concept that just happens to be there (and I'm against its very existence in the first place).

[bezirg]

LGTM

[bezirg]

Stylistic suggestion: having all versions (e.g. PlutusV1.., 1.0.0.) inside markdown `code` , PlutusV1, 1.0.0 etc

[bezirg]

Maybe you should explain also what protocol version is then?

[zliu41]

Protocol version has nothing to do with Plutus. I will explain such concepts in a glossary page.

[effectfully]

Depending on the protocol version or the ledger language version

Might be worth adding "or both" explicitly.

[bezirg]

Suggestion:

[Ledger Language Version](#ledger-language-version)

(i don't know if this is the right way to link in markdown)

[zliu41]

I don't find it a good reader experience to put a link in a page that points to a different part of the same page, unless that page is really long.

[ana-pantilie]

I think we should add a link to the documentation available here: https://github.com/IntersectMBO/cardano-ledger?tab=readme-ov-file#cardano-ledger. I found the specs linked there to be very valuable in understanding the EUTXO model.

[ana-pantilie]

These pages don't exist. Will they be added or were they removed/modified and we forgot to remove/modify the references?

[zliu41]

It does exist: https://plutus.cardano.intersectmbo.org/docs/category/working-with-scripts

[ana-pantilie]

Clicking on the links in the rich diff sent me to a page displaying:

404 - page not found
The 
zliu41/doc-version

 branch of 
plutus

 does not contain the path 
doc/docusaurus/docs/category/using-plutus-tx.

[zliu41]

It does work on the doc site, which appears to uses a different path finding mechanism than GH.

[ana-pantilie]

Does this application infrastructure exist anymore? This is about what Plutus Tools used to maintain, right? If yes, maybe we should remove this page entirely.

[zliu41]

Don't bother reviewing anything other than versions.md. A lot of them will be removed soon.

[ana-pantilie]

Plinth instead of Plutus Tx? Or should we transition everything in bulk when the renaming is made official?

[zliu41]

I'll do the renaming in one fell swoop.

[ana-pantilie]

I believe I understand what this is trying to say but I also think it's a bit ambiguous. It relies on "language in the sense of a programming language" having a clear definition when one is not given here and it's also not common knowledge what it might be.

How can we define what a "language version" is?
Would it make sense to say that the versioning PlutusV1 ... PlutusVn refers to the version of the UPLC evaluator? I believe that's not enough, because if V2 added new constructors to the UPLC AST then it also extended the syntax.
So is it a new UPLC language? Where the syntax can only be extended from Vn to Vn+1 and the semantics can be changed arbitrarily? Or can only the "builtin" semantics be changed arbitrarily? It's not clear to me what constraints we have here, and it would be very good to specify them.

All of these are independent of the ledger, so I'm not so sure of the "ledger language version" naming. I would say that yes the ledger interacts with each language version differently, but the "languages" themselves are different in isolation of the ledger. And I believe we should (or, more appropriately, the ledger should) specify how it "consumes" each language version, since from "Depending on the ledger language version, the ledger will pass different arguments to the validator." I understand that the API differs from one version to the other.

[zliu41]

Would it make sense to say that the versioning PlutusV1 ... PlutusVn refers to the version of the UPLC evaluator?

No, PlutusVn has nothing whatsoever to do with the UPLC evaluator.

but the "languages" themselves are different in isolation of the ledger.

No, the languages themselves are not different, which is the entire point I'm trying to make here. How are they different in isolation of the ledger?

[ana-pantilie]

No, the languages themselves are not different, which is the entire point I'm trying to make here. How are they different in isolation of the ledger?

PlutusV2 introduces new constructors to the UPLC language, right? What does this have to do with the ledger?

[effectfully]

No, the languages themselves are not different, which is the entire point I'm trying to make here. How are they different in isolation of the ledger?

I agree with @ana-pantilie on

but the "languages" themselves are different in isolation of the ledger.

The ledger language version (e.g. PlutusV1) literally determines the AST version (1.0.0 for PlutusV1) and possible semantics variants (DefaultFunSemanticsVariantA and DefaultFunSemanticsVariantsB for PlutusV1). The AST version is as close to a "programming language version" as it gets and the semantics variant literally affects the evaluation result of certain programs. The "languages" are very much different in isolation of the ledger (which only supplies us with a protocol version so that we can pick the right semantics variant, which preexists in UPLC and determines the behavior of a program) and in my opinion the fact that we don't recognize it is merely a bug in the code.

You could make a point that the whole concept of "languages" does not exist without the ledger, which would probably make some sense, given that we define that concept in the ledger API, but I don't think it makes any sense to talk about languages not being different in isolation of the ledger, because they either are different or the whole subject is moot.

Personally, I find the concept of semantics variants harmful (1 and 2) and as such I find it necessary to propagate the ledger language version into plutus-core (or whatever we'll end up having after we extricate Cardano-specific builtins from plutus-core), as well as the protocol version, hence for me the whole discussion about semantics variants is pointless, that concept should not exist at all. And once we recognize that, the ledger language version will be there in the innards of plutus-core making it a pretty regular language version.

[zliu41]

PlutusV2 introduces new constructors to the UPLC language, right? What does this have to do with the ledger?

No it doesn't. New constructors were introduced in a new Plutus Core Version (1.1.0), not in PlutusV2.

[zliu41]

The ledger language version (e.g. PlutusV1) literally determines the AST version (1.0.0 for PlutusV1)

Again, this will no longer be true in the near future. There is no fundamental reason why 1.1.0 cannot be used for PlutusV1.

[ana-pantilie]

So extending the semantics of UPLC does not warrant a new language version. Would it be enough to say that any changes to UPLC which are not backwards compatible create a new language version? Then again, why did we need a new language version for SoPs? Doesn't that only extend the language? Or does it also make some backwards-incompatible changes?

[zliu41]

Would it be enough to say that any changes to UPLC which are not backwards compatible create a new language version?

What do you mean by "language version"? I'm using the terms "ledger language version" and "Plutus Core version" here. Any changes to UPLC itself, whether backwards compatible or not, creates a new Plutus Core version, but does not create a new ledger language version.

[ana-pantilie]

So what is the exact relationship between "ledger language" versions and "Plutus Core" versions?

[zliu41]

A new Plutus Core version changes the languages - e.g., Plutus Core 1.1.0 introduces Case and Constr.

A new ledger language version does not change the language. The main difference is that the scripts will receive different arguments from the ledger.

[ana-pantilie]

I understand. I think it would be very helpful to add this to the document as an example. Thank you!

[zliu41]

The paragraph above does say "1.1.0 adds sums-of-products to the language by introducing two new AST node types: Case and Constr.".

[zliu41]

Stylistic suggestion: having all versions (e.g. PlutusV1.., 1.0.0.) inside markdown code , PlutusV1, 1.0.0 etc

I disagree - version numbers are not code, and I think most people don't put them in code blocks.

[zliu41]

Don't bother reviewing anything other than versions.md. A lot of them will be removed soon.

[zliu41]

I don't find it a good reader experience to put a link in a page that points to a different part of the same page, unless that page is really long.

[effectfully]

I disagree with this description a little, because I disagree with the current code structure a lot.

[effectfully]

I mean, they are different versions of the programming language, they have different features available, program may evaluate differently, costs are completely different etc.

[zliu41]

None of these are fundamentally why have multiple ledger language versions. E.g., currently they do have different features (different builtins, scott vs. sop), but in the near future those differences will go away. Same with the difference in costing.

The only real difference between V1, V2, V3: they receive different arguments from the ledger.

[effectfully]

I really do not agree with this apart from the "nowhere to specify" part, which in my opinion is nothing but a bug.

[zliu41]

As I said in my other comment - in the near future all other differences between V1, V2 and V3 will go away, and the only difference left would be: they receive different arguments from the ledger. There is no fundamental reason for there to be different programming language features among V1/V2/V3.

[effectfully]

As I said in my other comment - in the near future all other differences between V1, V2 and V3 will go away, and the only difference left would be: they receive different arguments from the ledger.

How can this possibly be true? We compute the semantics variant from the ledger language version (and the protocol version), it cannot be true that the only difference is in arguments, since the semantics variant affects the behavior of programs. The semantics variant is a derivative and in my view is a vestige of the past. It's the ledger language version and the protocol version that are real, the semantics variant is just a made up hash of the two.

In particular, V1 (or V2) and V3 behave differently for certain programs with a fixed protocol version. How is that not a difference? Sure we arrange that through the semantics variant, but that's just a detail of the implementation and not at all how we actually look at versioning ("we'll change this behavior in V3").

[zliu41]

The Plutus Core language can exist as a standalone programming language, without the node, ledger, or anything Cardano. It is turing complete and you can use it to do anything. For that reason, builtin semantics variant should be the first-class concept. When we talk about the language without Cardano, there is no need to introduce the concept of PlutusV1, V2, V3 etc.

The overwhelmingly main reason to introduce a new PlutusVx is due to a new ledger era, necessitating changes to the script context or other script arguments or return types. Admittedly, there's another potential reason to introduce a new PlutusVx, without a new ledger era, namely changing the behavior of a builtin function. But crucially, the only reason a new PlutusVx is needed in this case, rather than just changing the behavior at the next Protocol Version, is still Cardano-related: because of the nature of the blockchain, we want scripts that are already submitted to continue to have the old behavior, thus new PlutusVx, rather than new Protocol Version.

Without Cardano in the picture, not only would we not need Plutus V1/V2/V3, but we certainly wouldn't name them Plutus V1/V2/V3. How is it a better name than Builtin Semantics Variant V1/V2/V3? The latter is much more specific and accurate, and less confusing.

[zliu41]

btw one more point: the fact that the ledger language version and protocol version must be used together to determine the behaviors of builtin functions is another reason why the ledger language version is irrelevant when we are talking about the language itself, independent of Cardano: protocol versions only exist in Cardano, and without protocol version, the ledger language version does not even determine anything by itself.

[effectfully]

I agree with everything you've said apart from

Without Cardano in the picture, not only would we not need Plutus V1/V2/V3, but we certainly wouldn't name them Plutus V1/V2/V3. How is it a better name than Builtin Semantics Variant V1/V2/V3? The latter is much more specific and accurate, and less confusing.

(as I'm arguing in those threads that I've referenced earlier, making the semantics variant in Cardano a number/letter is a bad idea that we or our successors will very likely backtrack on. We think of it as a product type and it really should be a product).

I just do not find your point of view helpful for users, i.e. those people for whom these docs are written for. We're talking about Cardano here, why should we generalize anything beyond that?

without protocol version, the ledger language version does not even determine anything by itself.

Technically,

1. PlutusV3 solely determines the semantics variant
2. for earlier ledger language versions they determine the set of available semantics variant

But I see your point.

For that reason, builtin semantics variant should be the first-class concept.

I agree. I don't think it's helpful for these docs though. In Cardano the semantics variant is a combination of the ledger language version and the protocol version, nothing more than that. Hashing them into that A/B/C thing is just counter-productive and so is saying things like

When simply writing a standalone program, say one that takes two integers and returns their sum, or one that implements a Sudoku solver, the notion of ledger language version is completely irrelevant.

Replace addInteger in that example with consByteString (or even multiplyInteger!) and it simply falls apart. You're presuming that there's an addInteger operation, a very much Cardano-specific one, and then you're saying that the ledger language is completely irrelevant, which is simply not true, because it is relevant for the behavior of builtins. There's no Sudoku solver if you don't know how your builtins are supposed to behave, it's all one giant undefined behavior agent in that case.

[zliu41]

I just do not find your point of view helpful for users, i.e. those people for whom these docs are written for. We're talking about Cardano here, why should we generalize anything beyond that?

I find it easier to think about this in terms of two layers: (1) what is the language itself and the notions related only to the language; and then (2) how it works with Cardano, which is in principle just one of the applications of Plutus Core. Mixing them together makes it harder to untangle the different concepts.

Replace addInteger in that example with consByteString (or even multiplyInteger!) and it simply falls apart.

With consByteString, the ledger language version is still irrelevant in the sense that... it simply isn't a thing when talking about the language itself (layer 1). But I see your point: when running the script on Cardano, different ledger language versions do lead to different behaviors. My argument still holds but I could've phrase it more accurately.

[effectfully]

I find it easier to think about this in terms of two layers: (1) what is the language itself and the notions related only to the language; and then (2) how it works with Cardano, which is in principle just one of the applications of Plutus Core.

I agree that thinking in terms of two layers is best for us. I really disagree that it's helpful to users. In particular, ironically

Mixing them together makes it harder to untangle the different concepts.

mixing things up is more likely when you try to untangle them. This shows w.r.t. my point about consByteString, but also more generally when you say things like "when simply writing a standalone program". There's no such thing as a program written in some abstract Plutus, it's always some concrete form of Plutus and in this case it's Cardano Plutus (I'm not even sure if other Plutuses exist). Plutus-the-library is a language constructor: you can have your own builtins, your own versioning for them, your own versioning for the AST etc. But the moment you start "writing a program", these become fixed and you can no longer untangle it from its context, i.e. Cardano in our case.

This also shows in

You simply write your code in Plutus Tx, compile it to UPLC, and run it with a UPLC evaluator.

which, as I said earlier, is merely a bug in the implementation.

So no, I do not agree with

My argument still holds

and I most certainly do not agree with

I could've phrase it more accurately.

because these are supposed to be user-friendly docs, not the kind of docs that we write for ourselves with all the gory details and whatnot. All that phrased correctly context is going to be lost on folks who just want to know how to develop smart contracts. E.g. they will just think that the behavior of Plinth scripts is always the same regardless of the ledger language version, which is very much not the case.

Summarizing:

1. I believe that the ledger language version (likely it's "condensed" form) should be an explicit (rather than implicit) part of the semantics variant
2. Plinth users, i.e. Cardano users, should be able to control the ledger language version that they're using for their Plinth scripts right in those scripts
3. while the two-layered model that we've been discussing here is correct, it's counter-productive to put it in these docs as it adds a lot of unnecessary confusion. I really don't feel like this level of detail belongs to an "essential concepts" section

I think one way forward would be to ask the smart contract developers community to read the docs and tell us whether they find them helpful and what their takeaways are.

[zliu41]

I really disagree that it's helpful to users

I guess we'll agree to disagree. I definitely find discussing different things separately useful, whether I'm the developer or the user. Not dissimilar to modularity of code. Plus, this is something we are already doing: the documentation has a section talking about "using Plutus Tx", i.e., just language stuff, and another section for "working with scripts", i.e., Cardano-related. I think it's much better than merging them together - I fail to see how the latter improves readability.

All that phrased correctly context is going to be lost on folks who just want to know how to develop smart contracts

It's IMO fairly straightforward: when you are writing a standlone program, you can use builtin semantics variant to determine the builtin behavior. When you are writing a script, this is determined by (protocol version, ledger language version). Not that hard!

[effectfully]

No, the languages themselves are not different, which is the entire point I'm trying to make here. How are they different in isolation of the ledger?

I agree with @ana-pantilie on

but the "languages" themselves are different in isolation of the ledger.

The ledger language version (e.g. PlutusV1) literally determines the AST version (1.0.0 for PlutusV1) and possible semantics variants (DefaultFunSemanticsVariantA and DefaultFunSemanticsVariantsB for PlutusV1). The AST version is as close to a "programming language version" as it gets and the semantics variant literally affects the evaluation result of certain programs. The "languages" are very much different in isolation of the ledger (which only supplies us with a protocol version so that we can pick the right semantics variant, which preexists in UPLC and determines the behavior of a program) and in my opinion the fact that we don't recognize it is merely a bug in the code.

You could make a point that the whole concept of "languages" does not exist without the ledger, which would probably make some sense, given that we define that concept in the ledger API, but I don't think it makes any sense to talk about languages not being different in isolation of the ledger, because they either are different or the whole subject is moot.

Personally, I find the concept of semantics variants harmful (1 and 2) and as such I find it necessary to propagate the ledger language version into plutus-core (or whatever we'll end up having after we extricate Cardano-specific builtins from plutus-core), as well as the protocol version, hence for me the whole discussion about semantics variants is pointless, that concept should not exist at all. And once we recognize that, the ledger language version will be there in the innards of plutus-core making it a pretty regular language version.

[effectfully]

Please let's call it AST version. Having "Plutus Core version" and "plutus-core version" to mean different things is kind of horrible.

[zliu41]

I think AST version is fine, but it is unusual - it is common for a programming language to have a version - e.g. Python 3.7

[effectfully]

Python 3.7 corresponds to plutus-core-1.32, not 1.1.0.

[effectfully]

Depending on the protocol version or the ledger language version

Might be worth adding "or both" explicitly.

[effectfully]

It's just a random correlation.

That's a weird thing to say. The builtin semantics variant is solely determined by the protocol version and the ledger language version, it's basically like a unique hash of the two. I'm against viewing it as some abstract concept that just happens to be there (and I'm against its very existence in the first place).