Burninate HyCons

[Kodiologist]

This is an idea that's been mentioned briefly in other contexts (e.g.) but hasn't gotten its own issue. I want to remove cons cells because:

1. They don't seem to provide anything you couldn't get with lists, tuples, quasiquoting, HyExpression, or the like and a helper function or two.
2. They're warty. To my knowledge, the below features are all intentional.
   1. Dotted lists use an infix syntax, unlike everything else.
   2. Dotted lists aren't legal at the top level.
   3. The HyCons constructor autoboxes, unlike all other Hy model constructors.
   4. cons may not actually return a cons cell. For example, (cons 1 [2 3 4]) returns ['1 2 3 4]. (The single quote on the first value is from wart #iii.)

From a deep dive into issues and PRs, I found two explanations for why we have HyCons:

1. "We need cons support to be called a Lisp." (Add `cons' support #183)
2. @algernon has found it useful for writing algernon/adderall and hylang/hydiomatic.

I disagree with (1) because we've successfully represented Lisp data structures with Python data structures and other Hy model objects like HyExpression and HyList.

As for (2), I'm not sure I understand the explanation or example, perhaps because I'm not familiar with these libraries, but it looks like `(~?fn ~?name ~?params . ~?body) could be written `(~fn ~?name ~?params ~@?body). Note that we've already broken support for these two libraries in other ways (neither has been updated since April 2016).

How do we feel about this?

[vodik]

I'd be okay with it, honestly I've never found a use for it.

[vodik]

To further expand on this, and I've been meaning to comment on/come back to #1545, but I the ~@... maps conceptually better to Python's generalized tuple unpacking support:

[#*(range 10) #*(range 10)]

And while I find this super useful, I think because of that, I tend to build my stuff around that style.

[Kodiologist]

It is perhaps silly that #* and ~@ do very similar things but look completely different.

[vodik]

Maybe it makes sense to change ~@ to ~*?

Maybe it even makes sense to have a ~**? unquote keywords? Does that even make sense?

I need to think about this. I got some thoughts on #1545 as a rebuttle to @gilch that I just haven't had a moment to get composed.

[gilch]

We had the ~@ first, and it comes from Clojure, which is equivalent to ,@ in Common Lisp. The #* we added later (to replace apply, which didn't work as well) and came from Python. But Python didn't even have generalized unpacking before 3.5. Not saying it has to be this way, but that's where it came from.

A big difference is that ~@ unquotes (which means it has to be in a quasiquote to begin with) and #* doesn't. On recent Python, you can do both in a macro, `[~@(range 10) #* ~(range 10)], but note that the first one gets unpacked at compile time, and the second at runtime. Another big difference is that ~@ can also unpack something into function position, but #* ~ can't. `(~@['print 1 2]) works fine, but (#* ~['print 1 2]) doesn't because #* isn't even allowed there. We can't simply unify them.

makes sense to have a ~**?

Not really? What would it do? You can already ~@ splice into a HyDict, because it's also just a list. The model represents the code that constructs a hashtable, but it's not a hastable itself. It's a Python list. Suppose we could have a ~@@ that would turn, say, `(foo ~@@{:bar 1 :baz 2}) into (foo :bar 1 :baz 2), but then why not put it in a list in the first place and just ~@? `(foo ~@[:bar 1 :baz 2]). Hy code is made of lists, not lists and dicts, even when the code builds dicts when evaluated. We only need ~@.

Also, while we're at it, &rest looks nothing like #* either. I actually suggested changing this #1411 .

[gilch]

Clojure doesn't even have dotted lists, that's from Common Lisp. You can do things with dots that you can't with splices like,

=> (print . (1 2))
1 2

I'm not sure how useful that is.

You can't splice it because it's not quoted.
(print ~@(1 2)) ; NameError: name 'unquote_splice' is not defined.

Because linked lists made of conses are Common Lisp's primary data structure, used for data as much as code, this capability is more important than for Clojure (or Python or Hy). The auto-boxing also makes it awkward to use Hy models as general-purpose data structures, because it will wrap things it shouldn't when used that way. But the extended unpacking means we don't need this so much either. E.g. instead of,
(cons 1 [2 3 4])
We can
[1 #* [2 3 4]].
And similarly, (print . (1 2)) becomes (print #* [1 2]). But in this case we're deferring it to runtime.

Clojure does not have dotted lists, but it does have cons, which is also a nice, simple function that we can use in combination with higher-order functions and term-rewriting macros like -> and friends. It's fairly easy to turn an expression into a function with #%. Maybe we should move that to core. It would mostly take care of the HOF complaint.

But, #*/#** are still kind of awkward in macros hylang/hyrule#52, both in the defmacro body and in later use in things like ->. This should be improved.

So I'm not all that opposed to removing the dotted-list syntax. Clojure does fine without it. I'm not satisfied with cons in its present form, and I'm not convinced that #* suffices as a replacement. Maybe we could remove it for now and add back a better version later when we figure out what we want, like we did with let. cons is so closely related to our choice of Hy models that the decision should probably wait for #1542.

[Kodiologist]

Sounds reasonable.

And similarly, (print . (1 2)) becomes (print #* [1 2]). But in this case we're deferring it to runtime.

You could write (defmacro m [] `(print ~@'[1 2])) (m), but there probably ought to be some variation on eval-and-compile to let you do that kind of thing without making you write a macro that you only use once.

[vodik]

Could we also treat (print #* [1 2]) as an optimization problem?

[Kodiologist]

What would that imply? Making the compiler automatically recognize and simplify such constructs? I'm not sure how often they're actually used.

[vodik]

Well, technically, something like (print #* [a b c]) should always be compilable as (print a b c). It gets more complicated when we're looking at something like (print #* foo), but we can shunt that to runtime.

[gilch]

I feel like we should leave that type of optimization to the Python runtime, to make our astor output predictable for debugging purposes. I'm not sure how much CPython would optimize that kind of thing, but PyPy probably would.

Hy's compiler should probably optimize the things that are incidental to Hy itself, like not doing the hy_anon_var_1 assignments when they're never used.

[gilch]

there probably ought to be some variation on eval-and-compile to let you do that kind of thing without making you write a macro that you only use once.

Well, we can (eval `(print ~@[1 2])), but that's still runtime.

It kind of sounds like macrolet. We could probably implement that as a macro without altering the compiler.

If you just want one-off splices, maybe a tag macro to eval at compile time would suffice.

=> (deftag eval [form] (eval form))
from hy.core.language import eval
import hy
hy.macros.tag('eval')(lambda form: eval(form))

<function <lambda> at 0x0000019D4FB92E18>
=> #eval `(print ~@[1 2])
print(1, 2)

1 2

I'm still not sure how useful this is.