Add #[DELIM[ … ]DELIM] syntax for string literals

[Kodiologist]

• Closes add heredocs to Hy #1042.
• Closes Alternative syntax for string literals #1287.
• Replaces Implement #qC string literals #1327.

To do:

• Documentation
• NEWS

[Kodiologist]

@gilch @kirbyfan64 Is there any reason you guys aren't reviewing this? I thought we settled on the Lua style.

[gilch]

I thought we settled on the Lua style.

I thought we did, or I did, at least. There are competing priorities, that's all. I can make this a priority since you think it's important.

This can come later, but we'll want to at least update hy-mode for the new syntax, and vim-hy too, but no-one seems to be maintaining that. Since hy-mode isn't settled even for what we've got so far, I've been de-prioritizing syntax changes that would break it compared to other things.

But even before then, both emacs and vim should mostly highlight properly if Hy files if we pick #["[...]"] as the delimiters. I'd also like to get a basic linter going for .hy files #1398. Parlinter looks promising, but it won't work with the new syntax generally, just with #["[, which it will recognize as a string inside []. (The other proposed string syntax options probably can't even do that much.) We're not the only Lisp with this kind of problem, so we can probably get that fixed. I'd even translate it to Python and do it myself eventually, but priorities.

[Kodiologist]

There are competing priorities, that's all. I can make this a priority since you think it's important.

My own philosophy is that I should try to review PRs (that are ready for review, to my knowledge) before anything else (like writing issues, working on my own PRs, etc.), because PR reviewing seems to be the primary bottleneck for Hy development.

[gilch]

=> (. '#[foo[bar]foo] hashstring_delim)
from hy import HyString
HyString('bar').hashstring_delim
Traceback (most recent call last):
  File "c:\users\me\documents\github\hylang-hy\hy\importer.py", line 201, in hy_eval
    return eval(ast_compile(expr, "<eval>", "eval"), namespace)
  File "<eval>", line 1, in <module>
AttributeError: 'HyString' object has no attribute 'hashstring_delim'

I can't seem to access the delimiter from the model. Is the compiler stripping it out somewhere?

[gilch]

Looks like the macros can get to it though.

=> (defmacro foo [s] (. s hashstring_delim))
import hy
hy.macros.macro('foo')((lambda s: s.hashstring_delim))
<function <lambda> at 0x000001BDD82476A8>
=> (foo #[x[]x])
'x'
'x'

[Kodiologist]

I think what you're seeing in the first example is a limitation in how quoting works. The generated code is HyString('bar'), which doesn't set the attribute. It seems clear that this isn't right, but I'm not sure what needs to be fixed.

[Kodiologist]

I think HyASTCompiler._render_quoted_form needs an upgrade to set the attribute. I'm looking into it now.

[Kodiologist]

Done.

[gilch]

Tested too, that looks good.

Seeing those elif isinstance statements in a super-long method makes me want to move that logic to methods in the individual Hy models themselves. It's another issue though. For now, it's more consistent this way. The refactor should be in #1399, or at least wait until after that's merged.

Now that the delimiter is a keyword argument, I think I should put it in the HyString repr in #1360, at least when it doesn't have the default value of None. (This will have to be merged first, so I can rebase.) But perhaps it should have a shorter name than hashstring_delim now? It's not kwonly, so I could just put it second positionally, but it might be clearer with the word.

We also need news and docs, which should also specify that some number of ='s are the preferred delimiters when #[[ won't do, and there's not a specific reason to use something else.

[Kodiologist]

Seeing those elif isinstance statements in a super-long method makes me want to move that logic to methods in the individual Hy models themselves. It's another issue though. For now, it's more consistent this way.

Agreed on both counts.

But perhaps it should have a shorter name than hashstring_delim now?

It doesn't make a difference to me. hashstring_delim just seemed like the most obvious choice. Is there any particular alternative you'd prefer?

We also need news and docs

Will do.

which should also specify that some number of ='s are the preferred delimiters when #[[ won't do, and there's not a specific reason to use something else.

I disagree. I don't want to count =s. Let's let the reader decide, or at least defer this debate until a style-guide PR.

[gilch]

I don't want to count =s.

I don't want to count more than three, but how often does that come up?

at least defer this debate until a style-guide PR

OK, I guess.

hashstring_delim just seemed like the most obvious choice. Is there any particular alternative you'd prefer?

I didn't have a particular alternative in mind, but I guess brackets could work.

@kirbyfan64, anything else you'd like to add? Do you have a preference for the HyString reprs?

[Kodiologist]

Done.

[Kodiologist]

@gilch This good? I wasn't sure whether you wanted me to change hashstring_delim to brackets or you were just thinking out loud.

[gilch]

Go ahead and change it to brackets.

[Kodiologist]

@gilch Done.

[gilch]

This is minor, but I'd prefer to avoid single-character names that can be confused with other characters in Python, like lI1 or 0O. These are distinguishable in a good programmer font, but you might have to squint. This is the kind of thing flake8 should be able to catch.

[gilch]

The implementation seems good now.

I don't like the name "hashstring" it makes it sound like it has something to do with hash functions, but it doesn't. It's just from the # character. We could call them "bracket strings", unless you have a better idea?

[Kodiologist]

"Bracket string" seems as good as anything. I will likely get around to the mild inconvenience of editing this tomorrow.

[Kodiologist]

Both changes made.

[refi64]

Glad to see this finally getting in!

[ekaschalk]

Bracket string literals are now fully supported in hy-mode master branch.