Support for auto-completion

[samuell]

Is there any way to get auto-completion support (via e.g. gocode)?

If not possible right now, I think it would be an awesome addition.

[dwhitena]

Agreed!

[srhopkins]

This would be awesome.

[graphaelli]

Spent some time on this -

master...graphaelli:autocomplete - would welcome a hand and/or some feedback if anyone is interested.

[dwhitena]

That's great @graphaelli! I'm excited to take a look. It will probably be towards the end of this week though. Feel free to ping me here anything.

[dwhitena]

Any more progress on this @graphaelli? Can we help in any particular way?

[graphaelli]

Hey @dwhitena. Sorry about the silence, life etc. The biggest problem right now is that all of the code for the completion needs to be in a single cell - is there a way to dump the current state of the code accumulated by the notebook? Once that's set the tests are in place but could use a review There are a bunch of style-type things to implement too, like adding () to completions that are functions.

[dwhitena]

@graphaelli now worries at all. I totally understand, and no pressure. Just making sure you weren't blocked on me for any reason.

Regarding dumping the code from the notebook. There was a way to do this before we switched to gomacro, but I don't think there is any more. Is that something that would help? I can open an issue. Also, I will put it on my list of things to get that review done. Thanks again!

[SpencerPark]

gomacro has WriteDeclsToStream which looks like what we need.

I haven't tried it and don't know if the format is what you are looking for but hopefully it is a good starting point to look into. The environment to call it on is available in the interpreter (see the code for handleExecuteRequest) and assuming things are aptly named I would think this should give you all the code that was previously executed in the notebook.

EDIT: We might need to change up doEval to make sure that the executed code gets collected.

[t-0-m-1-3]

@graphaelli hey if you need an extra set of hands to help out, more than willing to pound keys for you

[dwhitena]

That would be great @t-0-m-1-3! Any thought @graphaelli?

[graphaelli]

Please have at it, I haven't touched the branch in ages

[cosmos72]

While studying how to integrate gophernotes / gomacro with gocode, I found this thread: nsf/gocode#500
where gocode author discusses the problems due to go 1.10 build cache, that strongly reduces the need for "go install", which is instead required for gocode to work.
Despite being almost a de-facto standard, gocode author @nsf discouragingly stated, among other things:

---

The problem is that a proper tool should work with source files. Hence, there are two options for me:

1. Make gocode work with source files.
2. Rewrite it completely.

Problem: lack of motivation and free time. I definitely favor second option, but it requires way more time than the first one.

I don't know what else to say.

---

From the technical point of view, I checked what gocode needs: a source file with the cursor position where to perform autocompletion, plus "go install" of all packages to use. And using my gomacro WriteDeclsToStream seems appropriate.

Any taker?

The alternative would be to use some other code analysis and auto-completion tool.

For example, does anyone have updated information on guru autocompletion support?
Visual Studio Code tried to evaluate it 2 years ago, but at the time it seemed too early - see their issue #288

[graphaelli]

That's a bummer, I didn't know about the 1.10 cache affecting this. Last I heard about this topic on golang-dev was: https://groups.google.com/d/msg/golang-dev/mLsGwoFwqoI/BVSdX7HeBQAJ. Seems like sourcegraph is worth a shot? I've never used it.

[jparklab]

While looking at gocode, and I thought that we can do similar using gomacro parser, and spent some time writing a different version of auto-completion using gomacro.

master...jpark-dev:feature/autocomplete

Since it uses interp.Interp object used by the kernel, it just works for multiple cells

The current version works for AssignStmt, and ExprStmt. I can easily add support for other statements
(but, it'd be nice if we upgrade gomacro version in vendor directory to the latest one, and I patched gophernote locally to use the latest gomacro. I need to back-port the code to make it work with the version in vendor directory..)

[cosmos72]

The same feature was requested for plain gomacro (see cosmos72/gomacro#21) so I recently added a method gomacro/fast/Interp.CompleteWords() upstream.
It implements auto-completion in text terminals by plugging into line-editing library https://github.com/peterh/liner.
It returns possible completions in most (not all) cases - if I have time, I will try to merge it into gopherdata in the next days and prepare a pull request.

Compared to master...jpark-dev:feature/autocomplete, my Interp.CompleteWords() has the advantage of being side-effects-free: @jpark-dev used Interp.RunExpr() for simplicity in his implementation, which unluckily means there are cases where typing things like

{ LaunchNuclearMissiles();

then pressing TAB at the end of the input will have unwanted side effects, i.e. it will execute (at least some of) the code already present in the Jupyter cell.

From the bright side, his implementation is already integrated with Jupyter: a very useful addition - especially from my point of view, as I am not an expert of Jupyter protocol.

[cosmos72]

My stab at auto-completion is the branch https://github.com/cosmos72/gophernotes/tree/autocomplete and corresponding pull request #111 - shamelessly reuses @jpark-dev integration with Jupyter.

I am interested in feedback on how it performs.

To @jpark-dev (and others, if they wish): a comparison with your https://github.com/jpark-dev/gophernotes/tree/feature/autocomplete_new_gomacro would be very interesting.

Screenshots:

[jparklab]

@cosmos72 It works great! As you pointed out, my implenetation executes the expressions, and I was not a big fan of it. Although I didn't anyone would launch nuclear missiles, but evaluating long-running expression is not good. (I didn't have a good idea how to do auto-completion without evaluting, and kinda hoped you can add that feature, and it happened! :))
I'd be nice to have completion for function calls(especially since the library I am playing with uses nested calls), but I will happily use work arounds.

BTW, jupyter notebook kinda supports auto completion for local varaibles

(I think the keyword in blue is automatically added by jupyter itself..)

[cosmos72]

Good to know, thanks!

Well, the "launch nuclear missile" was an extreme example of side effect :)

I wrote it to point out that, in my opinion, executing an expression when pressing TAB for auto-completion would be unexpected and confusing to programmers.

About auto-completion for local variables, chained function calls and other "complicated" expressions: I have some ideas on how to implement it. Not completely trivial, as it involves at least a partial type-checking, which gomacro currently performs only as a side effect of Interp.Compile().

But I don't want to call Interp.Compile() during auto-completion, because it has side-effects too: it partially executes declarations, in the sense that declares the types, constants, variables, function or method found in a declaration (without running the initialization expressions, which is performed by Interp.RunExpr())

[cosmos72]

added in commit 886f25f