EDA: Remove old vs-code states

[SmiteDeluxe]

Is your feature request related to a problem?

Currently the global vs-code state fills up over time making certain operations slow.

Desired solution

The goal is to save the Date of last update along with the State in the global state. Then in the findCurrentState method where those states are iterated once on a new panel startup anyway, the AST should be searched for the placeholder and pipeline name of the old states and in case it does not find that combination AND the states last update is more than a day ago, it should remove that state from the global state.

Possible alternatives (optional)

Just remove if older than 7 days and do not care about the placeholder existence.

Screenshots (optional)

No response

Additional Context (optional)

No response

[SmiteDeluxe]

@lars-reimann do you know how to best search for a placeholder in the AST given the pipeline name and placeholder name?

[lars-reimann]

Do you also know the package that the pipeline is contained in?

[SmiteDeluxe]

Not particularly, I mean I could search for it I reckon in some way in the AST but it is not info I have at the moment. @lars-reimann

[lars-reimann]

OK, the most robust way requires three pieces of information:

1. The URI of the document that contains the pipeline.
2. The name of the pipeline.
3. The name of the placeholder.

Then

1. Get the document by URI: const document = services.shared.workspace.LangiumDocuments.getDocument(uri)
2. Get the module: const module = <SdsModule>document.parseResult.value
3. Find the pipeline with the given name: const pipeline module.getModuleMembers().find(it => isSdsPipeline(it) && it.name === ...
4. Find the placeholder in the pipeline: const placeholder = getPlaceholderByName(pipeline.body, ...)

[lars-reimann]

@SmiteDeluxe After trying the EDA for some time, I'd suggest removing the interaction with the globalState to persist panels altogether. While data that is loaded from a file may rarely change, this is not true for derived data. This may change after any change to the program.

Example:

1. Open an EDA panel for filtered:

pipeline myPipeline {
    val titanic = Table.fromCsvFile("C:/Users/Lars/OneDrive/Desktop/test/titanic.csv");
    val filtered = titanic.filterRows((param1) ->
        param1.getValue("age") as Float >= 10
    );
}

2. Change the program:

pipeline myPipeline {
    val titanic = Table.fromCsvFile("C:/Users/Lars/OneDrive/Desktop/test/titanic.csv");
    val filtered = titanic.filterRows((param1) ->
        param1.getValue("age") as Float >= 60
    );
}

3. Open the EDA panel for filtered again. Now the data is outdated. Automatically refreshing outdated data without user interaction is exactly what we try to improve over the cells of notebooks.

While the runner is open in the background, it already takes care of caching for us. We should not duplicate this functionality again, trading some reduction in IPC for increased memory use. Using windowed requests, we can also limit the time needed.

If we want to persist state after the runner is closed, this should be handled by the runner, not here.

[SmiteDeluxe]

I see the value in that, yes @lars-reimann. Of course the caching here is not just that what the Runner is doing or should be doing as it is a lot about the action history, but I agree on that it can be a bit counter intuitive. By not having the global state we are taking away some efficiency and usability in one area and adding safety and usability in another aspect. Although the gain in usability might be of more value here as you say, maybe not for new users that have simple pipelines and work slowly but certainly for those that have big pipelines that change a lot.

[lars-reimann]

Let's talk about that tomorrow.

[lars-reimann]

🎉 This issue has been resolved in version 0.16.0 🎉

The release is available on:

• npm package (@latest dist-tag)
• npm package (@latest dist-tag)
• v0.16.0
• GitHub release

Your semantic-release bot 📦🚀