is it possible to add new xtr. functions using java ?

[sandys]

hi
we were looking to add a few functions to xtrasonnet xtr. namespace...but wanted to do it through java. is that possible ?

[jam01]

Yes, I'll add some docs on how to do that, but here's an example that might help

https://github.com/jam01/xtrasonnet/blob/0.5.4/xtrasonnet/src/test/java/io/github/jam01/xtrasonnet/spi/LibraryTest.java

[sandys]

this is awesome! one followup question - ive been tracking the recent pull of sjsonnet databricks/sjsonnet#166

it theoretically allows the end developer to extend and add functions to std namespace at development time (without recompiling the sjsonnet library).
Would something like that be possible in xtrasonnet ?

[jam01]

I hadn't seen this PR. There's been some activity in sjsonnet after a long while. I'll rebase my sjsonnet fork and will release a new ver if everything's good.

[sandys]

appreciate it! your fork is spectacular because it adds these extra functions in here.
It would be super great if there was a provision for user defined functions as well (maybe in a different namespace "usr").

[jam01]

I'll rebase the fork, but maybe I'm not exactly following...

The library method I linked above should do what you're looking for. You can write a Java function and call it from your xtrasonnet code under any namespace you define

[sandys]

@jam01 thanks for looking into this. really appreciate it!

so we tried, but it keeps throwing an error. We tried to stick as close to your example as possible. is there an example that shows this ?

[jam01]

can you share the error stack trace?

[sandys]

i finally got it to work by doing this. its hacky, but was trying to fix for a few hours - so please forgive the bad code.

import com.fasterxml.jackson.databind.ObjectMapper;
import io.github.jam01.xtrasonnet.Transformer;
import org.apache.commons.io.FileUtils;
import org.apache.commons.lang.RandomStringUtils;
import org.json.JSONArray;
import org.json.JSONObject;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import java.io.*;
import java.util.*;
public abstract class JsonnetLoader {

  private final Logger logger = LoggerFactory.getLogger(this.getClass());

  private Map<String, JsonnetArgs> args = new HashMap<>();
  private Map<String, String> xtraArgsMap = new HashMap<>();
  private JSONObject jsonObject;

  public JsonnetLoader() {}

  public void load(InputStream inputStream) {
    try {
      preconfigured();

      // Create Temp File With Unique Name
      String filename =
          RandomStringUtils.randomAlphanumeric(12) + "_" + System.currentTimeMillis() + ".jsonnet";
      File file = new File(System.getProperty("java.io.tmpdir") + File.separator + filename);

      BufferedReader br = new BufferedReader(new InputStreamReader(inputStream));

      StringBuilder sb = new StringBuilder();
      String line;
      while ((line = br.readLine()) != null) {
        sb.append(line).append(System.lineSeparator());
      }

      String text = sb.toString().replaceAll("[\r]+", "");

      PrintWriter printWriter = new PrintWriter(file);
      printWriter.write(text);
      printWriter.flush();
      printWriter.close();



      // Transform Jsonnet Args
      for (Map.Entry<String, JsonnetArgs> entry : this.args.entrySet()) {

        if (entry.getValue().getDataType().equals(DataType.STRING)) {
          String regex = "[^\\p{L}\\p{N}\\p{P}\\p{Z}]";

          xtraArgsMap.put(entry.getKey(), entry.getValue().getVal().replaceAll(regex, ""));

        } else if (entry.getValue().getDataType().equals(DataType.INTEGER)
            || entry.getValue().getDataType().equals(DataType.BOOLEAN)) {

          xtraArgsMap.put(entry.getKey(), entry.getValue().getVal());
        }
      }

      var res = Transformer.builder(text)
              .withLibrary(new XtraSonnetCustomFunc())
              .build()
              .transform(serializeXtraArgs(xtraArgsMap));

      // Get the String Output & Transform it into JsonnetSchema
      this.jsonObject = new JSONObject(res);

      // Delete File
      FileUtils.deleteQuietly(file);

    } catch (final Exception e) {
      throw new JsonnetLoaderException(e.getMessage());
    }
  }
  private static String serializeXtraArgs(Map<String, String> xtraArgsMap) {
    try {
      ObjectMapper objectMapper = new ObjectMapper();
      return objectMapper.writeValueAsString(xtraArgsMap);
    } catch (Exception e) {
      e.printStackTrace();
      return "{}";
    }
  }

  
}

[jam01]

no worries!

so did you achieve what you were looking to do?

[sandys]

So one of the things is we are struggling to conceptually map things from sjsonnet to your library. For example the jsonnet standard way to send extra args is tlavars and extvars. You use the Transform function as the main function - but I had to try and figure out how to map that. I don't think I have figured out entirely - which is what you see in my code above. If I could ask for just one help, that would be a basic java example that just loads a jsonnet file and passes some external args to execute.

…

On Thu, Jul 20, 2023, 23:49 Jose Montoya ***@***.***> wrote: no worries! so did you achieve what you were looking to do? — Reply to this email directly, view it on GitHub <#17 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAASYUY7BZVK2GXPNTKBFG3XRFZERANCNFSM6AAAAAA2MYW2FY> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[jam01]

Ok so, xtrasonnet does not keep the ext and tla vars functionality as is. ext vars are not supported entirely and tla vars are supported as 'inputs'.

Here's an example of how inputs are used:

https://github.com/jam01/xtrasonnet/blob/0.5.4/xtrasonnet/src/test/java/io/github/jam01/xtrasonnet/NestedDocumentsTest.java

Let me know if that helps

[sandys]

on a longer term, what is the plan with extvars and tla vars ? will u keep them this way ? we like xtrasonnet and will probably continue to use it in the longer term, but are worried if xtrasonnet will diverge too much from the published jsonnet standard.

this is not an unusual request from me - almost every project that uses jsonnet eventually debates on tla/ext . Even in our case, we get some data from a database and inject it into the jsonnet file - so want to figure out a standard way to refer to that data. Other people will want to pass data from a UI.

e.g. argoproj/argo-cd#1107 and argoproj/argo-cd#1107 (comment)
and https://tanka.dev/jsonnet/injecting-values

[jam01]

So.... Yeah, I have no issues retrofitting support for those vars. It may not be straight fwd, as I designed them out at the genesis of the project. Though I have been trying to move it back to be closer to jsonnet recently. So answer is I'll take a look to see how I can make them fit. Though I'm not having a lot of free time lately, so I'm unable to give an -------- Original Message --------

…

On Jul 25, 2023, 7:54 AM, Sandeep Srinivasa wrote: on a longer term, what is the plan with extvars and tla vars ? will u keep them this way ? we like xtrasonnet and will probably continue to use it in the longer term, but are worried if xtrasonnet will diverge too much from the published jsonnet standard. this is not an unusual request from me - almost every project that uses jsonnet eventually debates on tla/ext . Even in our case, we get some data from a database and inject it into the jsonnet file - so want to figure out a standard way to refer to that data. Other people will want to pass data from a UI. e.g. [argoproj/argo-cd#1107](argoproj/argo-cd#1107) and [argoproj/argo-cd#1107 (comment)](argoproj/argo-cd#1107 (comment)) and https://tanka.dev/jsonnet/injecting-values — Reply to this email directly, [view it on GitHub](#17 (comment)), or [unsubscribe](https://github.com/notifications/unsubscribe-auth/ACNAEFEPRGBBV4I4SHYD6IDXR7FYBANCNFSM6AAAAAA2MYW2FY). You are receiving this because you were mentioned.Message ID: ***@***.***>

[sandys]

thank you - you're awesome!

[jam01]

0.6.0 has been released which uses sjsonnet 0.4.5. With this one you should be able to pass in a custom Std aside from custom xtrasonnet Library.

The aspect of tla and ext will be addressed in a different release, because there's potential backwards breaking changes.

Feel free to re-open the PR if there's something missing.

[sandys]

thank you so much!!