The configuration file present in the same directory as the agent is not being picked-up in 3.4.16

[petrica]

Expected behavior

Default configuration file applicationinsights.json present in the same directory as the agent should be automatically picked-up by the agent when the application starts.

Actual behavior

The default application insights configuration is being applied and the configuration file is being ignored.

To Reproduce

• Provide a configuration file that sits next to the agent where you change the selfDiagnostics level to DEBUG.
• Do NOT specify the path to the configuration file using the environment variable APPLICATIONINSIGHTS_CONFIGURATION_FILE

"selfDiagnostics": {
  "destination": "console",
  "level": "DEBUG"
}

System information

Please provide the following information:

• SDK Version: 3.4.16
• OS type and version: Debian GNU/Linux 11 (bullseye)
• Application Server type and version (if applicable): OpenJDK Runtime Environment 18.9 (build 11.0.16+8)
• Using spring-boot? Yes

Logs

Picked up JAVA_TOOL_OPTIONS: -Xms512m -Xmx1024m -XX:MaxMetaspaceSize=300m -javaagent:/applicationinsights-agent-3.4.16.jar
OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
2023-09-11 09:05:27.791Z INFO c.m.applicationinsights.agent - Application Insights Java Agent 3.4.16 started successfully (PID 1, JVM running for 3.386 s)
2023-09-11 09:05:27.796Z INFO c.m.applicationinsights.agent - Java version: 11.0.16, vendor: Oracle Corporation, home: /usr/local/openjdk-11

[heyams]

@petrica can you share your full applicationinsights.json? you can redact data that you don't want to share.

[petrica]

Sure! However it's worth noting that the configuration file is being picked-up successfully if I use the APPLICATIONINSIGHTS_CONFIGURATION_FILE env variable to explicitly specify the path to it.

{
  "instrumentation": {
    "logging": {
      "level": "ALL"
    },
    "micrometer": {
      "enabled": false
    },
    "springScheduling": {
      "enabled": false
    }
  },
  "sampling": {
    "percentage": 100
  },
  "preview": {
    "captureLoggingLevelAsCustomDimension": true,
    "captureHttpServer4xxAsError": false,
    "captureLogbackMarker":  true,
    "captureLog4jMarker":  true,
    "instrumentation": {
      "apacheCamel": {
        "enabled": true
      }
    },
    "sampling": {
      "overrides": [
        {
          "telemetryType": "request",
          "attributes": [
            {
              "key": "http.url",
              "value": "https?://[^/]+/actuator.*",
              "matchType": "regexp"
            }
          ],
          "percentage": 0
        },
        {
          "telemetryType": "dependency",
          "attributes": [
            {
              "key": "db.system",
              "value": ".*",
              "matchType": "regexp"
            }
          ],
          "percentage": 10
        },
        {
          "telemetryType": "trace",
          "attributes": [
            {
              "key": "thread.name",
              "value": ".*redacted.*",
              "matchType": "regexp"
            }
          ],
          "percentage": 0
        },
        {
          "telemetryType": "trace",
          "attributes": [
            {
              "key": "logback.mdc.redacted.contextId",
              "value": "redacted.*",
              "matchType": "regexp"
            }
          ],
          "percentage": 0
        },
        {
          "telemetryType": "trace",
          "attributes": [
            {
              "key": "thread.name",
              "value": "Redacted.*",
              "matchType": "regexp"
            }
          ],
          "percentage": 0
        },
        {
          "telemetryType": "trace",
          "attributes": [
            {
              "key": "thread.name",
              "value": "redacted.*",
              "matchType": "regexp"
            }
          ],
          "percentage": 0
        },
        {
          "telemetryType": "trace",
          "attributes": [
            {
              "key": "logback.mdc.type",
              "value": "redactedA|redactedB",
              "matchType": "regexp"
            }
          ],
          "percentage": 0
        },
        {
          "telemetryType": "trace",
          "attributes": [
            {
              "key": "discard",
              "value": ".*",
              "matchType": "regexp"
            }
          ],
          "percentage": 0
        }
      ]
    },
    "processors": [
      {
        "type": "log",
        "body": {
          "toAttributes": {
            "rules": [
              "(?<discard>Redacted text to extract from log.*)"
            ]
          }
        }
      },
      {
        "type": "metric-filter",
        "exclude": {
          "matchType": "regexp",
          "metricNames": [
            ".*redactedA.*",
            ".*redactedB.*",
            ".*redactedC.*"
          ]
        }
      }
    ]
  },
  "selfDiagnostics": {
    "destination": "console",
    "level": "INFO"
  },
  "metricIntervalSeconds": 900
}

Thank you!

[heyams]

That implies that your json config is valid.
Can you make sure to delete APPLICATIONINSIGHTS_CONFIGURATION_FILE env var and try it again?
make sure it's named correctly "applicationinsights.json". If for some reason, something wrong with the json config, log will tell you exactly what caused the failure.

please turn on debug logging by adding the following to your applicationinsights.json:

{
"selfDiagnostics": {
"level": "DEBUG"
}
}

@petrica

[petrica]

Yes, I have tested various scenarios, without the env variable the file is not being picked up. If a revert the agent to 3.4.15, under the same setup the config file is being picked up.

[heyams]

@petrica i tested it myself. it picked up json config using 3.4.16.
can you share applicationinsights.log with me?

can you also help me check this?

echo ${APPLICATIONINSIGHTS_CONFIGURATION_FILE}
or if it has value, unset it and then run your app again:
unset APPLICATIONINSIGHTS_CONFIGURATION_FILE

Additionally, how do you attach java agent by running java -javaagent:? Log will be helpful in this case.

[petrica]

The applications run under Kubernetes, that was an oversight from my side. It seem that the DiagnosticsHelper code has changed between v3.4.15 and v3.4.16. With the new version the DiagnosticsHelper.isRpIntegration() returns true, which prevents the loading of the configuration file next to the agent.

DiagnosticsHelper.rpIntegrationChar() returns 'k'.

I believe an INFO log line that informs the developer that the default configuration will not be loaded because the DiagnosticsHelper.isRpIntegration() returns true.

 private static Configuration loadConfigurationFile(Path agentJarPath) {
    String configurationContent = getEnvVar(APPLICATIONINSIGHTS_CONFIGURATION_CONTENT);
    if (configurationContent != null) {
      return getConfigurationFromEnvVar(configurationContent);
    }

    Configuration configFromProperty = extractConfigFromProperty(agentJarPath);
    if (configFromProperty != null) {
      return configFromProperty;
    }

    String runtimeAttachedConfigurationContent =
        getSystemProperty(APPLICATIONINSIGHTS_RUNTIME_ATTACHED_CONFIGURATION_CONTENT);
    if (runtimeAttachedConfigurationContent != null) {
      return getConfiguration(runtimeAttachedConfigurationContent, JsonOrigin.RUNTIME_ATTACHED);
    }

    if (DiagnosticsHelper.isRpIntegration()) {
      // users do not have write access to agent directory in rp integrations
      // and rp integrations should not use applicationinsights.json because that makes it difficult
      // to merge rp intent and user intent
      return new Configuration();
    }

    Configuration configFromJsonNextToAgent = extractConfigFromJsonNextToAgentJar(agentJarPath);
    if (configFromJsonNextToAgent != null) {
      return configFromJsonNextToAgent;
    }

    // json configuration file is not required, ok to configure via env var alone
    return new Configuration();
  }

The solution is to use other methods of providing the configuration content.
e.g. by using the env variable: APPLICATIONINSIGHTS_CONFIGURATION_CONTENT or APPLICATIONINSIGHTS_CONFIGURATION_FILE

[heyams]

I will test 3.4.16 in an AKS environment. Thanks!

[heyams]

@petrica 3.4.17 GA is live. It was a regression bug.
Thank you very much for reporting!