Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Export logs only if the level/logger/target is enabled. #1147

Merged
merged 19 commits into from
Jul 29, 2023
Merged
Show file tree
Hide file tree
Changes from 10 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions opentelemetry-api/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -31,3 +31,4 @@ trace = ["pin-project-lite"]
metrics = []
testing = ["trace"]
logs = []
logs_level_enabled = ["logs"]
5 changes: 5 additions & 0 deletions opentelemetry-api/src/global/logs.rs
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,11 @@
fn emit(&self, record: crate::logs::LogRecord) {
self.0.emit(record)
}

#[cfg(feature = "logs_level_enabled")]
fn event_enabled(&self, level: crate::logs::Severity, target: &str) -> bool {
self.0.event_enabled(level, target)
}

Check warning on line 59 in opentelemetry-api/src/global/logs.rs

View check run for this annotation

Codecov / codecov/patch

opentelemetry-api/src/global/logs.rs#L57-L59

Added lines #L57 - L59 were not covered by tests
}

#[derive(Clone)]
Expand Down
8 changes: 8 additions & 0 deletions opentelemetry-api/src/logs/logger.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,22 @@ use std::{borrow::Cow, sync::Arc};

use crate::{logs::LogRecord, InstrumentationLibrary, KeyValue};

#[cfg(feature = "logs_level_enabled")]
use super::Severity;

/// The interface for emitting [`LogRecord`]s.

pub trait Logger {
/// Emit a [`LogRecord`]. If there is active current thread's [`Context`],
/// the logger will set the record's [`TraceContext`] to the active trace context,
///
/// [`Context`]: crate::Context
/// [`TraceContext`]: crate::logs::TraceContext
fn emit(&self, record: LogRecord);

#[cfg(feature = "logs_level_enabled")]
/// Check if the given log level is enabled.
fn event_enabled(&self, level: Severity, target: &str) -> bool;
}

/// Interfaces that can create [`Logger`] instances.
Expand Down
4 changes: 4 additions & 0 deletions opentelemetry-api/src/logs/noop.rs
Original file line number Diff line number Diff line change
Expand Up @@ -40,4 +40,8 @@

impl Logger for NoopLogger {
fn emit(&self, _record: LogRecord) {}
#[cfg(feature = "logs_level_enabled")]
fn event_enabled(&self, _level: super::Severity, _target: &str) -> bool {
true
}

Check warning on line 46 in opentelemetry-api/src/logs/noop.rs

View check run for this annotation

Codecov / codecov/patch

opentelemetry-api/src/logs/noop.rs#L44-L46

Added lines #L44 - L46 were not covered by tests
}
2 changes: 1 addition & 1 deletion opentelemetry-appender-tracing/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -19,4 +19,4 @@ tracing-subscriber = { version = "0.3.0", default-features = false, features = [
once_cell = "1.13.0"

[dev-dependencies]
opentelemetry-stdout = { path = "../opentelemetry-stdout", features = ["logs"] }
opentelemetry-stdout = { path = "../opentelemetry-stdout", features = ["logs"] }
11 changes: 11 additions & 0 deletions opentelemetry-appender-tracing/src/layer.rs
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,17 @@ where
event.record(&mut visitor);
self.logger.emit(log_record);
}

#[cfg(feature = "logs_level_enabled")]
fn event_enabled(
&self,
_event: &tracing_core::Event<'_>,
_ctx: tracing_subscriber::layer::Context<'_, S>,
) -> bool {
let severity = map_severity_to_otel_severity(_event.metadata().level().as_str());
self.logger
.event_enabled(severity, _event.metadata().target())
}
}

fn map_severity_to_otel_severity(level: &str) -> Severity {
Expand Down
1 change: 1 addition & 0 deletions opentelemetry-sdk/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@ default = ["trace"]
trace = ["opentelemetry_api/trace", "crossbeam-channel", "rand", "async-trait", "percent-encoding"]
jaeger_remote_sampler = ["trace", "opentelemetry-http", "http", "serde", "serde_json", "url"]
logs = ["opentelemetry_api/logs", "crossbeam-channel", "async-trait", "serde_json"]
logs_level_enabled = ["logs", "opentelemetry_api/logs_level_enabled"]
metrics = ["opentelemetry_api/metrics", "regex", "async-trait"]
testing = ["opentelemetry_api/testing", "trace", "metrics", "logs", "rt-async-std", "rt-tokio", "rt-tokio-current-thread", "tokio/macros", "tokio/rt-multi-thread"]
rt-tokio = ["tokio", "tokio-stream"]
Expand Down
7 changes: 7 additions & 0 deletions opentelemetry-sdk/src/export/logs/mod.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
//! Log exporters
use crate::Resource;
use async_trait::async_trait;
//#[cfg(feature = "logs_level_enabled")]
lalitb marked this conversation as resolved.
Show resolved Hide resolved
use opentelemetry_api::logs::Severity;
use opentelemetry_api::{
logs::{LogError, LogRecord, LogResult},
InstrumentationLibrary,
Expand All @@ -14,6 +16,11 @@
async fn export(&mut self, batch: Vec<LogData>) -> LogResult<()>;
/// Shuts down the expoter.
fn shutdown(&mut self) {}
//#[cfg(feature = "logs_level_enabled")]
lalitb marked this conversation as resolved.
Show resolved Hide resolved
/// Chek if logs are enabled.
fn event_enabled(&self, _level: Severity, _target: &str, _name: &str) -> bool {
lalitb marked this conversation as resolved.
Show resolved Hide resolved
true
}

Check warning on line 23 in opentelemetry-sdk/src/export/logs/mod.rs

View check run for this annotation

Codecov / codecov/patch

opentelemetry-sdk/src/export/logs/mod.rs#L21-L23

Added lines #L21 - L23 were not covered by tests
}

/// `LogData` associates a [`LogRecord`] with a [`Resource`] and
Expand Down
23 changes: 23 additions & 0 deletions opentelemetry-sdk/src/logs/log_emitter.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,10 @@
trace::TraceContextExt,
Context, InstrumentationLibrary,
};

#[cfg(feature = "logs_level_enabled")]
use opentelemetry_api::logs::Severity;

use std::{
borrow::Cow,
sync::{Arc, Weak},
Expand Down Expand Up @@ -225,4 +229,23 @@
processor.emit(data);
}
}

#[cfg(feature = "logs_level_enabled")]
fn event_enabled(&self, level: Severity, target: &str) -> bool {
let provider = match self.provider() {
Some(provider) => provider,
None => return false,

Check warning on line 237 in opentelemetry-sdk/src/logs/log_emitter.rs

View check run for this annotation

Codecov / codecov/patch

opentelemetry-sdk/src/logs/log_emitter.rs#L234-L237

Added lines #L234 - L237 were not covered by tests
};

let mut enabled = false;
for processor in provider.log_processors() {
enabled = enabled
|| processor.event_enabled(
level,
target,
self.instrumentation_library().name.as_ref(),
);

Check warning on line 247 in opentelemetry-sdk/src/logs/log_emitter.rs

View check run for this annotation

Codecov / codecov/patch

opentelemetry-sdk/src/logs/log_emitter.rs#L240-L247

Added lines #L240 - L247 were not covered by tests
}
enabled
}

Check warning on line 250 in opentelemetry-sdk/src/logs/log_emitter.rs

View check run for this annotation

Codecov / codecov/patch

opentelemetry-sdk/src/logs/log_emitter.rs#L249-L250

Added lines #L249 - L250 were not covered by tests
}
16 changes: 15 additions & 1 deletion opentelemetry-sdk/src/logs/log_processor.rs
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@
future::{self, Either},
{pin_mut, stream, StreamExt as _},
};
#[cfg(feature = "logs_level_enabled")]
use opentelemetry_api::logs::Severity;
use opentelemetry_api::{
global,
logs::{LogError, LogResult},
Expand All @@ -27,6 +29,9 @@
fn force_flush(&self) -> LogResult<()>;
/// Shuts down the processor.
fn shutdown(&mut self) -> LogResult<()>;
#[cfg(feature = "logs_level_enabled")]
/// Check if logging is enabled
fn event_enabled(&self, level: Severity, target: &str, name: &str) -> bool;
}

/// A [`LogProcessor`] that exports synchronously when logs are emitted.
Expand Down Expand Up @@ -93,9 +98,13 @@
)))
}
}

Ok(())
}

#[cfg(feature = "logs_level_enabled")]
fn event_enabled(&self, _level: Severity, _target: &str, _name: &str) -> bool {
true
}

Check warning on line 107 in opentelemetry-sdk/src/logs/log_processor.rs

View check run for this annotation

Codecov / codecov/patch

opentelemetry-sdk/src/logs/log_processor.rs#L105-L107

Added lines #L105 - L107 were not covered by tests
}

/// A [`LogProcessor`] that asynchronously buffers log records and reports
Expand All @@ -121,6 +130,11 @@
}
}

#[cfg(feature = "logs_level_enabled")]
fn event_enabled(&self, _level: Severity, _target: &str, _name: &str) -> bool {
true
}

Check warning on line 136 in opentelemetry-sdk/src/logs/log_processor.rs

View check run for this annotation

Codecov / codecov/patch

opentelemetry-sdk/src/logs/log_processor.rs#L134-L136

Added lines #L134 - L136 were not covered by tests

fn force_flush(&self) -> LogResult<()> {
let (res_sender, res_receiver) = oneshot::channel();
self.message_sender
Expand Down
7 changes: 7 additions & 0 deletions opentelemetry-user-events-logs/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -25,3 +25,10 @@ tracing = { version = "0.1.37", default-features = false, features = ["std"] }
tracing-core = "0.1.31"
tracing-subscriber = { version = "0.3.0", default-features = false, features = ["registry", "std"] }

[features]
logs = ["opentelemetry_api/logs"]
logs_level_enabled = ["logs", "opentelemetry_api/logs_level_enabled"]

[[example]]
name = "basic"
path = "examples/basic.rs"
6 changes: 5 additions & 1 deletion opentelemetry-user-events-logs/examples/basic.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,15 @@
use opentelemetry_appender_tracing::layer;
use opentelemetry_sdk::logs::LoggerProvider;
use opentelemetry_user_events_logs::{ExporterConfig, ReentrantLogProcessor};
use std::collections::HashMap;
use tracing::error;
use tracing_subscriber::prelude::*;

fn init_logger() -> LoggerProvider {
let exporter_config = ExporterConfig { keyword: 1 };
let exporter_config = ExporterConfig {
default_keyword: 1,
keywords_map: HashMap::new(),
};
let reenterant_processor = ReentrantLogProcessor::new("test", None, exporter_config);
LoggerProvider::builder()
.with_log_processor(reenterant_processor)
Expand Down
131 changes: 76 additions & 55 deletions opentelemetry-user-events-logs/src/logs/exporter.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
use eventheader::{FieldFormat, Level, Opcode};
use eventheader_dynamic::EventBuilder;
use std::borrow::Cow;
use std::collections::HashMap;
use std::fmt::Debug;
use std::sync::Arc;

Expand All @@ -17,18 +18,32 @@
#[derive(Debug)]
pub struct ExporterConfig {
/// keyword associated with user_events name
pub keyword: u64,
/// These should be mapped to logger_name as of now.
pub keywords_map: HashMap<String, u64>,
/// default keyword if map is not defined.
pub default_keyword: u64,
}

impl Default for ExporterConfig {
fn default() -> Self {
ExporterConfig { keyword: 1 }
ExporterConfig {
keywords_map: HashMap::new(),
default_keyword: 1,
}

Check warning on line 32 in opentelemetry-user-events-logs/src/logs/exporter.rs

View check run for this annotation

Codecov / codecov/patch

opentelemetry-user-events-logs/src/logs/exporter.rs#L29-L32

Added lines #L29 - L32 were not covered by tests
}
}

impl ExporterConfig {
pub(crate) fn get_log_event_keyword(&self) -> u64 {
self.keyword
pub(crate) fn get_log_keyword(&self, name: &str) -> Option<u64> {
self.keywords_map.get(name).copied()
}

Check warning on line 39 in opentelemetry-user-events-logs/src/logs/exporter.rs

View check run for this annotation

Codecov / codecov/patch

opentelemetry-user-events-logs/src/logs/exporter.rs#L37-L39

Added lines #L37 - L39 were not covered by tests

pub(crate) fn get_log_keyword_or_default(&self, name: &str) -> Option<u64> {
if self.keywords_map.is_empty() {
Some(self.default_keyword)

Check warning on line 43 in opentelemetry-user-events-logs/src/logs/exporter.rs

View check run for this annotation

Codecov / codecov/patch

opentelemetry-user-events-logs/src/logs/exporter.rs#L41-L43

Added lines #L41 - L43 were not covered by tests
} else {
self.get_log_keyword(name)

Check warning on line 45 in opentelemetry-user-events-logs/src/logs/exporter.rs

View check run for this annotation

Codecov / codecov/patch

opentelemetry-user-events-logs/src/logs/exporter.rs#L45

Added line #L45 was not covered by tests
}
}
}

Expand All @@ -51,59 +66,30 @@
options = *options.group_name(provider_name);
let mut eventheader_provider: eventheader_dynamic::Provider =
eventheader_dynamic::Provider::new(provider_name, &options);
eventheader_provider.register_set(
eventheader::Level::Informational,
exporter_config.get_log_event_keyword(),
);
eventheader_provider.register_set(
eventheader::Level::Verbose,
exporter_config.get_log_event_keyword(),
);
eventheader_provider.register_set(
eventheader::Level::Warning,
exporter_config.get_log_event_keyword(),
);
eventheader_provider.register_set(
eventheader::Level::Error,
exporter_config.get_log_event_keyword(),
);
eventheader_provider.register_set(
eventheader::Level::CriticalError,
exporter_config.get_log_event_keyword(),
);

eventheader_provider.create_unregistered(
true,
eventheader::Level::Informational,
exporter_config.get_log_event_keyword(),
);
eventheader_provider.create_unregistered(
true,
eventheader::Level::Verbose,
exporter_config.get_log_event_keyword(),
);
eventheader_provider.create_unregistered(
true,
eventheader::Level::Warning,
exporter_config.get_log_event_keyword(),
);
eventheader_provider.create_unregistered(
true,
eventheader::Level::Error,
exporter_config.get_log_event_keyword(),
);
eventheader_provider.create_unregistered(
true,
eventheader::Level::CriticalError,
exporter_config.get_log_event_keyword(),
);

if exporter_config.keywords_map.is_empty() {
println!(
lalitb marked this conversation as resolved.
Show resolved Hide resolved
"Register default keyword {}",
exporter_config.default_keyword
);
Self::register_events(&mut eventheader_provider, exporter_config.default_keyword)
}
for keyword in exporter_config.keywords_map.values() {
Self::register_events(&mut eventheader_provider, *keyword)

Check warning on line 77 in opentelemetry-user-events-logs/src/logs/exporter.rs

View check run for this annotation

Codecov / codecov/patch

opentelemetry-user-events-logs/src/logs/exporter.rs#L69-L77

Added lines #L69 - L77 were not covered by tests
}
UserEventsExporter {
provider: Arc::new(eventheader_provider),
exporter_config,
}
}

fn register_events(eventheader_provider: &mut eventheader_dynamic::Provider, keyword: u64) {
eventheader_provider.register_set(eventheader::Level::Informational, keyword);
eventheader_provider.register_set(eventheader::Level::Verbose, keyword);
eventheader_provider.register_set(eventheader::Level::Warning, keyword);
eventheader_provider.register_set(eventheader::Level::Error, keyword);
eventheader_provider.register_set(eventheader::Level::CriticalError, keyword);
}

Check warning on line 91 in opentelemetry-user-events-logs/src/logs/exporter.rs

View check run for this annotation

Codecov / codecov/patch

opentelemetry-user-events-logs/src/logs/exporter.rs#L85-L91

Added lines #L85 - L91 were not covered by tests

fn add_attributes_to_event(
&self,
eb: &mut EventBuilder,
Expand Down Expand Up @@ -176,14 +162,24 @@
&self,
log_data: &opentelemetry_sdk::export::logs::LogData,
) -> opentelemetry_sdk::export::logs::ExportResult {
println!("export_log_data called");

Check warning on line 165 in opentelemetry-user-events-logs/src/logs/exporter.rs

View check run for this annotation

Codecov / codecov/patch

opentelemetry-user-events-logs/src/logs/exporter.rs#L165

Added line #L165 was not covered by tests
let mut level: Level = Level::Invalid;
if log_data.record.severity_number.is_some() {
level = self.get_serverity_level(log_data.record.severity_number.unwrap());
}
let log_es = if let Some(es) = self.provider.find_set(
level.as_int().into(),
self.exporter_config.get_log_event_keyword(),
) {

let keyword = self
.exporter_config
.get_log_keyword_or_default(log_data.instrumentation.name.as_ref());

if keyword.is_none() {
return Ok(());
}

Check warning on line 177 in opentelemetry-user-events-logs/src/logs/exporter.rs

View check run for this annotation

Codecov / codecov/patch

opentelemetry-user-events-logs/src/logs/exporter.rs#L171-L177

Added lines #L171 - L177 were not covered by tests

let log_es = if let Some(es) = self
.provider
.find_set(level.as_int().into(), keyword.unwrap())

Check warning on line 181 in opentelemetry-user-events-logs/src/logs/exporter.rs

View check run for this annotation

Codecov / codecov/patch

opentelemetry-user-events-logs/src/logs/exporter.rs#L179-L181

Added lines #L179 - L181 were not covered by tests
{
es
} else {
return Ok(());
Expand Down Expand Up @@ -342,4 +338,29 @@
}
Ok(())
}

#[cfg(feature = "logs_level_enabled")]
fn event_enabled(&self, level: Severity, _target: &str, name: &str) -> bool {
println!("event_enabled called");
lalitb marked this conversation as resolved.
Show resolved Hide resolved
let (found, keyword) = if self.exporter_config.keywords_map.is_empty() {
(true, self.exporter_config.default_keyword)

Check warning on line 346 in opentelemetry-user-events-logs/src/logs/exporter.rs

View check run for this annotation

Codecov / codecov/patch

opentelemetry-user-events-logs/src/logs/exporter.rs#L343-L346

Added lines #L343 - L346 were not covered by tests
} else {
// TBD - target is not used as of now for comparison.
match self.exporter_config.get_log_keyword(name) {
Some(x) => (true, x),
_ => (false, 0),

Check warning on line 351 in opentelemetry-user-events-logs/src/logs/exporter.rs

View check run for this annotation

Codecov / codecov/patch

opentelemetry-user-events-logs/src/logs/exporter.rs#L349-L351

Added lines #L349 - L351 were not covered by tests
}
};
if !found {
return false;
}
let es = self
.provider
.find_set(self.get_serverity_level(level), keyword);
lalitb marked this conversation as resolved.
Show resolved Hide resolved
match es {
Some(x) => x.enabled(),
_ => false,

Check warning on line 362 in opentelemetry-user-events-logs/src/logs/exporter.rs

View check run for this annotation

Codecov / codecov/patch

opentelemetry-user-events-logs/src/logs/exporter.rs#L354-L362

Added lines #L354 - L362 were not covered by tests
};
false
}

Check warning on line 365 in opentelemetry-user-events-logs/src/logs/exporter.rs

View check run for this annotation

Codecov / codecov/patch

opentelemetry-user-events-logs/src/logs/exporter.rs#L364-L365

Added lines #L364 - L365 were not covered by tests
}
Loading
Loading