Observability in Spring Boot 4.1: Metrics, Structured Logs, and OTLP Traces

Observability is the ability to understand a running system from the telemetry it produces. For a Spring Boot microservice, that telemetry normally includes metrics, logs, and traces, but collecting all three signals is not enough by itself.

A useful system must also answer operational questions:

• Is the user-facing service meeting its latency and availability objectives?
• Which route, dependency, or release caused the regression?
• Can an engineer move from an alert to a trace and then to the relevant logs?
• Are metric labels bounded, or will cardinality exhaust the monitoring system?
• Are secrets and personal data excluded from telemetry?
• Does the telemetry pipeline continue working during an incident?
• Can the team control sampling, retention, and cost?

This guide uses Spring Boot 4.1, Micrometer, Prometheus, Spring Boot's native structured logging, OpenTelemetry over OTLP, and Grafana-compatible backends.

TL;DR Use Actuator and Micrometer for metrics, Spring Boot's built-in JSON logging instead of an unnecessary custom encoder, and spring-boot-starter-opentelemetry for OTLP traces. Keep metric labels low-cardinality, use trace sampling deliberately, and send telemetry through a collector rather than coupling the application to one backend.

Monitoring and Observability Are Related but Different

Monitoring checks known conditions:

HTTP error rate is above 2%
PostgreSQL pool usage is above 90%
Kafka consumer lag is increasing

Observability helps investigate questions that were not fully predicted:

Why did checkout latency increase only for mobile users?
Which downstream dependency dominates the slow traces?
Did the regression begin after one application version was deployed?
Are retries multiplying calls to one failing service?

Metrics, logs, and traces have different strengths.

Do not force every question into one signal. Use a shared service identity, environment, version, and trace context so the signals can be correlated.

Recommended Telemetry Flow

A production-oriented layout can be:

Spring Boot service
    |
    |-- /actuator/prometheus --> Prometheus
    |
    |-- OTLP traces -----------> OpenTelemetry Collector
    |                              |
    |                              +--> Tempo, Jaeger, or another backend
    |
    +-- JSON logs to stdout ----> Grafana Alloy or another log collector
                                   |
                                   +--> Loki or another log backend

Grafana
    |-- queries Prometheus
    |-- queries Loki
    +-- queries the trace backend

The collector layer provides buffering, retries, enrichment, filtering, and backend independence. The application should not need a code change merely because the trace backend changes.

Promtail should not be selected for a new design. It reached end of life in March 2026; Grafana directs new log collection work to Grafana Alloy or another supported client.

Dependencies for Spring Boot 4.1

A minimal Maven setup is:

<parent>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-parent</artifactId>
    <version>4.1.0</version>
    <relativePath />
</parent>

<properties>
    <java.version>25</java.version>
</properties>

<dependencies>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-actuator</artifactId>
    </dependency>

    <dependency>
        <groupId>io.micrometer</groupId>
        <artifactId>micrometer-registry-prometheus</artifactId>
        <scope>runtime</scope>
    </dependency>

    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-opentelemetry</artifactId>
    </dependency>

    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-webmvc</artifactId>
    </dependency>

    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-test</artifactId>
        <scope>test</scope>
    </dependency>
</dependencies>

The OpenTelemetry starter connects Spring Boot's Micrometer Observation support to OpenTelemetry and supplies OTLP trace exporting.

Do not add a second tracing bridge, a manually constructed OpenTelemetry SDK, and the Java agent at the same time without a clear design. Duplicate instrumentation can create repeated spans and confusing service graphs.

Application Configuration

spring:
  application:
    name: order-service
    version: ${APP_VERSION:local}

management:
  endpoints:
    web:
      exposure:
        include: health,info,prometheus

  endpoint:
    health:
      probes:
        enabled: true
      show-details: never

  metrics:
    tags:
      application: ${spring.application.name}
      environment: ${DEPLOYMENT_ENVIRONMENT:local}
      region: ${CLOUD_REGION:local}

  tracing:
    sampling:
      probability: ${TRACE_SAMPLING_PROBABILITY:0.10}

  opentelemetry:
    tracing:
      export:
        otlp:
          endpoint: ${OTEL_TRACES_ENDPOINT:http://localhost:4318/v1/traces}
      limits:
        max-attributes: 64
        max-attribute-value-length: 256

logging:
  structured:
    format:
      console: logstash

Important choices:

• only the required Actuator endpoints are exposed;
• health details are not published publicly;
• common metric tags use bounded values;
• production trace sampling defaults to 10%, not 100%;
• span limits bound attribute cost;
• JSON logging uses Spring Boot's native Logstash format.

Use 100% sampling in local testing when necessary. In production, start from an explicit budget and adjust based on traffic, incident needs, and backend capacity.

Metrics: Measure User Impact First

Start with the RED method for request-driven services:

• Rate: requests or business operations per second;
• Errors: failures divided by total attempts;
• Duration: latency distribution, especially high percentiles.

For infrastructure resources, use the USE method:

• Utilization
• Saturation
• Errors

Examples include:

• CPU utilization;
• JVM memory pressure;
• database connection-pool utilization;
• executor queue depth;
• Kafka producer errors;
• Kafka consumer lag.

Avoid creating dashboards that show dozens of JVM charts but cannot answer whether checkout is working.

Prometheus Metric Naming

Micrometer uses a backend-independent dotted name in application code and converts it to Prometheus naming conventions.

Use:

Counter.builder("order.processing")

rather than manually embedding the Prometheus _total suffix:

Counter.builder("order.processing.total")

The Prometheus registry converts a counter such as order.processing to an exported counter name ending in _total.

Prefer one metric with a bounded outcome label:

order_processing_total{outcome="success"}
order_processing_total{outcome="failure"}

instead of creating a different metric name for every result.

Cardinality Is a Reliability Constraint

A Prometheus time series is identified by its metric name and complete label set. These labels are dangerous:

user_id
order_id
email
request_id
raw_url
exception_message
SQL statement

Each unique value creates a new series.

Safer labels have a finite, controlled set:

route
method
status
outcome
region
service
dependency
operation

Use logs or sampled traces for individual identifiers when policy allows it. Do not turn Prometheus into an event database.

Create One Observation for Metrics and Traces

Spring Boot integrates Micrometer Observation with tracing. A completed observation can produce both a metric and a span.

@Service
public class OrderService {

    private final ObservationRegistry observationRegistry;
    private final OrderRepository orderRepository;

    public OrderService(
            ObservationRegistry observationRegistry,
            OrderRepository orderRepository
    ) {
        this.observationRegistry = observationRegistry;
        this.orderRepository = orderRepository;
    }

    public OrderResult process(CreateOrderCommand command) {
        Observation observation = Observation
                .createNotStarted(
                        "order.process",
                        observationRegistry
                )
                .contextualName("process order")
                .lowCardinalityKeyValue(
                        "order.channel",
                        normalizeChannel(command.channel())
                )
                .start();

        try (Observation.Scope ignored =
                     observation.openScope()) {

            OrderResult result =
                    orderRepository.create(command);

            observation.lowCardinalityKeyValue(
                    "outcome",
                    "success"
            );

            return result;
        } catch (RuntimeException exception) {
            observation.lowCardinalityKeyValue(
                    "outcome",
                    "failure"
            );
            observation.error(exception);
            throw exception;
        } finally {
            observation.stop();
        }
    }

    private String normalizeChannel(String channel) {
        return switch (channel) {
            case "WEB", "MOBILE", "PARTNER" -> channel;
            default -> "OTHER";
        };
    }
}

order.channel and outcome are low-cardinality values suitable for metrics.

An order ID is not. A high-cardinality identifier may be attached to a trace only after reviewing privacy, retention, and backend cost, but it should not become a metric label.

Use Histograms for Latency Objectives

A timer's count and total time are not enough to evaluate a latency objective. Configure histogram buckets around meaningful thresholds.

management:
  metrics:
    distribution:
      percentiles-histogram:
        order.process: true
      slo:
        order.process: 100ms,250ms,500ms,1s,2s

Prometheus can then calculate an approximate percentile:

histogram_quantile(
  0.95,
  sum by (le) (
    rate(order_process_seconds_bucket[5m])
  )
)

For alerting, prefer objective-based ratios over isolated percentile spikes.

Example error ratio:

sum(rate(http_server_requests_seconds_count{
  status=~"5.."
}[5m]))
/
sum(rate(http_server_requests_seconds_count[5m]))

The exact metric names and labels should be confirmed from the application's /actuator/prometheus output because conventions and instrumentation can differ by stack.

Structured Logging Without a Custom Encoder

Spring Boot supports ECS, GELF, and Logstash JSON formats directly.

logging:
  structured:
    format:
      console: logstash

This removes the need for a third-party Logback encoder in a basic setup.

Spring Boot's structured formats include MDC fields. When Micrometer Tracing is active, Spring Boot places traceId and spanId in the MDC and includes correlation information in logs by default.

A custom servlet filter that copies Span.current() into MDC is usually unnecessary and can run before a valid server span exists, duplicate fields, or mishandle asynchronous dispatch.

Add Structured Business Context

Use SLF4J's fluent key-value API:

private static final Logger log =
        LoggerFactory.getLogger(OrderService.class);

public void recordAccepted(
        String channel,
        String paymentMethod
) {
    log.atInfo()
            .addKeyValue(
                    "order.channel",
                    channel
            )
            .addKeyValue(
                    "payment.method",
                    paymentMethod
            )
            .log("Order accepted");
}

Use stable field names across services.

Recommended fields include:

event.name
operation
outcome
dependency
retry.attempt
message.topic
message.partition
service.version
deployment.environment
cloud.region

Do not log:

• access tokens;
• passwords;
• session cookies;
• payment-card data;
• complete request or response bodies;
• unredacted personal data;
• database connection strings;
• arbitrary headers.

Structured logging makes sensitive data easier to query, not safer to collect.

Keep Log Labels Bounded in Loki

Loki indexes labels rather than every word in the log message. That makes label selection critical.

Appropriate Loki labels:

service_name
environment
region
level
namespace

Poor labels:

trace_id
span_id
user_id
order_id
request_id
logger_name with thousands of values

Keep traceId and spanId as fields in the log body. Query them when investigating a trace, but do not promote them to indexed labels.

The original Promtail pattern of turning every trace ID into a Loki label would create severe cardinality.

Trace Propagation

Spring Boot propagates trace context automatically when HTTP clients are built from the auto-configured builders.

@Configuration
public class ClientConfiguration {

    @Bean
    RestClient inventoryClient(
            RestClient.Builder builder
    ) {
        return builder
                .baseUrl(
                        "http://inventory-service"
                )
                .build();
    }
}

Avoid:

RestClient.create(
        "http://inventory-service"
);

A client created manually outside the auto-configured builder may not receive the tracing interceptors.

The same principle applies to auto-configured RestTemplateBuilder and WebClient.Builder.

Manual Spans Versus Observations

Use an observation when the operation should generate both metrics and traces.

Use the lower-level Micrometer Tracer when a trace-only child span is needed.

Do not import the OpenTelemetry API throughout business code merely because the backend is OpenTelemetry. Micrometer's abstraction keeps most application code independent of the tracing implementation.

A custom span should describe a meaningful boundary:

reserve inventory
evaluate fraud policy
render invoice
call payment provider

Avoid spans around every private method. Excess spans increase cost and make traces harder to read.

Sampling Is a Data Policy

Spring Boot samples 10% by default. A fixed ratio is a starting point, not a complete policy.

Consider:

• normal traffic volume;
• trace backend capacity;
• retention period;
• incident response requirements;
• rare but critical operations;
• parent sampling behavior;
• whether errors need higher retention.

Head sampling decides before the full trace is known. Tail sampling in an OpenTelemetry Collector can retain traces based on the completed result, such as errors or high latency, but requires the collector to buffer trace data and should be capacity-planned carefully.

Never assume that a missing trace proves a request never happened. It may simply have been unsampled.

Virtual Threads and Context

Virtual threads improve the scalability of blocking I/O code, but they do not remove observability boundaries.

Important considerations:

• thread count is no longer a direct proxy for expensive platform threads;
• database connections, CPU, and downstream capacity remain bounded;
• task submission through an uninstrumented custom executor can lose context;
• detached work may outlive the parent request;
• logs from reused or unmanaged scopes can receive the wrong context.

Prefer Spring-managed, instrumented executors and structured task lifecycles. Do not manually copy MDC maps as the first solution; context propagation should be handled by the tracing and observation infrastructure.

Watch:

active requests
database pool usage
executor queue depth
CPU saturation
downstream latency
carrier-thread pinning symptoms

rather than alerting on a large virtual-thread count alone.

Kafka Observability

Kafka adds asynchronous boundaries. A producer trace and consumer trace may be linked through propagated message headers rather than one synchronous call stack.

Track:

• producer send latency and error count;
• consumer lag;
• processing duration;
• retry and dead-letter count;
• rebalance frequency;
• message age at consumption;
• records discarded by deserialization or validation.

Log topic and partition when useful, but do not turn partition, offset, or message key into unbounded metric labels.

When creating producers and consumers through Spring Boot auto-configuration, observation and tracing integrations can be applied consistently. Manually constructed clients require separate instrumentation.

Secure Actuator Endpoints

The Prometheus endpoint contains operational information. It should be exposed only to the monitoring network or protected through authentication.

Do not expose all endpoints:

management:
  endpoints:
    web:
      exposure:
        include: "*"

A safer baseline is:

management:
  endpoints:
    web:
      exposure:
        include: health,info,prometheus

Consider a separate management port and firewall policy in production.

Health checks should distinguish:

• liveness: should the process be restarted?
• readiness: should it receive traffic?

Do not make liveness depend on every external dependency. A temporary database failure should not necessarily create a restart loop.

Observe the Observability Pipeline

Telemetry collection can fail during the incident when it is most valuable.

Monitor:

• Prometheus target scrape success;
• time since last successful scrape;
• collector queue size and rejected data;
• OTLP export errors;
• log collector file or container offsets;
• Loki ingestion errors;
• trace-backend ingestion latency;
• storage utilization;
• dropped spans and logs;
• dashboard and alert-rule evaluation failures.

The collector should have bounded queues and retry policies. Unlimited buffering can convert a backend outage into local disk exhaustion.

Alert on Symptoms and Causes Separately

User-facing alerts:

availability objective burn rate
latency objective burn rate
checkout failure ratio
message processing delay

Cause-oriented alerts:

database pool saturation
Kafka lag
OTLP exporter failures
disk pressure
collector dropped spans

Page an engineer for user impact or imminent loss of service. Route diagnostic warnings to lower urgency unless they threaten an objective.

Testing Observability

Telemetry should be tested like any other contract.

Metrics tests

Verify:

• the expected meter is registered;
• label values stay within an allowed set;
• both success and failure paths are recorded;
• no business identifier becomes a tag.

Trace tests

Verify:

• server and client spans are connected;
• auto-configured HTTP clients propagate context;
• exceptions are recorded;
• sampling is overridden deliberately in tests that need exported traces.

Spring Boot does not configure reporting components automatically in every @SpringBootTest, so test the instrumentation contract separately from backend delivery.

Log tests

Verify:

• output is valid JSON;
• traceId and spanId appear inside a traced request;
• secrets and personal data are redacted;
• multiline exceptions remain parseable;
• key names match the ingestion pipeline.

End-to-end test

Generate one synthetic request and confirm:

request produces HTTP metrics
trace reaches the backend
log contains the same trace ID
Grafana links or queries can move between signals

A dashboard screenshot is not proof that correlation works.

Troubleshooting

/actuator/prometheus returns 404

Check:

• Actuator dependency;
• Prometheus registry dependency;
• endpoint exposure;
• management port and base path;
• Spring Security rules.

Prometheus target is down

Check the Prometheus target page and verify:

• container-to-host networking;
• DNS;
• TLS;
• authentication;
• firewall rules;
• application readiness;
• the exact metrics path.

host.docker.internal is not a universal Linux networking solution. Prefer placing local containers on one Compose network or configure the host gateway explicitly.

Logs are JSON but trace fields are missing

Check:

• tracing starter is present;
• the log statement runs inside an active observation;
• the request is processed on a context-propagated execution path;
• a custom executor is not dropping context;
• custom Logback configuration still uses Spring Boot's structured encoder.

Traces do not reach the backend

Check:

• OTLP endpoint and protocol;
• Collector receiver port;
• TLS and authentication;
• application exporter errors;
• collector queue and export errors;
• trace sampling;
• service name and environment attributes.

OTLP/HTTP normally uses port 4318; OTLP/gRPC normally uses 4317. A legacy Jaeger HTTP collector endpoint is not automatically an OTLP endpoint.

Loki performance degrades

Check label cardinality first. Remove trace IDs, user IDs, order IDs, and raw URLs from labels. Keep them in parsed fields or log content.

Metrics explode after a deployment

Inspect new label values. An exception message, path variable, or identifier may have been added as a tag.

Conclusion

A Spring Boot observability design is effective when it shortens the path from user impact to root cause without creating unbounded telemetry cost.

A practical setup should:

• start from SLOs and operational questions;
• expose Prometheus metrics through Actuator;
• keep labels low-cardinality;
• use Micrometer Observation for related metrics and spans;
• enable Spring Boot's native structured JSON logging;
• rely on built-in trace correlation instead of a custom MDC filter;
• export traces through OTLP;
• route telemetry through a supported collector;
• use Grafana Alloy rather than the end-of-life Promtail agent;
• secure management endpoints;
• sample and retain traces deliberately;
• test correlation across metrics, traces, and logs;
• monitor the telemetry pipeline itself.

Observability is not the amount of data collected. It is the system's ability to produce the right evidence when behavior is surprising.

Official References

• Spring Boot metrics
• Spring Boot tracing
• Spring Boot structured logging
• OpenTelemetry Protocol
• OpenTelemetry Collector
• Prometheus documentation
• Grafana Alloy
• Grafana Promtail end-of-life notice