Skip to content
@ylcnfrht/resilience
Buy me a coffee

OpenTelemetry

instrumentOpenTelemetry() exports the telemetry of every pipeline in the process to OpenTelemetry: traces with a span per execution and per attempt, and four metrics. One call at startup; no change to your pipelines.

Terminal window
npm i @ylcnfrht/resilience @opentelemetry/api   # @opentelemetry/api ^1.4 (optional peer dependency), plus the SDK of your choice
telemetry.ts
import { NodeSDK } from '@opentelemetry/sdk-node';
import { instrumentOpenTelemetry } from '@ylcnfrht/resilience/otel';


const sdk = new NodeSDK({ /* exporters, resource… */ });
sdk.start();


// After the SDK has registered its global providers.
const stop = instrumentOpenTelemetry();


process.on('SIGTERM', async () => {
  stop();
  await sdk.shutdown();
});

Traces

Section titled “Traces”

Each execution gets a span, and each call of your callback gets a child span:

resilience.pipeline payments          pipeline.name=payments  (ERROR if the final outcome failed)
├─ resilience.attempt                 attempt.number=0        (ERROR: 503)
│    event: resilience.OnRetry
├─ resilience.attempt                 attempt.number=1
│  └─ HTTP POST                       ← your own spans nest under the attempt
  • Execution span: resilience.pipeline <name>, where name is the execution’s operation name (execute(fn, { name })) or the pipeline name; plain resilience.pipeline when neither is set. It covers retries, delays and fallbacks.
  • Attempt span: resilience.attempt, one per call of the callback (the first call, each retry, each hedged attempt), made active while the callback runs. Spans you start inside, such as an instrumented fetch or database call, become its children.
  • Failures set the span status to ERROR, record the exception and set error.type.
  • Strategy events (retry, timeout, circuit opened, rate limiter rejected…) are added as span events named resilience.<Event> to the span active when they fire, with strategy.name and event.name.

Nesting needs a context manager, which every OpenTelemetry Node SDK registers.

Span attributes

Section titled “Span attributes”
Attribute When
pipeline.name always ('' for an unnamed pipeline)
operation.name when the execution has a name
operation.key when the execution has an operationKey
the pipeline’s tags new ResiliencePipelineBuilder({ name, tags: { team: 'billing' } })
the execution’s attributes execute(fn, { attributes: { tenant: 'acme' } })
your attributes() result see options
attempt.number attempt spans
error.type failed spans

Metrics

Section titled “Metrics”
Name Instrument Unit Attributes What
resilience.pipeline.duration histogram s base + outcome, error.type One sample per execution, retries and delays included.
resilience.attempt.duration histogram s base + attempt.number, outcome, error.type One sample per call of the callback. Its count is the number of attempts.
resilience.strategy.events counter base + strategy.name, event.name Retries, timeouts, circuit transitions, rejections, fallbacks, cache hits…
resilience.circuit.state observable gauge pipeline.name, strategy.name 0 closed, 1 half-open, 2 open, 3 isolated.
  • Base attributes: pipeline.name, operation.key ('' when not set), operation.name when set, the pipeline’s tags, the execution’s attributes, and the result of your attributes() option.
  • outcome is success or failure. error.type is set on failures: the error’s code if it has one (TIMEOUT, CIRCUIT_OPEN, ECONNRESET…), else its class name, else its typeof. The same function is exported as errorType(error).
  • Durations are recorded in seconds, as the semantic conventions ask, with bucket advice 0.005 … 30 s (OpenTelemetry’s default buckets are meant for milliseconds).
  • The counter skips the per-call events (OnPipelineExecuting, OnStrategyExecuted, OnPipelineExecuted, OnExecutionAttempt); those feed the histograms. Event names are listed in the events catalog.
  • attempt.number on the histogram is capped at 10 (it means the 10th retry or later), so a large maxRetryAttempts can’t create unbounded series. Attempt spans keep the exact number.
  • The circuit gauge reports a breaker from its first state change (opened, half-opened, closed, or synced from a shared store) on. It keeps the 1000 most recently changed circuits; older ones (e.g. per-host pipelines that createResilientFetch dropped) stop being reported.

Options

Section titled “Options”
Option Type Default Description
meter Meter metrics.getMeter('@ylcnfrht/resilience') Meter to create the instruments on.
tracer Tracer trace.getTracer('@ylcnfrht/resilience') Tracer for the spans.
metrics boolean true Record metrics.
traces boolean true Create execution and attempt spans.
spanEvents boolean true Add strategy events to the active span. Works with traces: false too, on your own spans.
attributes (input) => Attributes | undefined none Extra attributes for every span and data point. input is a telemetry event or the execution info; both have context and source. 
instrumentOpenTelemetry({
  attributes: ({ context }) => ({ tenant: String(context.properties.get('tenant') ?? 'none') }),
});

instrumentOpenTelemetry() returns a function that stops it (removes the span hooks, the event listener and the gauge callback).

Section titled “Related”