Skip to main content
Alpha

Telemetry & logging

NetScript treats observability as a built-in, not a bolt-on. Every service and plugin runtime is wired for OpenTelemetry — services serve their RPC handlers with trace-context propagation, the worker runtime wraps job dispatch, execution, scheduling, subprocess hand-off, and task execution in real OTel spans, and structured logs flow through the framework logger. The viewing surface is the Aspire dashboard at http://localhost:18888, which collects OTLP traces, metrics, and structured logs from every resource in the app graph (services, plugin APIs, background processors) the moment you run aspire start. You do not stand up Jaeger, Grafana, or a log shipper to get started — the AppHost provisions the OTLP collector and the dashboard for you.

A W3C traceparent header propagates from a service request through the worker runtime into a subprocess and across a saga boundary, keeping every span under one trace id.
Trace-context propagation: a single traceparent (W3C) flows service → worker → subprocess → saga, so spans from every runtime join one distributed trace in Aspire.

This page is the capability hub: what telemetry exists, how to emit it, where to view it, and the one place that is still a scaffold stub. For the full generated API of each unit, follow the reference links: the telemetry primitives live at /reference/telemetry/ and the structured logger at /reference/logger/.

What it is

Telemetry in NetScript is three OpenTelemetry signals — traces, metrics, and logs — wired through one package, @netscript/telemetry, and one viewing surface, the Aspire dashboard. Traces are the headline: a request enters a service, the worker runtime dispatches and executes a job, a subprocess runs a polyglot task, and a saga advances — and because a single W3C traceparent header propagates across every one of those process boundaries, the whole fan-out collapses into one trace tree. Metrics (worker counts, SSE connection stats) and structured logs ride the same OTLP export. The framework owns transport (OTLP/HTTP to the collector) and viewing (the dashboard); it now owns most of emission too — the worker, scheduler, queue, saga, and SSE runtimes are instrumented for you. The full mental model — what is framework-real versus a scaffold stub — is in Observability.

Learn → / Do →

What you get out of the box

Telemetry in NetScript spans three layers — emission (in framework runtimes and your handler code), transport (OTLP over HTTP to the collector), and viewing (the Aspire dashboard). The framework owns transport and viewing, and it now owns most of emission too: the worker runtime traces the entire job lifecycle automatically. The only thing you still wire by hand is custom spans inside a job handler — and even there the @netscript/telemetry helpers do the work.

Structured logging

The @netscript/logger unit gives every service and plugin a level-aware, structured logger. Config is declared in netscript.config.ts (logging: { level: 'info', format: 'text' }) and threaded through the runtime — text or JSON, filterable in the dashboard.

OpenTelemetry traces (real)

Service request paths propagate W3C trace context, and the worker runtime emits real spans for job dispatch, execution, scheduling, subprocess hand-off, and task.execute. @opentelemetry/api (^1.9) is in the catalog and wired into the request and job paths so spans flow to the collector.

OTLP → Aspire dashboard

The generated AppHost configures an OTLP endpoint (http://localhost:4318) and the Aspire dashboard (http://localhost:18888) so traces, metrics, and logs from every resource land in one place — no collector to deploy.

Per-resource health

Each plugin API exposes a liveness probe — workers :8091 GET /health, sagas :8092 GET /health/live, triggers :8093 GET /health, auth :8094 — surfaced as resource state in the dashboard.

Enable tracing & see a span

Tracing turns on from the environment@netscript/telemetry/config resolves a TelemetryConfig from the standard OTEL_* variables, and the Aspire AppHost already sets them for you. The fastest path is to run aspire start, trigger any job, and the runtime's automatic spans show up in the dashboard with no code change. The tab below adds a custom span on top of that automatic trace.

// netscript.config.ts — declare the structured-logging contract for the workspace.
// Tracing itself is enabled by OTEL_* env vars (the Aspire AppHost sets these); the
// config package reads them — there is no defineTelemetry() call to make.
import { defineConfig } from '@netscript/config';

export default defineConfig({
  name: 'my-app',
  version: '1.0.0',
  logging: { level: 'info', format: 'text' }, // level: debug|info|warn|error
  plugins: ['./plugins/workers/mod.ts'],
});

// Anywhere in app code you can inspect what telemetry resolved (log-safe, redacted):
import { describeTelemetryConfig, isTelemetryEnabled } from '@netscript/telemetry/config';
if (isTelemetryEnabled()) console.log(describeTelemetryConfig());
// → { enabled: true, endpoint: 'http://localhost:4318', protocol, serviceName, sampler }
// plugins/workers/jobs/health-check.ts — a real child span inside a handler.
// The dispatcher already wraps this job in traceJobExecution automatically;
// withChildSpan opens a child span under that parent. recordJobProgress's 3rd arg
// is a UNIT label (e.g. 'steps'), not a free-text message.
import { createSuccessResult, defineJobHandler } from '@netscript/plugin-workers-core';
import { withChildSpan, recordJobProgress } from '@netscript/telemetry/instrumentation';

const handler = defineJobHandler(async (ctx) => {
  recordJobProgress(1, 2, 'steps'); // → job.progress event on the trace

  const envCheck = await withChildSpan('check.environment', async (span) => {
    span.setAttribute('check.name', 'environment');
    return { ok: true };
  });

  recordJobProgress(2, 2, 'steps');
  return createSuccessResult({ healthy: envCheck.ok });
});

export default Object.assign(handler, { id: 'workers-plugin-health-check' as const });

Key types: the telemetry init/config surface

Configuration is read from the environment, not constructed — @netscript/telemetry/config resolves and caches a TelemetryConfig from the OTEL_* variables. These are the confirmed fields and the helpers that read them.

TelemetryConfig (@netscript/telemetry/config — resolved from OTEL_* env vars)
NameTypeDescription
enabled boolean Whether OpenTelemetry instrumentation is active (driven by OTEL_DENO / endpoint presence). Mirror it with isTelemetryEnabled().
endpoint string | undefined OTLP exporter endpoint URL (e.g. http://localhost:4318). Read directly via getOtlpEndpoint().
protocol string OTLP exporter protocol (e.g. http/protobuf), from OTEL_EXPORTER_OTLP_PROTOCOL.
serviceName string Service name reported to backends, from OTEL_SERVICE_NAME (or the default). Read via getServiceName().
serviceVersion string Service version reported to backends.
resourceAttributes Record Resource attributes parsed from OTEL_RESOURCE_ATTRIBUTES.
sampler string Trace sampler name, from OTEL_TRACES_SAMPLER (e.g. parentbased_always_on, traceidratio).
debug boolean Whether debug-level telemetry logging is on (OTEL_LOG_LEVEL).
Config helpers (@netscript/telemetry/config)
NameTypeDescription
getTelemetryConfig() → TelemetryConfig Resolve telemetry config from the OTEL_* environment variables.
getConfig() → TelemetryConfig Return the process-cached config (resetConfig() clears the cache, mainly for tests).
isTelemetryEnabled() → boolean Whether instrumentation is enabled for this process.
getOtlpEndpoint() → string | undefined The configured OTLP endpoint, when present.
getServiceName() → string The configured service name, or the default.
describeTelemetryConfig() → TelemetryConfigDescription A redacted, log-safe summary of the resolved config — safe to print in diagnostics.
OTEL_ENV_VARS const record The OTEL_* variable NAMES NetScript reads: OTEL_DENO, OTEL_EXPORTER_OTLP_ENDPOINT/_PROTOCOL, OTEL_SERVICE_NAME, OTEL_RESOURCE_ATTRIBUTES, OTEL_TRACES_SAMPLER, OTEL_LOG_LEVEL, the BSP/BLRP schedule delays, and OTEL_METRIC_EXPORT_INTERVAL.

Span & instrumentation helpers

Two subpaths cover hand-written instrumentation. @netscript/telemetry/tracer is the low-level facade over @opentelemetry/api — get a tracer, open a span, run a callback inside one. @netscript/telemetry/instrumentation is the NetScript-domain layer the runtimes themselves use — job, scheduler, queue, and SSE spans with the right attributes baked in. The /context subpath holds the traceparent propagation helpers that make the cross-boundary trace in the diagram above possible.

Tracer facade (@netscript/telemetry/tracer)
NameTypeDescription
getTracer(name, version) → Tracer Cached tracer for an instrumentation name/version. Domain shortcuts exist: getJobTracer/getQueueTracer/getWorkerTracer/getSchedulerTracer/getSagaTracer/getSSETracer/getKVTracer.
withSpan(tracer, name, fn, options) → Promise Run an async callback inside a span and close it on completion. withSpanSync is the synchronous form.
createSpan(tracer, name, options) → Span Create a span from a tracer + CreateSpanOptions (kind, attributes, parentContext, links). You end() it yourself.
setSpanAttributes / setSpanError / setSpanOk (span, …) → void Bulk-set attributes, mark a span failed (with an optional Error), or mark it OK. addSpanEvent(span, name, attrs?) adds an event.
getActiveSpan() / getActiveContext() → Span? / Context Read the span / OTel context active on the current async path. isTracingEnabled() gates work.
SpanKind / SpanStatusCode / TracerNames const Span-kind (INTERNAL/SERVER/CLIENT/PRODUCER/CONSUMER), status (UNSET/OK/ERROR), and the standard domain tracer-name strings.
NetScript instrumentation helpers (@netscript/telemetry/instrumentation)
NameTypeDescription
traceJobExecution(options, fn) → Promise Wrap a job run in a span with job.started/completed/failed events. Emitted automatically by the dispatcher; the supported entry point for tracing a job by hand.
withChildSpan(name, fn, attributes?) → Promise Open a child span under the active span and run fn inside it. The go-to for custom spans in a handler.
recordJobProgress(current, total, unit) → void Emit a job.progress event. The 3rd arg is a UNIT label (e.g. 'steps'), NOT a description. addJobStepEvent(stepName, attrs?) records job.step.* events.
startJobDispatchSpan / traceJobDispatch span / → Promise Open the dispatch span and return PropagationHeaders to carry traceparent to the executor. createJobSubprocessEnv injects traceparent/tracestate into a subprocess env.
scheduler & cron spans span / event createSchedulerStartSpan, createScheduleJobSpan, startSchedulerTickSpan, traceSchedulerTick, recordCronJobRun — cron/schedule dispatch traced end to end.
traceQueue(queue, options) / TracedQueue → TracedQueue Wrap a MessageQueue so enqueue/consume carry trace context. SSE helpers (startSSEConnection, createSSEEventSpan, traceSSEEvent) trace Server-Sent-Event streams.
Traceparent / context propagation (@netscript/telemetry/context)
NameTypeDescription
formatTraceparent(spanContext) → string Serialize a span context to a W3C traceparent header value. parseTraceparent(value) parses one back to a ParsedTraceparent | null.
injectContext(headers, ctx?) / extractContext(headers) → headers / → Context Inject the active trace context into outbound PropagationHeaders, or extract a remote one from inbound headers — the core of cross-service propagation.
withContext(ctx, fn) / withContextAsync(ctx, fn) → T / → Promise Run a callback with a given OTel context active so spans created inside attach to the right parent.
createMessageHeaders / resolveParentContextFromHeaders → headers / → Context Build propagation headers for a queue message, and resolve the parent context back out on the consumer side.
getTraceId(ctx?) / getSpanId(ctx?) → string? Read the current trace/span ids — handy for correlating structured log lines with a trace.

Service & RPC tracing toggle

Services opt into trace propagation through the oRPC layer rather than by hand. The @netscript/telemetry/orpc subpath ships an oRPC tracing plugin that opens a SERVER span per RPC and continues any inbound traceparent, plus an error-handling plugin that classifies failures. On the service builder this is the .withRPC({ traceContext: true }) toggle (see Services); the plugin factory below is the underlying surface if you mount oRPC yourself.

oRPC tracing surface (@netscript/telemetry/orpc)
NameTypeDescription
createTracingPlugin(options?) → TracingPlugin oRPC plugin that opens a SERVER span per call and continues an inbound traceparent. Options are TracingPluginOptions.
createErrorHandlingPlugin(options?) → ErrorHandlingPlugin Classifies errors as client | server | transient and records them on the span; takes an optional ErrorClassifier and ErrorLogger.
createTraceContext() → TraceContext Build the per-call trace context the plugin threads through. addEvent / setAttributes / getTraceId / getSpanId operate on the active span.

What is traced automatically

Before you write a single instrumentation call, the worker runtime already produces a usable trace tree in Aspire. The @netscript/telemetry instrumentation is wired into the dispatcher, the executor, and the scheduler, so the moment a job runs you get spans with attributes, durations, status, and lifecycle events — no scaffold changes required.

Automatic worker traces (emitted by the runtime — see /reference/telemetry/)
NameTypeDescription
traceJobExecution span Wraps each job's execution with attributes, duration, status, and job.started / job.completed / job.failed / job.exception events. Emitted by the dispatcher.
recordJobProgress event job.progress events carrying current / total / unit — the runtime records real progress as the job advances.
addJobStepEvent event job.step.* events for each step the runtime walks through.
scheduler spans span createSchedulerStartSpan / createScheduleJobSpan / startSchedulerTickSpan / recordCronJobRun — cron and schedule dispatch are traced end to end.
subprocess traceparent context The dispatcher injects W3C traceparent / tracestate into the subprocess env so out-of-process job runs continue the same trace (initJobTracing / runTracedJob / createJobSubprocessEnv).
task.execute span The multi-runtime task executor wraps each task run in a task.execute span — polyglot and TS tasks alike show up in the trace tree.

Instrument a handler

The two tabs below show the two emission paths a developer touches: structured logging through the framework logger, and custom spans inside a job handler. For custom spans, call the @netscript/telemetry helpers (traceJobExecution, withChildSpan, recordJobProgress) directly — they are the real, supported surface. Read the callout under the tabs to understand why you reach for the telemetry package rather than the scaffold's createJobTools(ctx) trace helpers.

// netscript.config.ts — declare the logging contract for the whole workspace.
import { defineConfig } from '@netscript/config';

export default defineConfig({
  name: 'my-app',
  version: '1.0.0',
  // level: 'debug' | 'info' | 'warn' | 'error'; format: 'text' | 'json'
  logging: { level: 'info', format: 'text' },
  plugins: ['./plugins/workers/mod.ts', './plugins/sagas/mod.ts'],
});

// Inside a job handler, the logger comes from the job tools (console-backed today,
// and surfaced under the resource's Console logs view in the Aspire dashboard).
// import { createJobTools } from './job-tools.ts';
const emit = (log) => {
  log.info('user provisioned', { userId: 'u_123', source: 'scaffold' });
  log.warn('rate limit approaching', { remaining: 4 });
};
// plugins/workers/jobs/health-check.ts — real custom spans inside a handler.
// The dispatcher already wraps this job in traceJobExecution automatically;
// withChildSpan + recordJobProgress let you add detail under that parent span.
import { createSuccessResult, createFailureResult, defineJobHandler } from '@netscript/plugin-workers-core';
import { withChildSpan, recordJobProgress } from '@netscript/telemetry/instrumentation';

const handler = defineJobHandler(async (ctx) => {
  const { log } = ctx;
  log.info('Starting workers plugin health check');

  // Real progress event — 3rd arg is a UNIT label, not a message.
  recordJobProgress(1, 2, 'steps');

  // withChildSpan opens a real child span under the job-execution span.
  const envCheck = await withChildSpan('check.environment', async (span) => {
    span.setAttribute('check.name', 'environment');
    return { ok: true };
  });

  recordJobProgress(2, 2, 'steps');
  if (!envCheck.ok) return createFailureResult('environment check failed');
  return createSuccessResult({ status: 'healthy' });
});

export default Object.assign(handler, { id: 'workers-plugin-health-check' as const });

Endpoints & ports

Telemetry has no single service of its own — it is emitted by every runtime and aggregated by Aspire. These are the real addresses you interact with, validated by the CLI E2E suite.

Observability surfaces (link to /reference/telemetry/ and /reference/logger/ for the generated APIs)
NameTypeDescription
http://localhost:18888 dashboard Aspire dashboard — traces, structured logs, metrics, and resource state for the whole app graph. Auth token printed by aspire start.
http://localhost:4318 OTLP/HTTP OTLP ingest endpoint the AppHost configures (aspire.config.json https profile). Runtimes export spans and logs here; the dashboard reads them back. This is the seam you point at a hosted backend.
GET :8091/health liveness Workers API health probe — reported as resource health in the dashboard.
GET :8092/health/live liveness Sagas API liveness route.
GET :8093/health liveness Triggers API health probe (Hono service).
:8094 service Auth API (auth-api) — its request spans propagate trace context like any other service.

View it in Aspire

With aspire start up, the dashboard at http://localhost:18888 gives you four views over the same telemetry stream — there is no separate tool to configure.

Aspire dashboard views (http://localhost:18888)
NameTypeDescription
Resources graph Live state of every app-graph resource: postgres, redis, workers-api, workers, sagas-api, sagas, triggers-api, triggers, auth-api. Health probes drive the status colour.
Console logs stream Per-resource stdout/stderr — the framework logger's text/JSON output lands here in real time.
Structured logs OTLP Structured log records exported over OTLP, filterable by resource, level, and attributes.
Traces OTLP Distributed traces collected from the OTLP endpoint (http://localhost:4318) — job dispatch/execution/scheduler/task spans and cross-service request spans as trace context propagates.

Production notes

Reference

This hub is intentionally thin — the full generated API lives in the reference.

telemetry