Telemetry & logging
NetScript treats observability as a built-in, not a bolt-on. Every service and plugin
runtime is wired for OpenTelemetry — services serve their RPC handlers with
trace-context propagation, the worker runtime wraps job dispatch, execution, scheduling,
subprocess hand-off, and task execution in real OTel spans, and structured logs flow through
the framework logger. The viewing surface is the Aspire dashboard at
http://localhost:18888, which collects OTLP traces, metrics, and structured logs from
every resource in the app graph (services, plugin APIs, background processors) the moment you
run aspire start. You do not stand up Jaeger, Grafana, or a log shipper to get started — the
AppHost provisions the OTLP collector and the dashboard for you.
This page is the capability hub: what telemetry exists, how to emit it, where to view it,
and the one place that is still a scaffold stub. For the full generated API of
each unit, follow the reference links: the telemetry primitives live at
/reference/telemetry/ and the structured logger at
/reference/logger/.
What it is
Telemetry in NetScript is three OpenTelemetry signals — traces, metrics, and
logs — wired through one package, @netscript/telemetry, and one viewing surface, the
Aspire dashboard. Traces are the headline: a request enters a service, the worker runtime
dispatches and executes a job, a subprocess runs a polyglot task, and a saga advances — and
because a single W3C traceparent header propagates across every one of those process
boundaries, the whole fan-out collapses into one trace tree. Metrics (worker counts, SSE
connection stats) and structured logs ride the same OTLP export. The framework owns transport
(OTLP/HTTP to the collector) and viewing (the dashboard); it now owns most of emission too —
the worker, scheduler, queue, saga, and SSE runtimes are instrumented for you. The full
mental model — what is framework-real versus a scaffold stub — is in
Observability.
Learn → / Do →
What you get out of the box
Telemetry in NetScript spans three layers — emission (in framework runtimes and your handler
code), transport (OTLP over HTTP to the collector), and viewing (the Aspire dashboard). The
framework owns transport and viewing, and it now owns most of emission too: the worker runtime
traces the entire job lifecycle automatically. The only thing you still wire by hand is custom
spans inside a job handler — and even there the @netscript/telemetry helpers do the work.
The @netscript/logger unit gives every service and plugin a level-aware, structured logger. Config is declared in netscript.config.ts (logging: { level: 'info', format: 'text' }) and threaded through the runtime — text or JSON, filterable in the dashboard.
Service request paths propagate W3C trace context, and the worker runtime emits real spans for job dispatch, execution, scheduling, subprocess hand-off, and task.execute. @opentelemetry/api (^1.9) is in the catalog and wired into the request and job paths so spans flow to the collector.
The generated AppHost configures an OTLP endpoint (http://localhost:4318) and the Aspire dashboard (http://localhost:18888) so traces, metrics, and logs from every resource land in one place — no collector to deploy.
Each plugin API exposes a liveness probe — workers :8091 GET /health, sagas :8092 GET /health/live, triggers :8093 GET /health, auth :8094 — surfaced as resource state in the dashboard.
Enable tracing & see a span
Tracing turns on from the environment — @netscript/telemetry/config resolves a
TelemetryConfig from the standard OTEL_* variables, and the Aspire AppHost already sets
them for you. The fastest path is to run aspire start, trigger any job, and the
runtime's automatic spans show up in the dashboard with no code change. The tab below adds a
custom span on top of that automatic trace.
// netscript.config.ts — declare the structured-logging contract for the workspace.
// Tracing itself is enabled by OTEL_* env vars (the Aspire AppHost sets these); the
// config package reads them — there is no defineTelemetry() call to make.
import { defineConfig } from '@netscript/config';
export default defineConfig({
name: 'my-app',
version: '1.0.0',
logging: { level: 'info', format: 'text' }, // level: debug|info|warn|error
plugins: ['./plugins/workers/mod.ts'],
});
// Anywhere in app code you can inspect what telemetry resolved (log-safe, redacted):
import { describeTelemetryConfig, isTelemetryEnabled } from '@netscript/telemetry/config';
if (isTelemetryEnabled()) console.log(describeTelemetryConfig());
// → { enabled: true, endpoint: 'http://localhost:4318', protocol, serviceName, sampler }
// plugins/workers/jobs/health-check.ts — a real child span inside a handler.
// The dispatcher already wraps this job in traceJobExecution automatically;
// withChildSpan opens a child span under that parent. recordJobProgress's 3rd arg
// is a UNIT label (e.g. 'steps'), not a free-text message.
import { createSuccessResult, defineJobHandler } from '@netscript/plugin-workers-core';
import { withChildSpan, recordJobProgress } from '@netscript/telemetry/instrumentation';
const handler = defineJobHandler(async (ctx) => {
recordJobProgress(1, 2, 'steps'); // → job.progress event on the trace
const envCheck = await withChildSpan('check.environment', async (span) => {
span.setAttribute('check.name', 'environment');
return { ok: true };
});
recordJobProgress(2, 2, 'steps');
return createSuccessResult({ healthy: envCheck.ok });
});
export default Object.assign(handler, { id: 'workers-plugin-health-check' as const });
Key types: the telemetry init/config surface
Configuration is read from the environment, not constructed — @netscript/telemetry/config
resolves and caches a TelemetryConfig from the OTEL_* variables. These are the confirmed
fields and the helpers that read them.
| Name | Type | Description |
|---|---|---|
enabled |
boolean |
Whether OpenTelemetry instrumentation is active (driven by OTEL_DENO / endpoint presence). Mirror it with isTelemetryEnabled(). |
endpoint |
string | undefined |
OTLP exporter endpoint URL (e.g. http://localhost:4318). Read directly via getOtlpEndpoint(). |
protocol |
string |
OTLP exporter protocol (e.g. http/protobuf), from OTEL_EXPORTER_OTLP_PROTOCOL. |
serviceName |
string |
Service name reported to backends, from OTEL_SERVICE_NAME (or the default). Read via getServiceName(). |
serviceVersion |
string |
Service version reported to backends. |
resourceAttributes |
Record |
Resource attributes parsed from OTEL_RESOURCE_ATTRIBUTES. |
sampler |
string |
Trace sampler name, from OTEL_TRACES_SAMPLER (e.g. parentbased_always_on, traceidratio). |
debug |
boolean |
Whether debug-level telemetry logging is on (OTEL_LOG_LEVEL). |
| Name | Type | Description |
|---|---|---|
getTelemetryConfig() |
→ TelemetryConfig |
Resolve telemetry config from the OTEL_* environment variables. |
getConfig() |
→ TelemetryConfig |
Return the process-cached config (resetConfig() clears the cache, mainly for tests). |
isTelemetryEnabled() |
→ boolean |
Whether instrumentation is enabled for this process. |
getOtlpEndpoint() |
→ string | undefined |
The configured OTLP endpoint, when present. |
getServiceName() |
→ string |
The configured service name, or the default. |
describeTelemetryConfig() |
→ TelemetryConfigDescription |
A redacted, log-safe summary of the resolved config — safe to print in diagnostics. |
OTEL_ENV_VARS |
const record |
The OTEL_* variable NAMES NetScript reads: OTEL_DENO, OTEL_EXPORTER_OTLP_ENDPOINT/_PROTOCOL, OTEL_SERVICE_NAME, OTEL_RESOURCE_ATTRIBUTES, OTEL_TRACES_SAMPLER, OTEL_LOG_LEVEL, the BSP/BLRP schedule delays, and OTEL_METRIC_EXPORT_INTERVAL. |
Span & instrumentation helpers
Two subpaths cover hand-written instrumentation. @netscript/telemetry/tracer is the
low-level facade over @opentelemetry/api — get a tracer, open a span, run a callback inside
one. @netscript/telemetry/instrumentation is the NetScript-domain layer the runtimes
themselves use — job, scheduler, queue, and SSE spans with the right attributes baked in.
The /context subpath holds the traceparent propagation helpers that make the
cross-boundary trace in the diagram above possible.
| Name | Type | Description |
|---|---|---|
getTracer(name, version) |
→ Tracer |
Cached tracer for an instrumentation name/version. Domain shortcuts exist: getJobTracer/getQueueTracer/getWorkerTracer/getSchedulerTracer/getSagaTracer/getSSETracer/getKVTracer. |
withSpan(tracer, name, fn, options) |
→ Promise |
Run an async callback inside a span and close it on completion. withSpanSync is the synchronous form. |
createSpan(tracer, name, options) |
→ Span |
Create a span from a tracer + CreateSpanOptions (kind, attributes, parentContext, links). You end() it yourself. |
setSpanAttributes / setSpanError / setSpanOk |
(span, …) → void |
Bulk-set attributes, mark a span failed (with an optional Error), or mark it OK. addSpanEvent(span, name, attrs?) adds an event. |
getActiveSpan() / getActiveContext() |
→ Span? / Context |
Read the span / OTel context active on the current async path. isTracingEnabled() gates work. |
SpanKind / SpanStatusCode / TracerNames |
const |
Span-kind (INTERNAL/SERVER/CLIENT/PRODUCER/CONSUMER), status (UNSET/OK/ERROR), and the standard domain tracer-name strings. |
| Name | Type | Description |
|---|---|---|
traceJobExecution(options, fn) |
→ Promise |
Wrap a job run in a span with job.started/completed/failed events. Emitted automatically by the dispatcher; the supported entry point for tracing a job by hand. |
withChildSpan(name, fn, attributes?) |
→ Promise |
Open a child span under the active span and run fn inside it. The go-to for custom spans in a handler. |
recordJobProgress(current, total, unit) |
→ void |
Emit a job.progress event. The 3rd arg is a UNIT label (e.g. 'steps'), NOT a description. addJobStepEvent(stepName, attrs?) records job.step.* events. |
startJobDispatchSpan / traceJobDispatch |
span / → Promise |
Open the dispatch span and return PropagationHeaders to carry traceparent to the executor. createJobSubprocessEnv injects traceparent/tracestate into a subprocess env. |
scheduler & cron spans |
span / event |
createSchedulerStartSpan, createScheduleJobSpan, startSchedulerTickSpan, traceSchedulerTick, recordCronJobRun — cron/schedule dispatch traced end to end. |
traceQueue(queue, options) / TracedQueue |
→ TracedQueue |
Wrap a MessageQueue so enqueue/consume carry trace context. SSE helpers (startSSEConnection, createSSEEventSpan, traceSSEEvent) trace Server-Sent-Event streams. |
| Name | Type | Description |
|---|---|---|
formatTraceparent(spanContext) |
→ string |
Serialize a span context to a W3C traceparent header value. parseTraceparent(value) parses one back to a ParsedTraceparent | null. |
injectContext(headers, ctx?) / extractContext(headers) |
→ headers / → Context |
Inject the active trace context into outbound PropagationHeaders, or extract a remote one from inbound headers — the core of cross-service propagation. |
withContext(ctx, fn) / withContextAsync(ctx, fn) |
→ T / → Promise |
Run a callback with a given OTel context active so spans created inside attach to the right parent. |
createMessageHeaders / resolveParentContextFromHeaders |
→ headers / → Context |
Build propagation headers for a queue message, and resolve the parent context back out on the consumer side. |
getTraceId(ctx?) / getSpanId(ctx?) |
→ string? |
Read the current trace/span ids — handy for correlating structured log lines with a trace. |
Service & RPC tracing toggle
Services opt into trace propagation through the oRPC layer rather than by hand. The
@netscript/telemetry/orpc subpath ships an oRPC tracing plugin that opens a SERVER span
per RPC and continues any inbound traceparent, plus an error-handling plugin that classifies
failures. On the service builder this is the .withRPC({ traceContext: true }) toggle (see
Services); the plugin
factory below is the underlying surface if you mount oRPC yourself.
| Name | Type | Description |
|---|---|---|
createTracingPlugin(options?) |
→ TracingPlugin |
oRPC plugin that opens a SERVER span per call and continues an inbound traceparent. Options are TracingPluginOptions. |
createErrorHandlingPlugin(options?) |
→ ErrorHandlingPlugin |
Classifies errors as client | server | transient and records them on the span; takes an optional ErrorClassifier and ErrorLogger. |
createTraceContext() |
→ TraceContext |
Build the per-call trace context the plugin threads through. addEvent / setAttributes / getTraceId / getSpanId operate on the active span. |
What is traced automatically
Before you write a single instrumentation call, the worker runtime already produces a usable
trace tree in Aspire. The @netscript/telemetry instrumentation is wired into the dispatcher,
the executor, and the scheduler, so the moment a job runs you get spans with attributes,
durations, status, and lifecycle events — no scaffold changes required.
| Name | Type | Description |
|---|---|---|
traceJobExecution |
span |
Wraps each job's execution with attributes, duration, status, and job.started / job.completed / job.failed / job.exception events. Emitted by the dispatcher. |
recordJobProgress |
event |
job.progress events carrying current / total / unit — the runtime records real progress as the job advances. |
addJobStepEvent |
event |
job.step.* events for each step the runtime walks through. |
scheduler spans |
span |
createSchedulerStartSpan / createScheduleJobSpan / startSchedulerTickSpan / recordCronJobRun — cron and schedule dispatch are traced end to end. |
subprocess traceparent |
context |
The dispatcher injects W3C traceparent / tracestate into the subprocess env so out-of-process job runs continue the same trace (initJobTracing / runTracedJob / createJobSubprocessEnv). |
task.execute |
span |
The multi-runtime task executor wraps each task run in a task.execute span — polyglot and TS tasks alike show up in the trace tree. |
Instrument a handler
The two tabs below show the two emission paths a developer touches: structured logging
through the framework logger, and custom spans inside a job handler. For custom spans,
call the @netscript/telemetry helpers (traceJobExecution, withChildSpan,
recordJobProgress) directly — they are the real, supported surface. Read the
callout under the tabs to understand why you reach for the telemetry package rather than the
scaffold's createJobTools(ctx) trace helpers.
// netscript.config.ts — declare the logging contract for the whole workspace.
import { defineConfig } from '@netscript/config';
export default defineConfig({
name: 'my-app',
version: '1.0.0',
// level: 'debug' | 'info' | 'warn' | 'error'; format: 'text' | 'json'
logging: { level: 'info', format: 'text' },
plugins: ['./plugins/workers/mod.ts', './plugins/sagas/mod.ts'],
});
// Inside a job handler, the logger comes from the job tools (console-backed today,
// and surfaced under the resource's Console logs view in the Aspire dashboard).
// import { createJobTools } from './job-tools.ts';
const emit = (log) => {
log.info('user provisioned', { userId: 'u_123', source: 'scaffold' });
log.warn('rate limit approaching', { remaining: 4 });
};
// plugins/workers/jobs/health-check.ts — real custom spans inside a handler.
// The dispatcher already wraps this job in traceJobExecution automatically;
// withChildSpan + recordJobProgress let you add detail under that parent span.
import { createSuccessResult, createFailureResult, defineJobHandler } from '@netscript/plugin-workers-core';
import { withChildSpan, recordJobProgress } from '@netscript/telemetry/instrumentation';
const handler = defineJobHandler(async (ctx) => {
const { log } = ctx;
log.info('Starting workers plugin health check');
// Real progress event — 3rd arg is a UNIT label, not a message.
recordJobProgress(1, 2, 'steps');
// withChildSpan opens a real child span under the job-execution span.
const envCheck = await withChildSpan('check.environment', async (span) => {
span.setAttribute('check.name', 'environment');
return { ok: true };
});
recordJobProgress(2, 2, 'steps');
if (!envCheck.ok) return createFailureResult('environment check failed');
return createSuccessResult({ status: 'healthy' });
});
export default Object.assign(handler, { id: 'workers-plugin-health-check' as const });
Endpoints & ports
Telemetry has no single service of its own — it is emitted by every runtime and aggregated by Aspire. These are the real addresses you interact with, validated by the CLI E2E suite.
| Name | Type | Description |
|---|---|---|
http://localhost:18888 |
dashboard |
Aspire dashboard — traces, structured logs, metrics, and resource state for the whole app graph. Auth token printed by aspire start. |
http://localhost:4318 |
OTLP/HTTP |
OTLP ingest endpoint the AppHost configures (aspire.config.json https profile). Runtimes export spans and logs here; the dashboard reads them back. This is the seam you point at a hosted backend. |
GET :8091/health |
liveness |
Workers API health probe — reported as resource health in the dashboard. |
GET :8092/health/live |
liveness |
Sagas API liveness route. |
GET :8093/health |
liveness |
Triggers API health probe (Hono service). |
:8094 |
service |
Auth API (auth-api) — its request spans propagate trace context like any other service. |
View it in Aspire
With aspire start up, the dashboard at http://localhost:18888 gives you four views over the
same telemetry stream — there is no separate tool to configure.
| Name | Type | Description |
|---|---|---|
Resources |
graph |
Live state of every app-graph resource: postgres, redis, workers-api, workers, sagas-api, sagas, triggers-api, triggers, auth-api. Health probes drive the status colour. |
Console logs |
stream |
Per-resource stdout/stderr — the framework logger's text/JSON output lands here in real time. |
Structured logs |
OTLP |
Structured log records exported over OTLP, filterable by resource, level, and attributes. |
Traces |
OTLP |
Distributed traces collected from the OTLP endpoint (http://localhost:4318) — job dispatch/execution/scheduler/task spans and cross-service request spans as trace context propagates. |
Production notes
Reference
This hub is intentionally thin — the full generated API lives in the reference.