Telemetry & logging

The most expensive turn in any change loop — whether the author is you or an AI agent — is the verification turn: did the change actually work, across every process it touched? NetScript closes that loop inside the framework. The runtimes emit real OpenTelemetry spans on their own, a single W3C traceparent groups a cross-process flow into one distributed trace, and a typed query surface reads the resulting spans back — so "did it work" is a trace you open (or query programmatically), not four console logs you correlate by eye.

Concretely, observability is a built-in, not a bolt-on. Every service and plugin runtime is wired for OpenTelemetry — services serve their RPC handlers with trace-context propagation (first-party oRPC and Hono instrumentation), the worker runtime wraps job dispatch, execution, scheduling, subprocess hand-off, and task execution in real OTel spans, and structured logs flow through the framework logger. The viewing surface is the Aspire dashboard at https://localhost:18888, which collects OTLP traces, metrics, and structured logs from every resource in the app graph (services, plugin APIs, background processors) the moment you run aspire start. You do not stand up Jaeger, Grafana, or a log shipper to get started — the AppHost provisions the OTLP collector and the dashboard for you.

This page is the capability hub: what telemetry exists, how to emit it, where to view it, and the one place that is still a scaffold stub. For the full generated API of each unit, follow the reference links: the telemetry primitives live at /reference/telemetry/ and the structured logger at /reference/logger/.

The story: one grouped trace, end to end

Picture the failure mode that eats an afternoon. A trigger fires, the scheduler dispatches, a queue hands the job to a worker, the job reports success — and the downstream callback never lands. With per-process logs you tail one console per resource and correlate timestamps by eye; an AI agent in the same position burns its turns asking you to check logs it cannot see.

In NetScript that whole flow is one grouped trace: trigger → scheduler dispatch → queue enqueue/dequeue → job.execute → the SDK's rpc.client callback span, all under a single trace id, because the runtimes propagate W3C trace context across every process boundary for you. Where many producers feed one consumer — a fan-in into a durable-streams processor — createFanInLinks attaches span links from each producer trace instead of re-parenting, so the consumer span shows exactly which upstream flows fed it. And this shape is not aspirational: the grouped cross-process trace is exercised by a non-mocked merge gate in the framework's own CI, which scaffolds a real project, runs the flow under Aspire, and reads the live spans back before a change can land.

The same spine covers AI workloads. Spans follow the upstream gen_ai.* semantic conventions (via createGenAiAttributes in @netscript/telemetry/attributes), so a traced agent turn shows the model call, the tool calls, and the durable write it produced — step by step, on one tree.

What it is

Telemetry in NetScript is three OpenTelemetry signals — traces, metrics, and logs — wired through one package, @netscript/telemetry, and one viewing surface, the Aspire dashboard. Traces are the headline: a request enters a service, the worker runtime dispatches and executes a job, a subprocess runs a polyglot task, and a saga advances — and because a single W3C traceparent header propagates across every one of those process boundaries, the whole fan-out collapses into one trace tree. Metrics (worker counts, SSE connection stats) and structured logs ride the same OTLP export. The framework owns transport (OTLP/HTTP to the collector) and viewing (the dashboard); it now owns most of emission too — the worker, scheduler, queue, saga, and SSE runtimes are instrumented for you. The full mental model — what is framework-real versus a scaffold stub — is in Observability.

Under the hood, @netscript/telemetry is structured as telemetry ports and adapters: application and runtime code programs against port contracts (TracerProviderPort, MeterPort, PropagatorPort, SpanLinkPort, TelemetryQueryPort — the @netscript/telemetry/otel subpath), with a zero-dependency default adapter (OtelDenoTracerProvider, bound to Deno's built-in OTLP exporter via OTEL_DENO=true) and an opt-in OpenTelemetry-SDK adapter (OtelSdkTracerProvider) for apps that bring the JS SDK — which also unlocks attribute-preserving span links. Attribute names follow one convention: upstream semconv keys (rpc.*, messaging.*, gen_ai.*, server.*) wherever OpenTelemetry defines them, and NetScript-owned keys under the single proprietary root netscript.* — correlated spans share netscript.correlation.id. The full rule set (span naming, SpanKind, status, propagation) is the telemetry convention.

Learn → / Do →

What you get out of the box

Telemetry in NetScript spans three layers — emission (in framework runtimes and your handler code), transport (OTLP over HTTP to the collector), and viewing (the Aspire dashboard). The framework owns transport and viewing, and it now owns most of emission too: the worker runtime traces the entire job lifecycle automatically. The only thing you still wire by hand is custom spans inside a job handler — and even there the @netscript/telemetry helpers do the work.

Enable tracing & see a span

Tracing turns on from the environment — @netscript/telemetry/config resolves a TelemetryConfig from the standard OTEL_* variables, and the Aspire AppHost already sets them for you. The fastest path is to run aspire start, trigger any job, and the runtime's automatic spans show up in the dashboard with no code change. The tab below adds a custom span on top of that automatic trace.

// netscript.config.ts — declare the structured-logging contract for the workspace.
// Tracing itself is enabled by OTEL_* env vars (the Aspire AppHost sets these); the
// config package reads them — there is no defineTelemetry() call to make.
import { defineConfig } from '@netscript/config';

export default defineConfig({
  name: 'my-app',
  version: '1.0.0',
  logging: { level: 'info', format: 'text' }, // level: debug|info|warn|error
  plugins: ['./plugins/workers/mod.ts'],
});

// Anywhere in app code you can inspect what telemetry resolved (log-safe, redacted):
import { describeTelemetryConfig, isTelemetryEnabled } from '@netscript/telemetry/config';
if (isTelemetryEnabled()) console.log(describeTelemetryConfig());
// → { enabled: true, endpoint: 'http://localhost:4318', protocol, serviceName, sampler }

// plugins/workers/jobs/health-check.ts — a real child span inside a handler.
// The dispatcher already wraps this job in traceJobExecution automatically;
// withChildSpan opens a child span under that parent. recordJobProgress's 3rd arg
// is a UNIT label (e.g. 'steps'), not a free-text message.
import { createSuccessResult, defineJobHandler } from '@netscript/plugin-workers-core';
import { withChildSpan, recordJobProgress } from '@netscript/telemetry/instrumentation';

const handler = defineJobHandler(async (ctx) => {
  recordJobProgress(1, 2, 'steps'); // → job.progress event on the trace

  const envCheck = await withChildSpan('check.environment', async (span) => {
    span.setAttribute('check.name', 'environment');
    return { ok: true };
  });

  recordJobProgress(2, 2, 'steps');
  return createSuccessResult({ healthy: envCheck.ok });
});

export default Object.assign(handler, { id: 'workers-plugin-health-check' as const });

Key types: the telemetry init/config surface

Configuration is read from the environment, not constructed — @netscript/telemetry/config resolves and caches a TelemetryConfig from the OTEL_* variables. These are the confirmed fields and the helpers that read them.

Span & instrumentation helpers

Two subpaths cover hand-written instrumentation. @netscript/telemetry/tracer is the low-level facade over @opentelemetry/api — get a tracer, open a span, run a callback inside one. @netscript/telemetry/instrumentation is the NetScript-domain layer the runtimes themselves use — job, scheduler, queue, and SSE spans with the right attributes baked in. The /context subpath holds the traceparent propagation helpers that make the cross-boundary trace in the diagram above possible.

Service, RPC & HTTP tracing

Services opt into trace propagation through first-party instrumentation rather than by hand — NetScript wraps the upstream OTel packages instead of reinventing them. The @netscript/telemetry/orpc subpath ships an oRPC tracing plugin (backed by the upstream @orpc/otel instrumentation) that opens a SERVER span per RPC and continues any inbound traceparent, plus an error-handling plugin that classifies failures. The @netscript/telemetry/hono subpath does the same for plain HTTP: createHonoTracingMiddleware wraps Hono's first-party @hono/otel middleware and layers NetScript service naming and W3C propagation on top — the service builder registers it outermost for you. On the service builder the RPC side is the .withRPC({ traceContext: true }) toggle (see Services); the factories below are the underlying surface if you mount oRPC or Hono yourself.

What is traced automatically

Before you write a single instrumentation call, the worker runtime already produces a usable trace tree in Aspire. The @netscript/telemetry instrumentation is wired into the dispatcher, the executor, and the scheduler, so the moment a job runs you get spans with attributes, durations, status, and lifecycle events — no scaffold changes required.

Instrument a handler

The two tabs below show the two emission paths a developer touches: structured logging through the framework logger, and custom spans inside a job handler. For custom spans, call the @netscript/telemetry helpers (traceJobExecution, withChildSpan, recordJobProgress) directly — they are the real, supported surface. Read the callout under the tabs to understand why you reach for the telemetry package rather than the scaffold's createJobTools(ctx) trace helpers.

// netscript.config.ts — declare the logging contract for the whole workspace.
import { defineConfig } from '@netscript/config';

export default defineConfig({
  name: 'my-app',
  version: '1.0.0',
  // level: 'debug' | 'info' | 'warn' | 'error'; format: 'text' | 'json'
  logging: { level: 'info', format: 'text' },
  plugins: ['./plugins/workers/mod.ts', './plugins/sagas/mod.ts'],
});

// Inside a job handler, the logger comes from the job tools (console-backed today,
// and surfaced under the resource's Console logs view in the Aspire dashboard).
// import { createJobTools } from './job-tools.ts';
const emit = (log) => {
  log.info('user provisioned', { userId: 'u_123', source: 'scaffold' });
  log.warn('rate limit approaching', { remaining: 4 });
};

// plugins/workers/jobs/health-check.ts — real custom spans inside a handler.
// The dispatcher already wraps this job in traceJobExecution automatically;
// withChildSpan + recordJobProgress let you add detail under that parent span.
import { createSuccessResult, createFailureResult, defineJobHandler } from '@netscript/plugin-workers-core';
import { withChildSpan, recordJobProgress } from '@netscript/telemetry/instrumentation';

const handler = defineJobHandler(async (ctx) => {
  const { log } = ctx;
  log.info('Starting workers plugin health check');

  // Real progress event — 3rd arg is a UNIT label, not a message.
  recordJobProgress(1, 2, 'steps');

  // withChildSpan opens a real child span under the job-execution span.
  const envCheck = await withChildSpan('check.environment', async (span) => {
    span.setAttribute('check.name', 'environment');
    return { ok: true };
  });

  recordJobProgress(2, 2, 'steps');
  if (!envCheck.ok) return createFailureResult('environment check failed');
  return createSuccessResult({ status: 'healthy' });
});

export default Object.assign(handler, { id: 'workers-plugin-health-check' as const });

Endpoints & ports

Telemetry has no single service of its own — it is emitted by every runtime and aggregated by Aspire. These are the real addresses you interact with, validated by the CLI E2E suite.

View it in Aspire

With aspire start up, the dashboard at https://localhost:18888 gives you four views over the same telemetry stream — there is no separate tool to configure.

Close the loop: read the trace back in code

The dashboard answers "what happened and where" for a human. The same telemetry has a typed, in-language read side for programs — and for an AI agent verifying its own change without a browser. @netscript/telemetry/query publishes a TelemetryQueryPort contract with an Aspire-backed reader: createAspireTelemetryQuery(...) (or createTelemetryQuery(...) for the port type) reads the Aspire /api/telemetry/* surface into package-owned read models — TelemetryTrace, TelemetrySpan (with its events and links), TelemetryLog, TelemetryResource, TelemetryMetric. So the verification turn becomes an assertion: fetch the trace by id, check that the span you expected is present with the status you expected — no console to tail, no screenshot to read.

Production notes

Reference

This hub is intentionally thin — the full generated API lives in the reference.

telemetry