Tune the worker runtime

Trade throughput against isolation by tuning four real knobs — worker concurrency, the runner mode, per-task Deno permissions, and timeouts/retries — without touching a single job handler.

The same process-payment handler runs unchanged whether you give it one in-process slot or ten subprocess-isolated workers. Everything below is a deployment setting: it lives in config/official-plugins/mod.ts (defineWorkers(...)), in per-task runtime config, or in an environment variable read by the worker entrypoint. See Background jobs for the handler-authoring side.

Prerequisites

Knob 1 — Default concurrency

concurrency on the workers config is the size of the Web Worker pool the runner spins up: each slot is its own V8 isolate, so raising it buys parallelism at roughly 20–40 MB per isolate. The schema default is 2.

Show the type before the call site:

// config/official-plugins/mod.ts
import { defineWorkers } from '@netscript/plugin-workers-core/config';

export const workers = defineWorkers({
  jobsDir: './workers/jobs',
  tasksDir: './workers/tasks',
  queueProvider: 'auto',
  queueName: 'jobs',
  concurrency: 4, // pool size: 4 isolates → ~80-160 MB. Raise for throughput, lower to bound memory.
  enabled: true,
  groups: [],
});

Knob 2 — Per-topic scaling (concurrency + mode)

Different topics deserve different parallelism. A WorkerGroup binds a queue topic to its own scaling policy, so you can run a hot webhooks topic at concurrency 10 while a heavy reports topic stays at 1. The mode selects whether the worker and scheduler share one runner (combined) or run as separate processes (distributed).

// config/official-plugins/mod.ts
import { defineWorkers } from '@netscript/plugin-workers-core/config';

export const workers = defineWorkers({
  jobsDir: './workers/jobs',
  tasksDir: './workers/tasks',
  queueProvider: 'auto',
  queueName: 'jobs',
  concurrency: 2,
  enabled: true,
  groups: [
    {
      topic: 'webhooks',
      scaling: { mode: 'combined', concurrency: 10 }, // burst-tolerant ingest
      retention: { kvDays: 7, dbDays: 90 },
      jobs: [
        {
          id: 'process-webhook-payload',
          name: 'Process Webhook Payload',
          entrypoint: './process-webhook-payload.ts',
          timeout: 30_000,
          maxRetries: 3,
          tags: ['webhook'],
        },
      ],
    },
  ],
});

Knob 3 — The per-task Deno worker runtime & permissions

A polyglot task (defineTask, or a TaskConfig entry in workers/runtime/) runs as a spawned process. For the deno runtime, the task's permissions object is compiled directly into the child deno run command's --allow-* flags — a deno task with no permissions set falls back to --allow-all, so set them deliberately. The shape mirrors Deno's permission model: each field is boolean (grant/deny all) or a string[] allowlist.

The framework also ships named presets so you do not hand-roll the object. permissions from @netscript/plugin-workers-core exposes minimal, none, network, filesystem, readOnly, subprocess, full, and allAccess.

// workers/tasks/charge-customer.task.ts
import { defineTask } from '@netscript/plugin-workers-core';

// Grant only what the task needs: outbound to Stripe, read its config, nothing else.
export default defineTask('charge-customer')
  .runtime('deno')
  .entrypoint('./charge-customer.ts')
  .timeout(30_000)
  .retry(3)
  .permissions({ net: ['api.stripe.com'], read: ['./config'], env: ['STRIPE_KEY'] })
  .build();

// workers/tasks/sync-report.task.ts
import { defineTask, permissions } from '@netscript/plugin-workers-core';

// 'network' preset = net + env only; spread to tighten one field further.
export default defineTask('sync-report')
  .runtime('deno')
  .entrypoint('./sync-report.ts')
  .timeout(120_000)
  .permissions({ ...permissions.network, read: ['./data'] })
  .build();

// workers/runtime/tasks/v1.0.0.json — declarative, no rebuild
{
  "version": "1.0.0",
  "tasks": [
    {
      "id": "charge-customer",
      "name": "Charge Customer",
      "runtime": "deno",
      "entrypoint": "./charge-customer.ts",
      "timeout": 30000,
      "maxRetries": 3,
      "permissions": { "net": ["api.stripe.com"], "env": ["STRIPE_KEY"] }
    }
  ]
}

Knob 4 — Runner pool size at the process level

The numbers above configure definitions. The actual pool the background process spins up is read from an environment variable when plugins/workers/bin/combined.ts (or worker.ts) starts. That entrypoint reads WORKERS_CONCURRENCY (default 1) and passes it to the Web Worker pool.

# Override the running pool size for the background worker process.
WORKERS_CONCURRENCY=8 deno run -A plugins/workers/bin/combined.ts

Choosing a runner mode

How a handler is isolated is the WORKER_RUNTIMES tunable — three modes, same handler. The scaffold default is the Web Worker isolate path (one V8 isolate per pool slot).

In-production pitfalls

See also

Background jobs

Polyglot tasks

Choose a queue provider

Run a polyglot task

workers