Choose a queue provider

Scope: how to pick the right backend for @netscript/queue — the in-memory test adapter, the zero-config Deno KV default, Redis, RabbitMQ (AMQP), and PostgreSQL — and how to either let auto-discovery select one or pin one explicitly with provider + connection. This is the decision recipe; for the enqueue/consume/cron mechanics see Queue / KV / cron.

A NetScript queue is provider-agnostic by design: the same createQueue("jobs") runs on an in-memory adapter in a unit test, on Deno KV on your laptop, and on a real broker under Aspire — without touching the call site. The only thing that changes is which backend the factory resolves. This recipe is about making that choice deliberately.

Prerequisites

The five backends at a glance

There are four selectable production providers in the QueueProvider enum — Deno KV, Redis, RabbitMQ, and PostgreSQL — plus the MemoryQueueAdapter from @netscript/queue/testing for tests and examples. Pick by the row that matches your deployment, then jump to the matching step.

Step 1 — Default: let auto-discovery pick

If you pass no provider, the factory probes the Aspire environment and selects the first available backend in a fixed order. Write provider-neutral code and let the environment decide:

// queue.ts
import { createTypedQueue } from "@netscript/queue";
import { z } from "zod";

const JobSchema = z.object({ id: z.string(), kind: z.string() });

// No provider → auto-discovery. Probe order: RabbitMQ → Redis → Deno KV.
export const jobs = createTypedQueue("jobs", JobSchema);

The probe order is RabbitMQ (AMQP) → Redis → Deno KV. Deno KV is the always-available fallback when no broker is discoverable, so the same code runs unchanged from laptop to production as you provision real backends under Aspire.

Step 2 — Pin a provider explicitly

To take auto-discovery out of the loop, pass provider from the QueueProvider enum. Connection details go under connection.<provider>; every URL is optional and falls back to Aspire service discovery when omitted.

// queue.ts
import { createQueue, QueueProvider } from "@netscript/queue";

const jobs = createQueue("jobs", {
  provider: QueueProvider.Redis,
  connection: {
    redis: {
      // url optional → falls back to getRedisConnectionFromEnv() / Aspire.
      url: Deno.env.get("REDIS_URL"),
      // options are passed through to ioredis.
      options: {},
    },
  },
});

// queue.ts
import { createQueue, QueueProvider } from "@netscript/queue";

const jobs = createQueue("jobs", {
  provider: QueueProvider.RabbitMQ,
  connection: {
    rabbitmq: {
      // url optional → derived from Aspire's rabbitmq service (amqp://).
      url: Deno.env.get("AMQP_URL"),
      // queueName defaults to the queue's logical name.
      queueName: "jobs",
    },
  },
});

// queue.ts
import { createQueue, QueueProvider } from "@netscript/queue";

const jobs = createQueue("jobs", {
  provider: QueueProvider.DenoKv,
  connection: {
    denoKv: {
      // path optional → Aspire discovery or the default shared Deno KV.
      path: Deno.env.get("DENO_KV_URL"),
      verbose: false,
    },
  },
});

Step 3 — Select the PostgreSQL backend

The PostgreSQL provider gives you a SQL-durable queue with row-claim semantics (FOR UPDATE SKIP LOCKED), a visibility timeout, ack/nack, and a dead-letter store — so concurrent consumers never double-claim a message. Because it is never auto-discovered, you must name it:

// queue.ts
import { createQueue, QueueProvider } from "@netscript/queue";

// provider: QueueProvider.Postgres is the ONLY way to select this backend.
const jobs = createQueue("jobs", {
  provider: QueueProvider.Postgres,
  connection: {
    postgres: {
      // url optional → falls back to Aspire's getPostgresUri().
      url: Deno.env.get("DATABASE_URL"),
      // tableName defaults to 'message_queue'.
      tableName: "message_queue",
    },
  },
});

await jobs.enqueue({ id: "job-1", kind: "reindex" });

Reach for Postgres when you already run it under Aspire and want the queue and your application data in one transactional store, rather than standing up a dedicated broker.

Step 4 — Use the in-memory backend in tests

For unit tests and examples, construct the MemoryQueueAdapter directly from the testing sub-path. It is volatile and in-process — nothing to provision, and it is never selected by auto-discovery, so production code paths stay untouched:

// jobs_test.ts
import { MemoryQueueAdapter } from "@netscript/queue/testing";

const queue = new MemoryQueueAdapter<{ id: string }>();

await queue.enqueue({ id: "job-1" });
await queue.listen(async (message) => {
  // assert on `message` here
});

Step 5 — Add concurrency with createParallelQueue

Any provider can be wrapped for concurrent processing with createParallelQueue. Pass a concurrency greater than 1 and a single listener processes that many messages at once — ideal for I/O-bound work (HTTP calls, DB queries). At concurrency: 1 (the default) it is identical to createQueue.

// queue.ts
import { createParallelQueue, QueueProvider } from "@netscript/queue";

// 8 messages processed concurrently on one listener, pinned to Redis.
const queue = createParallelQueue("notifications", {
  concurrency: 8,
  provider: QueueProvider.Redis,
});

await queue.listen(async (message) => {
  await deliver(message); // these run in parallel
});

For CPU-bound work, prefer Web Workers over queue concurrency — see Tune the worker runtime.

In-production pitfalls

See also

KV, queues & cron — the concept-level tour of queues, KV, and cron, and how Aspire wires the four queue backends.

Background jobs — when you want a managed worker job (persistence, retries, an HTTP trigger) instead of a raw queue.

Queue / KV / cron — the mechanics recipe: enqueue, consume, and schedule. This page pairs with it: choose the backend here, run it there.

For the full QueueProvider enum, the QueueConnectionOptions shape, createParallelQueue / ParallelQueueOptions, the MessageQueue interface, and the QueueError hierarchy, see

queue .