Add a queue and a cron schedule

Your import job runs one file at a time. A real ERP sync gets bursts — a supplier drops twenty files at once — and needs recurring work that no file triggers, like a nightly full re-sync. This chapter adds the two pieces that make that durable: a queue provider sized for throughput, and a cron schedule that fires on a cadence rather than an event.

1. 1 · Scaffold
2. 2 · Import job
3. 3 · Polyglot transform
4. 4 · Queue & cron
5. 5 · Deploy

What you will build

By the end of this chapter your workers config will name a queue provider and a worker concurrency so bursts of imports drain in parallel instead of one-by-one, and a new scheduled trigger (defineScheduledTrigger) will enqueue a re-sync job on a cron cadence. You will also know the one concurrency naming gotcha to set explicitly so it does not bite you under Aspire.

Before you begin

You need the my-erp/ workspace from Chapter 2 with the import-products job and product-import-trigger working, and aspire start healthy. Confirm the job is registered:

curl 'http://localhost:8091/api/v1/workers/jobs'

Expected: import-products appears in the list. If not, return to Chapter 2 and re-run netscript generate plugins.

Step 1 — Choose a queue provider

A NetScript queue is provider-agnostic: the same job-enqueue path runs on the zero-config Deno KV default on your laptop and on a real broker under Aspire, without touching your job code. The QueueProvider enum has four selectable production backends, and auto-discovery picks one for you when you do not pin one:

For the ERP sync, the right default is to let the environment decide: write provider-neutral config and you get Deno KV locally, and the Aspire cache once it is up — redis by default, or garnet via --cache-backend. The auto-discovery probe order is RabbitMQ → Redis → Deno KV.

Step 2 — Size worker concurrency in config

concurrency on the workers config is the size of the worker pool the runner spins up — each slot is its own V8 isolate (~20–40 MB), so raising it buys parallelism at a memory cost. This is where a burst of imports gets drained in parallel. Set it in the generated worker config:

// config/official-plugins/mod.ts
import { defineWorkers } from '@netscript/plugin-workers-core/config';

export const workers = defineWorkers({
  jobsDir: './workers/jobs',
  tasksDir: './workers/tasks',
  queueProvider: 'auto', // Deno KV locally; the Aspire cache once up (redis default, garnet alt).
  queueName: 'jobs',
  concurrency: 4, // pool size: 4 isolates → ~80–160 MB. Raise for throughput, lower to bound memory.
  enabled: true,
  groups: [],
});

For per-topic control — a hot imports topic at concurrency 10 while a heavy reports topic stays at 1 — use a WorkerGroup with its own scaling: { mode, concurrency }. The full per-topic and runner-mode knobs are in Tune the worker runtime.

Step 3 — Add a cron schedule

Some ERP work is time-driven, not file-driven: a nightly full re-sync, an hourly cleanup of stale staging files. That is a scheduled trigger — defineScheduledTrigger(handler, spec) from @netscript/plugin-triggers-core/builders. Like the file-watch trigger, its handler returns an array of effects; here it enqueues a job on a cron cadence.

// plugins/triggers/scheduled-resync.ts
import { defineScheduledTrigger, enqueueJob } from '@netscript/plugin-triggers-core/builders';
import type { JobDefinition } from '@netscript/plugin-workers-core';

const importProductsJob = {
  id: 'import-products' as JobDefinition<'import-products'>['id'],
  name: 'Import Products',
  topic: 'default',
  entrypoint: './workers/jobs/import-products.ts',
} satisfies JobDefinition<'import-products'>;

export const dailyResyncSchedule = defineScheduledTrigger(
  (event) => Promise.resolve([enqueueJob(importProductsJob, { payload: event.payload })]),
  {
    id: 'daily-resync-schedule',
    cron: '0 6 * * *', // every day at 06:00
    timezone: 'UTC',
    description: 'Runs a full product re-sync every morning.',
    tags: ['resync', 'products', 'scheduled'],
  },
);

export default dailyResyncSchedule;

Step 4 — Register the new trigger

The scheduled trigger has to be picked up by the triggers runtime. Regenerate the registries:

netscript generate plugins

Then restart aspire start (or let it hot-reload) so the triggers processor loads daily-resync-schedule.

Verify your progress

First confirm the config type-checks with the new concurrency and provider settings:

deno task check

Expected: a clean check. Then confirm the cron schedule registered — for a fast feedback loop you can temporarily set its cron to */2 * * * * (every two minutes), regenerate, restart Aspire, wait, and read the executions feed:

curl 'http://localhost:8091/api/v1/workers/executions?limit=10'

Expected: an import-products execution appears on the cron cadence with no file dropped — the schedule, not a file event, enqueued it. Set the cron back to 0 6 * * * when you are done testing.

• [ ] config/official-plugins/mod.ts sets concurrency and queueProvider.
• [ ] deno task check is clean.
• [ ] daily-resync-schedule is registered (after netscript generate plugins + Aspire restart).
• [ ] A scheduled import-products execution appears on the cron cadence with no file dropped.
• [ ] You set WORKERS_CONCURRENCY explicitly (or rely on config concurrency) rather than the ignored Aspire WORKER_CONCURRENCY.

What you built

A workers config that names a queue provider and a worker concurrency so import bursts drain in parallel, and a defineScheduledTrigger cron that enqueues a re-sync on a cadence — plus the knowledge to set concurrency where it actually takes effect. The ERP sync now scales and runs recurring work. The last chapter runs the whole thing under Aspire.