Version v1.x · .NET

Runtime Command Control

Use hosted runtime commands to run schedules now, pause or resume jobs, and update cron safely.

Runtime Command Control

DurableStack v1.2.x supports hosted runtime command control for recurring schedules.

This lets operators issue commands from the hosted platform, and lets worker runtimes apply them safely through the runtime-control sync loop.

How it works

  1. Worker runtime syncs with hosted control (/v1/runtime/control/sync) on an interval.
  2. Worker uploads schedule snapshot and local command receipts.
  3. Hosted API returns pending commands for that tenant.
  4. Worker leases each command locally, executes the schedule admin operation, and writes a success/failure receipt.
  5. Next sync uploads receipts so command history in the app reflects execution outcome.

This model is resilient to transient outages and avoids duplicate command execution across multiple worker replicas.

Supported command operations

  • Run schedule now
  • Pause schedule
  • Resume schedule
  • Update cron expression and time zone

In runtime terms, these map to schedule-admin operations:

  • run_schedule_now -> RunScheduledJobNowAsync(...)
  • set_schedule_enabled (false) -> pause
  • set_schedule_enabled (true) -> resume
  • update_schedule_cron -> UpdateScheduledJobCronAsync(...)

Required configuration

Runtime command control uses eventing credentials and endpoint settings:

  • Eventing.TenantId
  • Eventing.ClientSecret
  • Eventing.IngestionApiBaseUrl

Command sync defaults are enabled and can be tuned:

  • Eventing.RuntimeControlEnabled (default true)
  • Eventing.RuntimeControlSyncPath (default /v1/runtime/control/sync)
  • Eventing.RuntimeControlSyncIntervalSeconds (default 5)
  • Eventing.RuntimeControlMaxReceiptUpload (default 200)
  • Eventing.RuntimeControlCommandLeaseDurationSeconds (default 30)

Example:

builder.Services.AddDurableStackPostgres(connectionString, options =>
{
    options.Eventing.TenantId = "tenant_...";
    options.Eventing.ClientSecret = "secret_...";
    options.Eventing.RuntimeControlEnabled = true;
    options.Eventing.RuntimeControlSyncIntervalSeconds = 5;
});

Safety and behavior guarantees

  • Commands are applied by workers that currently sync for the tenant.
  • Local command lease guards against duplicate execution by multiple replicas.
  • Unknown/missing schedules return a failed receipt (schedule_not_found) instead of silent success.
  • Invalid payloads return failed receipts (invalid_payload).
  • Unsupported command types return failed receipts (unsupported_command_type).

Practical notes

  • In local development, hosted control commands require the worker to reach the hosted/API endpoint configured in Eventing.IngestionApiBaseUrl.
  • If webhook-like inbound networking is unavailable in your dev setup, runtime control still works because sync is outbound from worker to API.
  • Keep worker names unique per process/container for clean operational diagnostics.