Defining Queues

To define a queue for your app, add your queue to the queues object on the options passed to joystick.app().

Example Usage

import joystick from '@joystick.js/node';

joystick.app({
  queues: {
    example: {
      run_on_startup: true,
      concurrent_jobs: 10,
      jobs: {
        example_job: {
          run: (payload = {}, job = {}) => {
            // Handle work for the job here...
            // Call one of job.completed(), job.failed(), job.requeue(), or job.delete() after work is finished.
          }
        },
      },
    },
  },
  routes: { ... }
});

In Joystick, a queue is a constantly running process (ticking every 300ms) that pulls database entries representing jobs to run. Joystick uses a FIFO (first in, first out) approach to job execution.

Above, we've defined a simple queue called example. On the queues object passed to joystick.app(), we added a key/value pair where the key is the queue name and the value is an object defining its behavior and jobs.

Running Jobs

A job is just an arbitrary function called at a specified time. The job object passed to your run() function includes methods for managing the job’s state:

• job.completed(): Marks the job as successfully completed.
• job.failed(): Marks the job as failed.
• job.requeue(): Schedules the job to run again later.
• job.delete(): Removes the job from the queue.

Cleaning Up Jobs

By default, jobs remain in the database even after completion or failure. To automatically delete them, use the cleanup option:

import joystick from '@joystick.js/node';

joystick.app({
  queues: {
    example: {
      run_on_startup: true,
      concurrent_jobs: 10,
      cleanup: {
        completed_after_seconds: 60,
        failed_after_seconds: 60,
      },
      jobs: {
        example_job: {
          run: (payload = {}, job = {}) => {
            // Work here...
            job.completed();
          }
        },
      },
    },
  },
});

Retrying Jobs After Server Restart

To retry jobs that were running when the server restarted, use retry_jobs_running_before_restart: true.

joystick.app({
  queues: {
    example: {
      retry_jobs_running_before_restart: true,
      jobs: { ... }
    }
  }
});

Setting Max Job Attempts

To limit retries for a job, use max_attempts:

jobs: {
  example_job: {
    max_attempts: 5,
    on_max_attempts_exhausted: (job = {}) => {
      // Handle exhausted attempts
    },
    run: (payload, job) => { ... }
  }
}

Automatically Requeue on Failure

Set requeue_on_failure: true to retry failed jobs automatically.

Preflight Options

Use preflight options to control whether jobs can be added or run.

jobs: {
  example_job: {
    preflight: {
      on_before_add: (job_to_add, db, queue_name) => true,
      okay_to_run: (payload, job) => true,
      requeue_delay_in_seconds: 60,
    },
    run: (payload, job) => { ... }
  }
}

API

Definition

queues: {
  [queue_name]: {
    run_on_startup: boolean,
    concurrent_jobs: integer,
    cleanup: {
      completed_after_seconds: integer,
      failed_after_seconds: integer,
    },
    retry_jobs_running_before_restart: boolean,
    jobs: {
      [job_name]: {
        preflight: {
          on_before_add: (job_to_add, db, queue_name) => boolean,
          okay_to_run: (payload, job) => boolean,
          requeue_delay_in_seconds: integer,
        },
        max_attempts: integer,
        on_max_attempts_exhausted: (job) => void,
        requeue_on_failure: boolean,
        run: (payload, job) => void
      }
    }
  }
}

Parameters

queues object required

An object defining queues and their jobs for background work.

[queue_name] object

A named queue configuration.

run_on_startup boolean

Whether the queue should start automatically when the server starts.

concurrent_jobs integer

The maximum number of jobs to run simultaneously in this queue.

cleanup object

Options for automatically removing completed or failed jobs from the database.

completed_after_seconds integer

Seconds to wait before removing completed jobs.

failed_after_seconds integer

Seconds to wait before removing failed jobs.

retry_jobs_running_before_restart boolean

Retries jobs that were running before the server restarted.

jobs object

An object containing definitions for each job in the queue.

[job_name] object

A job definition.

preflight object

Preflight options for the job.

on_before_add function

Decides if the job can be added to the queue.

okay_to_run function

Decides if the job can be run now.

requeue_delay_in_seconds integer

Seconds to wait before retrying if not ready.

max_attempts integer

Maximum retry attempts before failing permanently.

on_max_attempts_exhausted function

Function to call when max attempts are exhausted.

requeue_on_failure boolean

Requeue failed jobs automatically.

run function required

The function to execute for the job.