Ent Tracking

Job Tracking

The job tracking subsystem allows you to track and query the current state of a job within Faktory Enterprise.

Configure Tracking

Tracked jobs must have a custom attribute of "track":1. By default, jobs are not tracked.

Setup

The TRACK SET command takes a JSON hash which contains a set of optional attributes for a JID while it is working:

{
  "jid":           "1234567abcdef",         // required
  "percent":       12,                      // optional
  "desc":          "Calculating...",        // optional
  "reserve_until": "2020-08-01T11:43:20Z",  // optional
}

• jid is required since job status is specific to each job
• percent allows the job to provide a "percentage complete" value
• desc allows the job to report its latest checkpoint in a readable form, i.e. a description
• reserve_until allows the job to extend its current reservation until the given RFC3339 timestamp. You cannot make it sooner than the initial reservation so there's no point in setting this unless you know the job is long-running and needs more time.

APIs

• Go: https://github.com/contribsys/faktory/blob/main/client/tracking.go
• Ruby: Faktory::Trackable and Client APIs

Following a Job

The TRACK GET command takes a JID and returns a JSON hash of the attributes above.

> TRACK GET 1234567abcdef
{
  "jid":        "1234567abcdef",
  "state":      "enqueued",
  "updated_at": "2020-01-27T21:10:45Z",
}

Once a job starts working, the worker can call TRACK SET to give tracking updates as job execution proceeds.

> TRACK GET 1234567abcdef
{
  "jid":        "1234567abcdef",
  "percent":    12,
  "desc":       "Calculating...",
  "state":      "working",
  "updated_at": "2020-01-27T21:10:45Z",
}

Once the job is finished, callers can still request the job track for 30 minutes. Note desc and percent will retain the last value from TRACK SET, Faktory does not touch them.

> TRACK GET 1234567abcdef
{
  "jid": "1234567abcdef",
  "percent": 98,
  "desc": "Clearing caches...",
  "state": "success",
  "updated_at": "2020-01-27T21:10:45Z",
}

The Dreaded "unknown" Job State

There are several reasons why a job's state might be unknown:

• The JID is invalid or was never actually enqueued.
• The job was not tagged with the track variable in the job's custom attributes: custom:{"track":1}.
• The job's tracking structure has expired in Redis. It lives for 30 minutes and a big queue backlog can lead to expiration.

Notes

• Job tracks expires 30 minutes after job reservation expiration so clients can still poll for values even after job success/failure.
• Valid job states are "unknown", "enqueued", "working", "success", "failed", and "dead".
• If a job is enqueued but does not start working within 30 minutes, it's possible for the track to expire and further requests to return unknown.