Priority Queue Pattern
Process high-priority messages first: priority lanes, separate queues per priority, and ensuring low-priority messages don't starve.
Why Priority Matters
Not all work is equal. A payment confirmation email is more urgent than a weekly newsletter. A premium customer's export job should not wait behind a free-tier user's bulk job. The Priority Queue pattern ensures high-priority messages are processed before low-priority ones, even when the queue has a backlog.
Implementation Approaches
| Approach | How It Works | Pros | Cons |
|---|---|---|---|
| Native priority queue | Broker supports priority field on messages; highest priority delivered first | Simple, single queue | Few brokers support it (RabbitMQ does; SQS, Kafka do not) |
| Separate queues per priority | N queues (high, medium, low); consumers check high first, then lower | Works with any broker; clear separation | More complex consumer logic; harder to share consumers |
| Weighted consumers | Assign more consumer instances to the high-priority queue | Throughput-weighted rather than absolute priority | Low-priority still gets some throughput; may not meet SLA |
| Priority lanes with separate worker pools | Dedicated consumers per queue tier; high-priority pool is larger | Strong isolation; predictable SLAs per tier | Resource overhead of separate pools |
Separate Queues Per Priority (Most Common)
Each consumer implements a polling loop that checks the HIGH queue first. Only if it is empty does it check MEDIUM, and then LOW. This ensures high-priority work is never waiting behind low-priority work, as long as consumers are not fully saturated processing high-priority items exclusively.
async function pollWithPriority(
queues: { high: Queue; medium: Queue; low: Queue }
) {
while (true) {
// Check high priority first
let msg = await queues.high.receive({ waitTimeSeconds: 0 });
// Fall through to medium if high is empty
if (!msg) {
msg = await queues.medium.receive({ waitTimeSeconds: 0 });
}
// Fall through to low if medium is empty
if (!msg) {
msg = await queues.low.receive({ waitTimeSeconds: 20 }); // long-poll
}
if (msg) {
await processMessage(msg);
await msg.queue.acknowledge(msg.receiptHandle);
}
}
}Preventing Starvation
If the HIGH queue is always non-empty, LOW-priority messages may starve indefinitely. Strategies to prevent starvation:
- Age-based promotion — track how long a message has been waiting; promote it to a higher-priority queue after a maximum wait time (e.g., LOW messages older than 10 minutes become MEDIUM)
- Weighted polling — process N high-priority messages per 1 low-priority message rather than checking exhaustively
- Capacity reservation — allocate at least K consumer cycles per period to lower-priority queues regardless of higher queue depth
- SLA monitoring — alert when low-priority queue depth or age exceeds a threshold, triggering emergency capacity expansion
RabbitMQ Native Priority Support
RabbitMQ supports native priority queues via the `x-max-priority` queue argument (1–255). Messages carry a `priority` property; higher-priority messages are dequeued first. The broker maintains separate sub-queues per priority level internally. This is convenient for simple cases but has limits: priorities are per-message, not per-message-type, and very high max-priority values increase memory overhead.
Kafka Does Not Support Message Priority
Kafka is a log, not a traditional priority queue. Messages within a partition are always delivered in offset order. To implement priority with Kafka, use separate topics per priority tier and dedicated consumer groups — consuming the high-priority topic first in your application logic.
Interview Tip
Priority queue questions often arise in the context of designing notification systems, job schedulers, or customer support ticketing. Discuss: how you assign priorities (user tier, message type, business criticality), the starvation risk and age-based promotion to mitigate it, and how you monitor queue lag per priority tier to meet SLA targets.