# Follow a job from accepted to finished

Original LockedIn Labs instruction · 40-minute Queue 101 lab · reviewed 21 October 2026.

An engineer asks your application to prepare a maintenance draft. That may take
longer than the original web request. A queue lets the application retain the job
while a separate worker performs it. It is a software component, not a language,
an agent or a promise that the task is finished.

You will follow one fictional job, see why it can arrive twice, and run real local
database writes to compare lost work, duplicated work and a reused result. No
account, cloud resource, model or payment is required.

## 1. Meet the message, worker and acknowledgement

Think of a work-order desk: accepting an order, assigning it to someone and
finishing it are different events. In software, a producer submits a message to
a queue. A worker receives that message and does the work. An acknowledgement
tells the queue that this delivery has been handled.

| Term | What it means here | Familiar comparison |
| --- | --- | --- |
| Message | The instruction to prepare draft J-12 | A work order |
| Queue | Storage and delivery of pending messages | The waiting tray |
| Worker | A program that handles a message | The person doing the job |
| In flight | Delivered, with acknowledgement still pending | Checked out of the tray |
| Visibility period or lease | A bounded interval before work may become available again | A temporary assignment |
| Acknowledgement | Confirmation to the broker about this delivery | Marking the order handled |
| Operation key | Stable identity of the intended business action | The same order number across attempts |

The example key is `north / J-12:draft:v1`. The tenant, job and action stay the
same when another worker retries. The delivery receipt changes. A receipt for a
delivery and a receipt proving a business effect are different records.

Follow the visual one event at a time. Before advancing, predict both the message
state and the number of saved drafts. A queue cannot infer a database commit from
silence. A worker can stop after committing the draft but before acknowledging.

## 2. Run four delivery cases

Download [the Python lab](/templates/fde-queue-delivery-lab.py) into a new working
folder. You need Python 3.11 or later. It uses only the standard library.

```sh
python3 --version
python3 fde-queue-delivery-lab.py test
python3 fde-queue-delivery-lab.py demo
```

The program creates separate temporary SQLite databases and removes them when it
finishes. Each event reads committed state through a new database connection.
The queue is a small teaching model: expiry uses explicit integer ticks and the
interruption is an omitted acknowledgement. No worker is killed and no vendor
broker is contacted. The database writes and transaction rollbacks are real.

Make predictions first. Then inspect the final event in each case:

| Case | Deliveries | Saved effects | Explanation |
| --- | --- | --- | --- |
| `normal` | 1 | 1 | Write the draft, then acknowledge. |
| `duplicate-effect` | 2 | 2 | Write, omit acknowledgement, then write again on redelivery. |
| `reuse-effect` | 2 | 1 | Recognize the same action and reuse its committed result. |
| `early-ack` | 1 | 0 | Acknowledge first, then stop before the draft exists. |

Notice that both normal and early-ack end with a settled message. A settled queue
entry alone cannot demonstrate that the intended business action happened. Keep
the trace lines that establish each conclusion.

## 3. Make the business action safe to repeat

Idempotency means repeating an operation has the same intended effect as doing it
once. Here the safe path writes the effect and its receipt in one SQLite
transaction. On a repeat, it checks the same tenant, operation key and payload,
then reuses the recorded outcome. A changed payload under the same key is a
conflict; a different action needs a deliberately different key.

Read `apply()` in the Python file. `BEGIN IMMEDIATE` starts the write transaction.
The receipt and draft commit together. An exception between them rolls both
back. In `test_receipt_rolls_back_with_effect`, the next attempt can still create
the draft because no false success receipt remains.

Experiment in your own copy:

1. In `run_case`, change the second call to use `operation="J-12:draft:v2"`.
   Predict why the safe replay now creates a second effect. Run `demo`.
2. Restore it. Change the second call to `payload="different draft"` instead.
   Predict and observe `operation_conflict` in the safe case. The refusal is
   intentional: it must not silently call changed work a repeat.
3. Restore the original. Explain why `receive(now=4)` refuses a new delivery but
   `receive(now=5)` permits one in this model. Inspect the boundary test.
4. Draw the two commits separately: the business effect, then acknowledgement.
   Mark where the interruption occurs and which evidence the next worker reads.

This model checks the current delivery receipt at acknowledgement time. It does
not cancel an old process or fence a remote service. A real service must enforce
authorization again at its write boundary and bind replay to the intended input.
For an external action such as sending an email, a local receipt and the remote
effect do not share a SQLite transaction. Use the destination's supported
idempotency and reconciliation contract; an unknown outcome is not permission to
send again blindly.

## 4. Recognize the cloud vocabulary

These are related delivery concepts, not interchangeable APIs. Start with the
provider's documented delivery mode and client behavior before choosing retries.

| Service | Temporary delivery state | Successful handling |
| --- | --- | --- |
| Amazon SQS standard queue | Visibility timeout | Delete the handled message |
| Google Cloud Pub/Sub pull subscription | Outstanding message with an acknowledgement deadline | Acknowledge the message on that subscription |
| Azure Service Bus Peek-Lock | A message lock | Complete the message |

SQS standard delivery can produce duplicates even during a visibility period.
Pub/Sub subscriptions have their own delivery and acknowledgement settings.
Service Bus also offers Receive-and-Delete, which has different loss behavior
from Peek-Lock. Never translate the local lab into an exactly-once claim about a
cloud service or an external business effect.

For your chosen service, record the delivery mode, acknowledgement operation,
expiry behavior, bounded retry policy and the owner of repeatedly failing work.
A dead-letter destination holds messages for investigation; it does not repair
them. Ordering, retention, permissions and operational alerts need explicit
decisions too.

Official references checked 21 October 2026:

- [Amazon SQS visibility timeout](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-visibility-timeout.html).
- [Google Cloud Pub/Sub subscriptions and acknowledgement](https://docs.cloud.google.com/pubsub/docs/subscription-overview).
- [Azure Service Bus locks and settlement](https://learn.microsoft.com/en-us/azure/service-bus-messaging/message-transfers-locks-settlement).
- [SQLite transaction behavior](https://www.sqlite.org/lang_transaction.html).

## 5. Explain it to the receiving engineer

Keep your four predictions, the actual traces, one changed-key experiment and a
diagram showing effect before acknowledgement. Explain the result without using
the phrase “the queue handles it.” Name exactly which component recognizes a
repeat and what evidence it stores.

Transfer challenge: the draft is now an email sent through another company's
API. Your worker times out after submitting it. State what is unknown, how you
would query or reconcile the original operation, and when a human must decide.
The local transaction is evidence for a local write only.

Continue into the [worker recovery lab](/training/guides/agent-engineering#1c-recover-an-abandoned-worker-lease)
for actual process interruption, lease fencing and competing workers, or open
the [guided Queue 101 lesson](/app/academy?course=agent-engineering&lesson=queues-first-delivery&view=focus)
with your existing training access.