Bad event handling can turn one small failure into stale listings, duplicate owner records, and wasted sales time. In real estate pipelines, the fix is usually simple: retry only short-term failures, space retries with backoff and jitter, use idempotency keys to block duplicate writes, and send bad events to a dead-letter queue instead of dropping them.
I’d boil the article down to this:
- Retry short-term failures only like timeouts, rate limits, and 5xx errors
- Do not retry permanent failures like missing fields, bad IDs, and schema errors
- Use backoff with jitter so thousands of retries do not hit at once
- Use idempotency keys so the same event does not write twice
- Split main queues, retry queues, and DLQs so failed events do not block new work
- Set stop rules and owners so stuck events get fixed by the right team
The business cost is hard to ignore. U.S. companies lose about $600 billion each year to bad data, and 62% say 20%–40% of their records are incomplete or wrong. In real estate, that can mean a listing marked “Closed” in one system but still “Active” in another, or a missed [phone_verification_result](https://batchdata.io/phone-number-verification) that leaves outreach running on bad numbers.
Here’s the short version of what works:
| Area | What I’d do | Why it matters |
|---|---|---|
| Error handling | Retry transient errors, fail fast on permanent ones | Stops wasted queue traffic |
| Timing | Use exponential backoff plus 20%–30% jitter | Prevents retry spikes |
| Duplicate control | Use a stable idempotency key per logical event | Stops double writes |
| Queue design | Separate main, retry, and dead-letter flows | Keeps new events moving |
| Ownership | Assign each DLQ by event type | Makes recovery clear |
If you run property enrichment tasks like listing updates, owner syncs, or phone checks, this approach helps cut silent data loss before cleanup turns into bulk re-runs and repair scripts.
Building a Reliable, Secure and Efficient Event Ingestion Pipeline
sbb-itb-8058745
The Problem: Where Real Estate Events Fail and Why Manual Fixes Fall Short
Real estate events tend to break at integrations, queues, or database writes. When that happens, retries help stop a failure from turning into silent data loss. The answer is controlled retries, not manual cleanup.
Transient Failures vs. Permanent Failures
You should retry only failures that can recover.
Transient failures include network timeouts, HTTP 429 rate-limit responses, temporary database locks, and 5xx server errors. These often clear up on their own. Send the same request again after a short delay, and it may go through.
Permanent failures are a different story. A malformed parcel ID, a missing required field, a schema mismatch between producer and consumer, or an event that points to a missing target record will fail every single time, no matter how often you retry it.
That split shapes the whole retry policy. Retrying a timeout makes sense. Retrying a payload with an invalid identifier does not. It just clogs the queue and hides the actual issue. A good pipeline sends these two failure types down different paths from the start.
| Failure Type | Common Examples | Correct Response |
|---|---|---|
| Transient | Timeouts, 429 rate limits, 5xx errors, DB lock contention | Bounded retries with exponential backoff |
| Permanent | Invalid IDs, missing fields, schema mismatches, bad data types | Fail fast, route to dead-letter queue, fix at source |
How Property Updates, Owner Syncs, and Phone Verification Events Break
Each event type fails in its own way.
A property status update sent to a search index or analytics service may time out during heavy load. The update times out before the service confirms receipt, so the property still shows as "active" even though the transaction already closed.
An owner sync between a deal management system and a marketing platform can fail halfway through during a short network issue. That leaves the destination system with stale contact data.
A phone verification event sent to a third-party API during a high-volume outreach push may get a 429 response. Without retries, that event disappears, leaving numbers unverified and increasing compliance risk.
Why Re-Runs and Bulk Reloads Add Risk
Manual replay jobs often replay stale snapshots. If a team reruns a nightly property update job after finding a failure, any price cuts, status changes, or ownership transfers recorded after the original run can be overwritten by older data.
Bulk reloads of owner and contact records bring the same kind of risk. Without strong de-duplication and idempotency, a reload can create multiple owner records for the same person. Sometimes the only difference is phone formatting or a shortened name. That splits communication history across duplicate profiles.
As AWS guidance notes, APIs with side effects are not safe to retry unless idempotency is enforced, because without it, the same side effect can execute twice. Manual replays work on batches, not on the failed events themselves, so they can overwrite good data or create duplicates. At scale, one bad replay can damage records that were already correct.
That is why the next step is retry rules, backoff timing, and idempotency keys.
The Solution: Retry Rules, Backoff Timing, and Idempotency to Stop Silent Data Loss
Real Estate Event Retry Blueprint: Stop Silent Data Loss
Three controls work together to stop silent data loss: policy-based retry rules, exponential backoff with jitter, and idempotency keys. Used together, they help events recover cleanly while keeping data intact.
Set Retry Rules by Error Type and Event Type
Not every event should get the same retry budget. A property status update coming from real estate datasets like an MLS feed is high-volume and time-sensitive, so 3–5 retries over about 5–15 minutes makes sense. An owner sync that pushes enriched contact data into a CRM usually deals with stricter API quotas, so 3 retries over 30 minutes is a safer limit. A phone verification call to an external API is billed per request and rate-limited, so keeping retries to **1–2 attempts with short delays – 30 seconds, then 2 minutes – ** helps control cost and throttling risk.
Permanent errors should skip retries altogether. A 400, 404, or 422 response from a verification API, missing required fields, or a schema validation failure should go straight to a dead-letter queue. Retrying those events just burns capacity and makes the root issue harder to spot.
| Event Type | Max Retries | Retry Window | Notes |
|---|---|---|---|
| Property update (MLS/county feed) | 3–5 | 5–15 minutes | High volume; short outages are common |
| Owner record sync (CRM/marketing) | 3 | 30 minutes | Stricter API quotas; consistency priority |
| Phone verification (external API) | 1–2 | ~2 minutes | Billed per call; rate limits are tight |
Use Exponential Backoff With Jitter
Fixed-interval retries are a bad bet. If thousands of property update events fail during a short MLS outage and all retry at the exact same second, the feed gets slammed the moment it comes back. Exponential backoff with jitter fixes that by spacing retries out and randomizing each delay.
A property update might retry after 30 seconds, 5 minutes, and 1 hour, with 20%–30% jitter added to each delay. That spreads load across time instead of piling it into one moment. Route retries through delayed queues so workers can stay responsive when traffic jumps.
Backoff manages load. Idempotency handles duplicates.
Prevent Duplicate Writes With Idempotency Keys
When an event gets retried, the consumer may process it more than once. Without idempotency, that can mean duplicate property records, repeated owner writes, or double charges on phone verification calls.
The fix is simple: use a stable key for one logical event. For property updates, combine property_id with an event_version or source_timestamp. For example, a price change from $325,000 to $319,000 recorded at 2026-07-30T15:42:00-05:00 should have one key that stays the same across retries. For owner syncs, pair owner_id with a sync_timestamp or sync_batch_id. For phone verification, use a verification_request_id for the session.
Store each key with the operation result in a persistent store. If the event shows up again, return the stored success. If it failed in a permanent way, send it to dead-letter handling. That stops duplicate writes when the same event arrives late or gets delivered more than once.
Queue Handling: Retry Queues, Dead-Letter Queues, and When to Stop
Retry rules and backoff timing only do their job if your queue setup backs them up. If everything flows through one path, a burst of failing events can clog the line for new work. And once that happens, it gets a lot harder to spot what broke and why.
Separate Main Queues From Delayed Retry Queues
A clean setup uses a main queue for new, first-pass events and one or more delayed retry queues for controlled reprocessing. Producers like listing ingestion services, owner sync jobs, and phone verification handlers send new events to the main queue.
If a consumer runs into a transient error, it should republish the same event to a delayed retry queue. That retry should include an incremented retry count and keep the original idempotency key.
The same consumer logic should process both paths. If the retry path applies different validation rules or skips idempotency checks, you can end up with mismatched outcomes that are painful to trace. Some platforms use a dedicated retry topic before sending exhausted messages to a dead-letter topic, with a separate service used to inspect or replay them after the backend comes back online.
When ordering matters at the entity level – like a stream of property updates for the same parcel – partition by property_id or owner_id. That keeps events for the same record in order, even across retry attempts.
Move Non-Recoverable Events to a Dead-Letter Queue
A dead-letter queue (DLQ) is where events go when automated retries won’t fix the problem. Usually, that means the message hit its retry budget or the error is plainly permanent.
AWS recommends enabling a DLQ on SQS queues to help prevent message loss. Azure Service Bus dead-letters messages after MaxDeliveryCount is exceeded, which defaults to 10 delivery attempts, and records the reason and error description. Google Pub/Sub uses a default of 5 maximum delivery attempts before routing to a dead-letter topic, with a configurable ceiling of 100.
| Destination | Purpose | Common Error Type | Impact on Data Loss |
|---|---|---|---|
| Retry Queue | Hold failed events for delayed reprocessing | Transient timeouts, throttling, short outages | Low risk; events are preserved and retried |
| Dead-Letter Queue | Isolate events that cannot recover automatically | Validation failures, permanent errors, business-rule violations | Moderate risk if unmonitored; events are safe but need follow-up |
| Discard | Drop the event entirely | Duplicate, corrupted, or malicious events | Highest risk; should be rare and tightly controlled |
In real estate pipelines, discard should be rare. If a phone verification event comes back as "invalid phone number" from a phone verification API such as BatchData’s, that event belongs in the DLQ – not the trash. Why? Because the contact record may need to be fixed before outreach can continue.
Define Stop Conditions and Assign Recovery Ownership
Retries need a hard stop. The clearest stop conditions are:
- The retry budget is exhausted
- A validation failure is confirmed – such as a missing parcel ID or an invalid ZIP code that fails schema checks on every attempt
- A business rule violation is detected, like a property status change that contradicts a closing record
When any of those happen, send the event to the DLQ and store the original timestamp, the last-attempt timestamp, the source service name, the current retry count, and structured error details. That metadata gives your team a much better shot at finding the root cause fast, especially if multiple events from the same source start failing at once.
Every DLQ needs an owner. Recovery should be assigned by event type so that when owner sync events or property updates land in dead-letter, someone is on the hook to inspect them, fix the issue, and replay them. Otherwise, you’re not managing failures – you’re just watching the count climb on a dashboard.
Putting It Into Practice: A Real Estate Retry Blueprint
Here’s how to turn the retry rules above into one clear blueprint that engineers and ops teams can use day to day.
A Retry Policy Map for Common Real Estate Events
Use the map below as the starting policy for property updates, owner syncs, and phone verification events.
| Event Type | Max Attempts | Backoff Pattern | Idempotency Key Design | DLQ Trigger |
|---|---|---|---|---|
| Property Update (price, square footage, bed/bath changes) | 7 | Exponential: 1s → 2s → 4s → 8s → 16s → 32s → 64s, +0–500 ms jitter | property_id + event_type + payload_version | Permanent validation error or property not found |
| Owner Record Sync (CRM, contact enrichment) | 10 | Exponential starting at 5s, capped at 10 minutes, with jitter | owner_id + source_system + payload_hash | Permanent auth failure or exhausted attempts |
| Phone Verification (SMS, voice) | 3 | Fixed steps: 2s → 10s → 30s | phone_number + channel + verification_session_id | Exhausted attempts or permanent telephony failure (blocked number, unreachable country code) |
This same setup works for enrichment, skip tracing, and verification workflows too. For BatchData workflows, use job_id + owner_id for bulk skip tracing and session-scoped keys for phone verification.
What Teams Gain From Well-Designed Retries
Well-designed retries cut silent drops, duplicate writes, and wasted outreach. And when DLQ ownership is tied to each event type, teams spend less time fixing broken pipelines and more time working with property, owner, and contact data they can trust.
FAQs
How do I tell transient errors from permanent ones?
Transient errors are short-lived, so a retry often does the job. Common examples include HTTP 429 rate limits and 5xx server errors. These issues often clear after a brief pause, which makes exponential backoff a good fit.
Permanent errors are different. Cases like 400 Bad Request or 403 Forbidden usually mean there’s a problem with the request itself or with authorization, and retrying won’t solve that. If a payload still fails after every retry attempt, move it to a dead-letter queue for manual review.
What should an idempotency key include for real estate events?
Use stable, specific resource identifiers. For property events, that usually means the APN or a specific MLS ID.
If one identifier isn’t enough, use a composite key instead, like an APN plus an address hash or a system-generated record ID. That helps stop duplicate writes, because repeat calls can update the same record instead of creating a new one.
When should an event go to a dead-letter queue?
Move an event to a dead-letter queue when it keeps failing after all retry attempts or when the payload is malformed.
In real estate workflows, send the event to the DLQ for manual review after the exponential backoff sequence runs out – for example, 1s, 4s, 16s, 64s, and 256s. This keeps stubborn failures isolated, so they don’t hold up the rest of the pipeline, and it cuts the risk of missing property updates or owner syncs.
