Service Broker Demystified - Fire and Forget Anti-Pattern

After you work with Service Broker for some time you'll learn about the dreaded "fire-and-forget" (anti-)pattern.  In this post I'll show you exactly why fire-and-forget is so bad.  I'll also show you the one simple rule to remember to avoid accidentally modeling fire-and-forget. Finally, we'll learn how to fix it production systems where fire-and-forget has bitten you.  In the next post we'll cover how to model a real fire-and-forget design safely...using a monolog.  

People claim they avoid Service Broker because it is too confusing. I started a blog series called Service Broker Demystified because SB really isn't that difficult. Today we are going to look at the fire-and-forget anti-pattern.  A lot of noobs will accidentally model their first SSB designs as fire-and-forget. Let's take a step back and look at how SSB message communications work.

Service Broker conversations are always dialogs (that's why the syntax is BEGIN DIALOG CONVERSATION).  In SQL Server a dialog involves EXACTLY two parties...one called the Initiator and the other called the Target.  At a bare minimum the conversation goes like this:

1. Initiator begins dialog and sends a message
2. Target receives the message, processes it, and acknowledges it.  The target always ends the conversation first.  
3. Initiator ends the dialog on the initiator side.  

In my post on CLOSED conversations I discussed why it is so important that the target always ends the conversation first.  If you don't remember this then you probably have modeled the fire-and-forget pattern which is dangerous.  Fire-and-forget (f&f) is where the initiator sends a message and immediately ends the conversation because it doesn't need or care for the target acknowledging its receipt of the message.  

But my experience is that many (most) SSB implementations really should be simple f&f patterns.  The most common use case I've seen for SSB is where you need to send a message to another SSB service to initiate some asynchronous processing after an event occurs.  For instance, a trigger sees a data change which pushes a message onto a queue to launch an ETL process.  The initiator (the trigger) couldn't care less when the ETL process completes, or even if completes without error...that is some other process' job to monitor that.  

What I just described is a monolog.  A one-way conversation.  A one-sided dialog.  That's f&f.  

What's wrong with fire-and-forget?  You risk lots of side-effects like 

• Conversation population explosion (I've written about that here and [[Monitoring Service Broker|here]]).  This can be fixed after-the-fact with some clever coding (END CONVERSATION WITH CLEANUP;)
• CLOSED conversations never being purged.  Again, can be fixed with END CONVERSATION WITH CLEANUP;.  
• Errors being lost forever.  This one is particularly nasty because you'll only ever find out there's a problem when a user starts complaining.  

Let's demo that.  

Fire-and-forget at it's worst

If you want to follow along you can download the demo script here.  

We'll start by creating the most basic send and receive queue.  

Next we'll send a message and immediately END CONVERSATION (Line 33).  This is the f&f anti-pattern.  

Note that we have no errors and we have two messages in the ReceiverQ.  They both have the same conversation_handle.  The first message is the actual message and the second message is the EndDialog, corresponding ot the END CONVERSATION.  

Next we want the receiver to process the message(s)...

...and note the messages are processed without any error...  

So far so good.  There's no problem with f&f that I can see. 

But now let's assume that something "changed" on the receiver service.  Perhaps it begins throwing errors due to a logic bug in your activator code...perhaps there is a disk space issue...or perhaps someone decided to attach a contract to the receiver.  Let's look at what happens.  Here we create a simple constraint that messages much be well-formed XML.  Of course or original message was NOT XML, so it should now fail.  

Let's start over and see what happens.  We'll build another f&f message that is NOT XML and send it.  

No errors so far and we again see the same two messages in the ReceiverQ.  So far nothing has changed.  

Ah, now let's see what happens when the receiver tries to process the message.  Remember, it should throw some kind of error about the message not being valid XML.  

Nope, all Queues are clean!  

Your initial reaction may be that our messagetype/contract is not working.  They are.  

Let's think about what happened.  When we sent the message we immediately did END CONVERSATION meaning that the sending service does not care to see ANY messages, even failures.  The error is lost forever.  You may be thinking that the error should maybe be retained SOMEWHERE...like sys.transmission_queue maybe.  You'd be wrong.  That queue is solely to retain messages that failed due to communication problems.  Our failure was due to a contract problem .  

This is why fire-and-forget is so dangerous.  You'll likely never see the side effects of f&f until you accidentally change something in your prod system.  And then determining that you've just been bitten by f&f is not easy because no error logging is maintained.  

Summary

Lots of people try to model a fire-and-forget pattern because the need for this use case is so prevalent.  Don't do that.  Simply remember that the target always ends the conversation first.  If you remember that then you won't end up with lost errors like the above demo.  In the next post I'll cover how to safely monitor a monolog, which is the use case that most people need and is often confused with fire-and-forget.  

You have just read "Service Broker Demystified - Fire and Forget Anti-Pattern" on davewentzel.com. If you found this useful please feel free to subscribe to the RSS feed.