DaveWentzel.com            All Things Data

Service Broker Demystified - Fire and Forget Anti-Pattern

After you work with Service Broker for some time you'll learn about the dreaded "fire-and-forget" (anti-)pattern.  In this post I'll show you exactly why fire-and-forget is so bad.  I'll also show you the one simple rule to remember to avoid accidentally modeling fire-and-forget. Finally, we'll learn how to fix it production systems where fire-and-forget has bitten you.  In the next post we'll cover how to model a real fire-and-forget design safely...using a monolog.  

People claim they avoid Service Broker because it is too confusing. I started a blog series called Service Broker Demystified because SB really isn't that difficult. Today we are going to look at the fire-and-forget anti-pattern.  A lot of noobs will accidentally model their first SSB designs as fire-and-forget. Let's take a step back and look at how SSB message communications work.

Service Broker conversations are always dialogs (that's why the syntax is BEGIN DIALOG CONVERSATION).  In SQL Server a dialog involves EXACTLY two parties...one called the Initiator and the other called the Target.  At a bare minimum the conversation goes like this:

  1. Initiator begins dialog and sends a message
  2. Target receives the message, processes it, and acknowledges it.  The target always ends the conversation first.  
  3. Initiator ends the dialog on the initiator side.  

In my post on CLOSED conversations I discussed why it is so important that the target always ends the conversation first.  If you don't remember this then you probably have modeled the fire-and-forget pattern which is dangerous.  Fire-and-forget (f&f) is where the initiator sends a message and immediately ends the conversation because it doesn't need or care for the target acknowledging its receipt of the message.  

But my experience is that many (most) SSB implementations really should be simple f&f patterns.  The most common use case I've seen for SSB is where you need to send a message to another SSB service to initiate some asynchronous processing after an event occurs.  For instance, a trigger sees a data change which pushes a message onto a queue to launch an ETL process.  The initiator (the trigger) couldn't care less when the ETL process completes, or even if completes without error...that is some other process' job to monitor that.  

What I just described is a monolog.  A one-way conversation.  A one-sided dialog.  That's f&f.  

What's wrong with fire-and-forget?  You risk lots of side-effects like 

  • Conversation population explosion (I've written about that here and here).  This can be fixed after-the-fact with some clever coding (END CONVERSATION WITH CLEANUP;)
  • CLOSED conversations never being purged.  Again, can be fixed with END CONVERSATION WITH CLEANUP;.  
  • Errors being lost forever.  This one is particularly nasty because you'll only ever find out there's a problem when a user starts complaining.  

Let's demo that.  

Fire-and-forget at it's worst

If you want to follow along you can download the demo script here.  

We'll start by creating the most basic send and receive queue.  

Next we'll send a message and immediately END CONVERSATION (Line 33).  This is the f&f anti-pattern.  

Note that we have no errors and we have two messages in the ReceiverQ.  They both have the same conversation_handle.  The first message is the actual message and the second message is the EndDialog, corresponding ot the END CONVERSATION.  

Next we want the receiver to process the message(s)...

...and note the messages are processed without any error...  

So far so good.  There's no problem with f&f that I can see. 

But now let's assume that something "changed" on the receiver service.  Perhaps it begins throwing errors due to a logic bug in your activator code...perhaps there is a disk space issue...or perhaps someone decided to attach a contract to the receiver.  Let's look at what happens.  Here we create a simple constraint that messages much be well-formed XML.  Of course or original message was NOT XML, so it should now fail.  

Let's start over and see what happens.  We'll build another f&f message that is NOT XML and send it.  

No errors so far and we again see the same two messages in the ReceiverQ.  So far nothing has changed.  

Ah, now let's see what happens when the receiver tries to process the message.  Remember, it should throw some kind of error about the message not being valid XML.  

Nope, all Queues are clean!  

Your initial reaction may be that our messagetype/contract is not working.  They are.  

Let's think about what happened.  When we sent the message we immediately did END CONVERSATION meaning that the sending service does not care to see ANY messages, even failures.  The error is lost forever.  You may be thinking that the error should maybe be retained SOMEWHERE...like sys.transmission_queue maybe.  You'd be wrong.  That queue is solely to retain messages that failed due to communication problems.  Our failure was due to a contract problem .  

This is why fire-and-forget is so dangerous.  You'll likely never see the side effects of f&f until you accidentally change something in your prod system.  And then determining that you've just been bitten by f&f is not easy because no error logging is maintained.  

Summary

Lots of people try to model a fire-and-forget pattern because the need for this use case is so prevalent.  Don't do that.  Simply remember that the target always ends the conversation first.  If you remember that then you won't end up with lost errors like the above demo.  In the next post I'll cover how to safely monitor a monolog, which is the use case that most people need and is often confused with fire-and-forget.  

 


You have just read "Service Broker Demystified - Fire and Forget Anti-Pattern" on davewentzel.com. If you found this useful please feel free to subscribe to the RSS feed.  

5 comments

Comment: 
Hi, thank you for you articles, very useful. You are using in that example 2 queues. It seems the 2nd one used only for receiving reply about closed dialog. Is it possible to use only one queue in situations like this?

Comment: 
Again, SSB is more confusing than JMS or an AMQP-type system like RabbitMQ, or even Apache Kafka.  You can thank MS for this.  You have to remember the rules:TARGET q/service must END CONVERSATION firstINITIATOR q/server must END CONVERSATION second You certainly could use one queue/service but then you have to be really careful to ensure you are getting the ACKs (END CONV) in the right order and in the right place in the code.  It works if you know what you are doing with SSB but think about the next developer who may not understand SSB.  In my mind it's better to think of the TARGET as the actual worker queue (topic in Kafka) and the INITIATOR as merely some goofy mechanism that you use to publish the message.  Or think if it as a coordinator.  In *most* designs I've seen that aren't complex "workflows" that's all the INITIATOR does.  The next thing to remember is that the INITIATOR always needs an activator that pulls the EndDialog messages and ACKs them.  Once you understand that then you quickly realize that the INITIATOR activator is also a great spot to put your generic errorhandler that re-queues failed messages.  HTH

Comment: 
Hi, thank you for you articles, very useful. You are using in that example 2 queues. It seems the 2nd one used only for receiving reply about closed dialog. Is it possible to use only one queue in situations like this?

Comment: 
I may actually want to fire and forget. If a developer takes the time to learn SQL Server Service Broker only to find that they have to write extra code to handle the response - despite the many obvious benefits, then you shouldn't preach the merits on why they should always wait. If we think about the command response, it means that the initiator has to wait for a response and the implication is that something will be done with the response, perhaps logging it to a file, so then we check that the file was written, now the file is written do we then write that the file was written to a database and then check that the database record has written. Clearly Command/Response is how we want to do it the vast majority of times for best practices but please qualify both the merits/demerits of Fire and Forget rather than covering the gaping hole that this technology doesn't seem to allow fire and forget. Many would consider command response an anti pattern because it makes the developer have to write additional code that they may not want to.

Comment: 
I couldn't agree more with you.  For the vast majority of cases fire and forget is probably what you do want and is closer to what JMS and AMPQ systems offer you.  Unfortunately there's no way to safely do that with SSB.  

Add new comment