Service Broker Demystified - CLOSED conversations

Those new to Service Broker are confused by CLOSED conversations.  They hang around even when they are CLOSED.  A quick google search will tell you this is for security purposes.  In this post I'll show you exactly why that is.  Also, invariably, one day a DBA will see a huge number of CLOSED conversations that never get cleaned out.  I'll also cover why that happens and how to avoid it.  

I've heard that people don't want to use Service Broker because it is too confusing. I started a blog series called Service Broker Demystified because SSB really isn't that difficult to understand. But there are some confusing aspects to SSB that I'm covering in this blog series.  Today we are going to cover why CLOSED conversations seem to stick around in your queues.  

Every CLOSED conversation will stay in your queue for at least 30 minutes.  Every one.  

This is easy to demo.  You can download the repro script to try on your own.  

Let's build some basic scaffolding:

(Figure 1)

That gives us a basic send and receive setup.  Now we'll send the most basic message to the receiver:  

(Figure 2)

We get two conversation_endpoint entries.  One for the initiator and one for the target.  Both are conversing.  Both have the same conversation_handle. 

(Figure 3)

The next step is for the receiver to ack the message.  We want to keep this simple so we merely perform an END CONVERSATION.  Again, we want to keep it simple:  

(Figure 4)

Now we see that our conversation state_desc has changed.  We see the receiver has been set to closed and the initiator is set to DISCONNECTED_INBOUND, meaning the far_service has disconnected.  

(Figure 5)

Finally, the sender should always close its conversation on its side.  This is usually done as an activated proc on the sending queue.  

(Figure 6)

Here I noted the time I executed the command.  We see the initiator conversation is GONE yet the receiver's ack message is listed as CLOSED.  Note also that the security_timestamp is set to about 30 minutes later, which is when this message will be purged.  

(Figure 7)

Using that basic dialog pattern we can learn a lot about how Service Broker messages and dialogs work.  For instance:

...So why do CLOSED conversations live for 30 minutes in your queues?  

To prevent replay attacks.  In a previous post I mentioned that whenever SSB communicates to another service it will, by default, encrypt the message unless you specifiy ENCRYPTION=OFF when you start the dialog.  This provides us with some interprocess security but it is still possible that the original dialog can be replayed (for instance, by a "man in the middle") and the valid dialog can be fraudulently repeated.  By maintaining the CLOSED conversation handle for 30 minutes SSB assures that if it sees that conversation handle again from (what appears to be) the same network host with the same encryption key then it knows that this might be replay attack (or just programmer error) and the conversation will not be reopened and processed.  

So how exactly would a man-in-the-middle attack work?  

After the target closes his conversation it is marked as CLOSED (see Figure 5).  That is the entry that is maintained for 30 minutes.  In this case we can rest assured that both the conversation_handle and the conversation will not be altered by a third party.  SSB doesn't need to maintain or track this for the sending side of the conversation.  There's nothing really that we can do if a third party spoofs that side of the conversation.  But even that can't be spoofed because the entire conversation_handle is marked as CLOSED anyway (see Figure 7 above).  We don't need to maintain both sides of the conversation.  Just one.  

In security engineering a nonce is an arbitrary number used only once in a cryptographic communication.  The conversation_handle is similar in spirit to a nonce word.  Both are random or pseudo-random "things" issued during an authentication protocol.  Once the nonce is seen by either side (the same conversation_handle in a CLOSED state), then the communication channel cannot be reused or replayed.  HTTP digest authentication works this way and this is how SSB was modeled.  The timestamp is included, just like with digest authentication to ensure timeliness of the messages.  This is also why you can declare a LIFETIME on a dialog in SSB.  

...so why does sys.conversation_endpoints CLOSED entries sometimes NOT disappear after 30 mins?  

When this happens you'll usually see thousands of entries just like Figure 5.  Lots of CLOSED messages.  This means you didn't model your dialogs correctly or you have a bug.  Here is the simple rule:  the target always does END CONVERSATION first.  No Exceptions!!!

This is the pattern that you SHOULD be using...if you aren't then you are susceptible to "conversation population explosion".  

1. Initiator starts dialog and sends message
2. Target RECEIVEs the request
   1. Target may do some processing and send another message back to the sender (optional)
   2. Initiator receives and processes that response and (optionally) sends another response or acknowledgement
3. Target does END CONVERSATION
4. Initiator processes END CONVERSATION and issues its own END CONVERSATION (usually via an activator on the initiator queue)

...so why does sys.conversation_endpoints sometimes get tons of DISCONNECTED_INBOUND entries?  

Look at Figure 5 again.  This happens when the Initiator issues the END CONVERSATION before the Target.  This is a bug or design flaw.  The cause is the initiator is doing END CONVERSATION before the sender.  

In a future post I'll show you how to properly model a monolog which is sometimes called a "fire and forget" message.  Let's say you have some process that you want to send a message to.  You don't care if it completes or even if it throws an error.  You model it this way:  

1. Initiator starts dialog and sends message and immediately does END CONVERSATION
2. Target RECEIVEs the request and does the work

What happens?  You leak conversation handles and sys.conversation_endpoints begins to fill up.  Don't model your conversations that way.  The initiator END'd first and the target never did any END CONVERSATION.

...so I found a bug in my design and I leaked a bunch of conversation_handles.  How do I fix it?  

END CONVERSATION conversation_handle WITH CLEANUP

How can you remember to model your dialogs properly?  

Just use CB protocol.  If you are too young to remember that, just go watch Smoky and the Bandit.  This is known as voice procedure and is used in other industries as well.  Here's an example:  

You have just read "Service Broker Demystified - CLOSED conversations" on davewentzel.com. If you found this useful please feel free to subscribe to the RSS feed.