Nortel VoIP's single voice issue

A very good story for learning how firewall manage UDP rules and SIP hand-shaking!

An Avaya voice gateway with a traditional firewall, if we only open UDP/TCP 5060 for SIP and some RTP ports range, does it work? See the below story :-)

Last year, I encountered a problem that we always encountered single voice problem with our checkpoint firewall, and the most interesting part is, if we picked up the phone call within around 60 secs, the communication is OK, but if we hear waiting music more than 60 sec, our counterpart will not hear us! 

Here is the scenario:

client B --- client B's GW --- client A's GW --- client A

We mirror Avaya Voice GW traffic into our sniffer:

Firewall rules:

B --> A UDP port 5060

B <--> A UDP port [RTP port range] 

So normally it should work, because for a normal SIP connection, B will initial a 3-way hands shake with A, then the connection setup, and both A and B will use the established channel to communicate with each other. 

But in our case, after channel established, A send an INVITE request to B as well, which like an answer to B's request! This is how Avaya server work!? Avaya server tried to setup another tunnel to back B to transfer voice traffic, if B not answer this request the single voice happened! This is very abnormal for a SIP voice call, from B's Voice GW, we could see RTP ports changed, which is very weird!

In the beginning

Sent:

609087 2012-04-20 11:15:41.587380 114.66.80.253 114.66.80.181 UDP Source port: 11790 Destination port: 5442

Received:

609104 2012-04-20 11:15:41.662456 114.66.80.181 114.66.80.253 UDP Source port: 5442 Destination port: 11790

Then

Sent:

609525 2012-04-20 11:15:45.107355 114.66.80.253 114.66.80.181 UDP Source port: 11790 Destination port: 5478

Received:

609537 2012-04-20 11:15:45.224009 114.66.80.181 114.66.80.253 UDP Source port: 5478 Destination port: 11790

After weeks of checking with Avaya, I found that Avaya implement a very complicated algorithm in its Voice GW. In Avaya GW, IVR and normal voice traffic are transferred in different channels and more weird part is, if IVR plays, IVR will take the existed channel to transfer its recorded music and it will setup another voice channel by "INVITE" back to the caller!!! If B does not process another "INVITE" the RTP session won't start! If you heard IVR, it means that Avaya GW already established another channel with the counterpart. That’s why in B's RFC compliant Voice GW, it saw some ports change, that’s why we lost connection after several secs, because our server would like to setup a new tunnel to transfer voice traffic!!!

But why we it works fine within 60 secs!?

Because firewall treat UDP rules by a very obsolete manner. 

We all know that, for a TCP we could keep a session, because it will have a lot of para to monitor, (syn, ack number, port number, even the flags); but for UDP, we only have "SIP, SPORT; DIP, DPORT" so how can we keep those rules? In checkpoint, it just maintain a timer, if no matches for a while, it will time-out the rule. So hacker could exploit this vulnerability to compromise the network. But in our case, it helped Avaya within 60 sec. Within 60 sec after B called A, Avaya could "exploit" this vulnerability to "INVITE" back to B's GW to accomplish the "INVITE" back, but after 60 sec, the timer timeout, it won't work any more. So we have to explicitly open a "back-door" for Avaya server to help him to "INVITE" back to the caller. 

Solution: add another rule for B <-- A udp 5060, that's all! 

Appendix:

SIP: http://www.ietf.org/rfc/rfc3261.txt

Stateful firewall: http://en.wikipedia.org/wiki/Stateful_firewall