Are you sure it's N9 problem and not your firewall problem?
I had a similar issue at home with my VoIP phone, but it was due to the fact that my stateful firewall was leaving open the STP/RTP ports only if I was calling.

Solution was to add port forwarding for STP/RTP service to the IP address of the VoIP phone device.