Tuesday, May 12, 2009

XP Firewall option "My network (subnet) only" blocks traffic from local subnet

Problem: The weirdest problem occurred on a Windows XP Service Pack 3 computer: I changed a firewall rule scope from "Any computer" to "Local subnet only," only to find that the service did not accept traffic from my local subnet anymore. I started investigating, and ended up testing several different services and ports. The same thing happened: Once the port or service had been restrained to the scope "My network only", no traffic from comptuers on the local subnet was allowed through.

Symptoms: I noticed first because I tried to ping the computer. The name was not resolved, because the UDP 137 port (part of the File and Printer sharing entity) for NetBios name resolving blocked when set to "my network only" scope. Same thing happened to the VNC server service - once the 5900 port or the VNC server service was set to "my network only", it was no longer possible to connect to the comptuer from another local host.

Resolution: Sifting through probably a few dozens of webpages left me empty handed. At the end, I decided to rebuild the firewall settings from scratch by clicking the "Restore default settings" button of the advanced tab in Windows Firewall. When I now selected a "Local subnet only" scope, it worked like a charm. My firewall configuration was obviously messed up and needed a reset.

Reason:
Who knows?

Apart from understanding what went wrong, the hardest thing in such a situaiton is to know when you should stop wasting time searching for the reason and resort to a tedious rebuild of firewall rules. Most boring: I still do not know what had went wrong, only what solved it. :(