Why do these firewall tests say I've failed? Should I be worried?

I don’t know the specific tests you may be using, but in all honesty it
doesn’t surprise me.

It really depends on what they mean by “failed”, and what you mean by
“legit”. The real question is do you have anything to worry about. Quite
often those tests don’t give you a straight answer. Even my favorite test
tends to overstate the risk.

So, what to do?

One thing to remember is that if your firewall test is provided by a
firewall provider, they of course have an incentive to make your current
situation appear dire, and in need of their product.

My favorite test site is GRC’s Shields
Up. The bad news: it’s difficult to navigate, slightly difficult to
understand, and definitely what I would consider to be somewhat alarmist.
However, it’s authoritative and unbiased.

But here’s the problem with this, and I would assume pretty much any of
the firewall testers…

I ran it, and got the message (in big red letters, no less) that I had
“FAILED”.

Now as you can imagine, I consider my setup pretty darned secure. And
yet it’s somewhat disconcerting to see that big red FAILED.

So how had my setup “failed”? To try and keep it from getting too
technical, a single port tested on my router responded to an external
probe, saying that it was “closed” and not accepting connections. All
other ports didn’t respond at all. Not responding at all is considered
preferable since any response, even a response that says “go away”
confirms the existence of something at that address.

Now, is that a failure? Depends on what you’re looking for. It’s
certainly not a failure or a risk to me and as a result I honestly
don’t care. But it means actually examining the results and understanding
the so-called failure.

And therein lies the problem. I’m behind a NAT router, which when running
properly provides all the firewall I feel I need. Testing with a
vulnerability test like Shields UP is nice, but in reality I’d feel just
as secure had I not tested just knowing how NAT routers work.

With a software firewall, like the Windows firewall, things are
different. Without knowing what kind of failures the tests are reporting
it’s difficult to know if you have a problem or not. The solutions more
often than not are not a different software firewall, but
rather understanding the vulnerabilities exposed and changing system
configuration to resolve them. A good example might be Windows File
Sharing. You could have that enabled through your firewall – but if
you’re connected to the internet, that’s a vulnerability you’ll want to
close by either disabling that functionality, or closing the hole in the
firewall. Other failures may not be as concerning, and may require no
real action in practice.

If you’re at all concerned, I really recommend using a NAT router.
Normally a router would be used to share a single internet connection
among more than one computer, but even when used with a single computer
it provides a very effective firewall without installing any additional
software on your machine.