Case of an FQDN Issue

The phrase, "I can't access my shared drive" was intermittent, but becoming common for a remote location connected via an MPLS circuit. Without hesitation the finger was pointed at the network and my phone rang. People connect to shared drives everyday, but it is one of those things they take for granted. Behind the scenes there are many layers of technology, protocols, and devices working together to make those connections happen. I can’t count the number of ways to hinder performance of a network share or prevent it from working altogether. *SPOILER* If you’re like me and you like to know the big picture first, the problem in this case was DNS. Read on for the details, or just know a great test is to place a ‘.’ at the end of your DNS path. To kick things off I asked for a screen shot of the error. This is what I received: Obviously, this was not very helpful (are error messages ever?)....
Read More

Problems After Updates?

One of the most common problems any IT admin faces is a software update. While software updates are generally considered a good thing, because they patch security flaws, fix bugs, try to improve performance, and more, they are also a common source of problems. Every admin knows to be ready for calls after a scheduled maintenance window. This issue was no different.   A ticket came in stating users could not access a web app through the backend system after an upgrade to Java 1.7. The server, java, and app logs all looked ok and appeared to be running properly. Also, interestingly, the web app worked when accessed directly from a web browser. This sounded like a perfect opportunity for a quick packet capture and analysis. Here is what was produced:     *Note: In order to maintain the SSL session info I could not anonymize this, so I'm just using a screen shot instead of sharing the capture on CloudShark.   I've done quite a few...
Read More

Case of the Large Header

Just because you can do something doesn't always mean you should. One such example of this is using large HTTP headers. While the HTTP specification itself doesn't set boundaries, most web servers have default limits around 8 KB. Other devices in the path such as firewalls/WAFs, proxies, and load balancers also have similar limits.   Problem The application testers were receiving a reset error. Their application and web server logs did not show any problems.   Analysis The first question asked was, "If the web server isn't sending the reset error, what is?" In this case we found there were several devices in the path including a domain firewall and a load balancer. The firewall admin saw two-way traffic hitting an accept rule and passing through. That left the load balancer. The load balancer admin confirmed via a packet capture that it was, in fact, sending a reset near the end of the TCP stream. Why would the load balancer send a reset? A load balancer does exactly that....balances...
Read More
Case of the Named Pipes

Case of the Named Pipes

Problem I have come to expect vague error messages that seemingly blame the network. This one is no different. Server Error in '/' Application. The network path was not found Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code. Exception Details: System.ComponentModel.Win32Exception: The network path was not found Source Error: An unhandled exception was generated during the execution of the current web request. Information regarding the origin and location of the exception can be identified using the exception stack trace below. (The stack trace is long and doesn't actually provide root cause either, so I omitted it to keep this post brief.)   In this case, a web server was providing the error above after receiving a user request and attempting to connect to a backend database. Unless there are other dependencies, the problem is between the web server and database. Both ping tests and telnet tests to...
Read More