Mysterious Double Instance Hampering Performance - I study the existing code base. Confer with a colleague. Then I determine the optimal plan to change the functionality to load only a slice of all the dat...
I decided to do a little digging. The way I knew there was no database access was that I tried a "tnsping". That command checks to see if the database can be contacted. I got back an error that the target host or object does not exist. Now that might mean that the actual database was down. However I tried accessing the database from another machine and found that it was up and running.
So I dug a little deeper. We configure our workstations to use a tnsnames file to resolve database aliases. I ensured that we were using the configuration file that that DBAs set up. Then I made sure that file existed. So far so good. Finally I checked that our database alias was in that configuration file. It was.
Next I looked at the configuration data for our alias. The config file just translates the alias to a domain name and a port on that machine where the database listens. I tried a regular ping of the domain name of the server. Bamm. My machine could not resolve the domain name. Well I found out the IP address of our server. When I plugged that into a local copy of the config file, the door was opened. Full access to the database.
The problem must be some sort of Domain Name Server issue. Ping cannot resolve my well known domain name. Time to get the network guys involed. I don't even have to bother my DBA team.
They interviewed one of his opponents. The opponent was some sort of kid grand master at chess. He said that when he played this high school kid, he was hanging on for dear life due to the aggressive onslaught of attacks on the chess board. How could a kid rise up in chess skill so fast. Well it turns out the boy was cheating.
The chess rules allow you to bring in a PDA to record your chess rules. You are only allowed to run a certain chess move recording program. That program has been certified as being able to take over the PDA and ensure no other programs (that might be used to cheat) are running. Turns out this kid was somehow able to get around it.
What should have been suspicious is a kid with just a good rating all of a sudden starts playing like he is Bobby Fischer. That just does not happen. They also interviewed some of this kid's previous coaches. They knew he was okay, but nothing like a Bobby Fischer. In fact, a past opponent had complained that it felt as if the dude was cheating. They just could not find any evidence of it before.
In the end, a judge was called into the state competition match, took possession of the PDA, and found a rogue program being run. Here is the funny part of the story. The kid said this was the first time he cheated. Yeah right. The moral of the story is to trust but verify.