The Observer Effect

I have run into a performance problem with one of my databases at work. We are constantly adding new records to the database and the system runs fine. When we try to augment the normal load with some extra data, the system slows to a crawl and I have been trying to figure out why.

I have a tool that will help me monitor performance. Every 2 seconds, it queries the program sending data into the database and asks how many packets of data are waiting to be saved. When the system is keeping up, this number is 0. When it is running slow, the number grows. Yesterday I got tied up with other things and turned my monitoring program off. I wasn't really paying close attention, but the performance seemed to increase because I wasn't watching.

This morning I wanted to see if there was a true performance increase or if it just seemed that way. I started loading data and made a note of the time. Yesterday it took almost exactly 2 hours to complete. This morning it took 1 hour and 40 minutes. I thought that was interesting and so I ran the test again. Sure enough, the second test completed in 1 hour and 50 minutes. While my monitoring of the system didn't slow things down too much, it did have an effect.

I guess you could say that the old cliche about a watched pot never boiling can be true. In Physics, this is called the Observer Effect, where merely observing something changes what is being observed.