I took a very quick look at your server code, and i saw you use a multithreaded architecture.
So the first thing that comes to my mind is that you could have syncronization problems. I don't digged any far in the code, but i couldn't see any use of critical sections or other syncronization mechanisms.

So it could be from deadlock or race conditions and the like.