PDA

View Full Version : Forum Software - Auto repair fix



AthenaOfDelphi
05-05-2009, 03:34 PM
Hi all,

Apologies to everyone... I've just lost you your 'Unread posts since last visit' status.

As some of you may be aware, this morning there was a period when the site was offline due to a database problem. We basically ran out of user connections due to hanging queries that appear to have been caused by corruption of one (or more) of the database tables.

To try and stop this (and the other database corruption problems we have experienced in the past) from happening again, I am in the process of implementing a more advanced automatic repairer within the main body of the forums codebase. Unfortunately I'm not as familiar with SMF as I would potentially like to be, so some of this is a little trial and error (mainly getting the optimum location for the auto repair code), so if the board has problems over the next couple of hours, please bear with me.

Athena

Wizard
05-05-2009, 03:58 PM
Keep up the good work you busy busy little worker bee ;)

AthenaOfDelphi
05-05-2009, 05:02 PM
Quick update...

Well... I've implemented a new auto repair mechanism for the forums. This is somewhat more advanced than the quick fix one I implemented weeks ago.

It doesn't repair on every request as the original one did, but it does repair more tables... mainly the ones that have so far caused problems.

So, if you get a database problem, refresh your browser a few times and keep an eye out for 'AEF' in the top left corner. If that appears it means the auto repair code has reported some messages which are embedded in the document as HTML comments. For this to work, the site now adds a new session cookie which counts requests (it's the one with _rpc on the end), so please don't block this.

I have already this afternoon experienced a database error, and I kept refreshing the page and it sorted itself out.

However, there is one more tweak I would like to make to it, to make it more robust (and hopefully able to automagically resolve the situation we had this morning). I'm working with Traveler and Savage to try and fix the problem thats blocking this. Once thats done, we should hopefully be able to automatically recover from the DB errors that have caused us problems since the site went live with SMF.

So, to re-iterate... if you get a database problem (not 'Unable to connect' or 'Exceeded maximum connections), then try refreshing the page upto 10 times... you should hopefully get an AEF in the top left and the board should spring back into life. If it doesn't and you have AEF, look at the page source, copy the comment at the top that details the auto repair results and mail it to me. Do NOT sit there constantly refreshing the page if it doesn't fix the problem in 10 refreshes.

Thats it for now.

Athena

paul_nicholls
06-05-2009, 10:53 AM
Great work Athena :)
cheers,
Paul

Ñuño Martínez
06-05-2009, 01:41 PM
I don't understand why are you having so much problems with SMF. I mean, I had a phpBB2 forum in my web for some time and I hadn't that problems. Of course it hadn't so much users as PGD ::) I'm just curious. Are the mods you're using? Or may be SMF isn't finished? ???

AthenaOfDelphi
06-05-2009, 02:48 PM
Off the record ;) I'm not entirely sure that it's not the server itself. I have talked with Savage and Traveler about it and none of us can understand the problems. I've certainly never experienced anything like it and I run a MySQL server which gets absolutely thrashed for our browser game... touchwood, it's never experienced anything like this.

Is it SMF thats causing it... somehow I don't think so because it's not the data within the tables thats getting corrupted... its things like the index files which we have no direct control over.

So, to try and keep the site running with minimal intervention, I've resorted to these automatic repairs... it's not ideal and Traveler and I have discussed a couple of tweaks I'm going to make to them, but if it keeps the site running, it will have to do.

As for why we're feeling the effects more with SMF than we did with phpBB... I'm not sure, but I do think that SMF makes a lot more database updates in the course of normal operations (the majority of the time people are reading the site) than phpBB did.

Rest assured that we are trying our best to resolve the problems... but it's just not that easy, especially when you don't have full shell access to the server.

arthurprs
06-05-2009, 04:42 PM
i don't think the problem is caused by SMF, i have 2 forums running and i never had to repair de DB