Recent performance issues ...

R. · 05 May 2005, 01:58

were due to a DOS (= Denial Of Service) attack. A self-made DOS, that is.

Short explanation:

There is a configuration option for the new homepage which allows you to display new posts since your last login.
This is the description:

Quote:

Set this option to 'Yes' if you would like to display the number of new posts since a user's last visit. Note: On larger forums, this option may be somewhat server intensive. If you experience problems with the page loading slowly, try turning this option off.

Well, somewhat server intensive is more than correct. Depending on the number of concurrent online users, this option simply kills your database server.

However, I have changed a few things now, hopefully resolving this issue. If not, I'll have to turn that option off.

Please excuse the inconveniencies.

SuperLoafMan · 05 May 2005, 02:02

wondered what was hapening....I have been experiencing the slow loading pages but since the forum has come back up tonight the pages are loading fine...and fast...what ever you did R it seems to have worked.....so far...

Tim · 05 May 2005, 10:03

Last night i couldn't not come on the forums, and when i did, it was so slow, the internet explorer told me he couldn't load the page... But now it is fixed.

Bren · 05 May 2005, 11:24

likewise, was going so icredibly slow...but works just fine at moment

Chris · 05 May 2005, 14:43

Ditto!!!

Although i got a very worried text from someone at almost midnight worrying that she ahd pressed something to make it all break down!

Cathie · 05 May 2005, 14:46

Everything's working fine now here

Tim · 05 May 2005, 15:37

indeed, at the moment it is, i think, a bit faster then it was before. Or maybe itis my connection

Chris · 06 May 2005, 23:04

Seems to be chugging a bit tonight!!!

R. · 06 May 2005, 23:15

Honestly, I don't know why.
As far as I know, I have eliminated all "bad" queries. Sometimes it looks like that this server simply cannot execute 10 simple(!) database queries simultaneously. Makes me wonder how this iron survived the migration.

Tim · 07 May 2005, 11:48

It is strange. The last couple of evenings the forum is so slow. Maybe it has to do with the latest adjustments.
But it is getting slower, then you post something, it will take ages to load, if it will load. And eventually the forum crashes over here. Totally dead. After a while i'm able to go on the forums again. Very strange.

R. · 07 May 2005, 15:34

The reason behind the bumpy performance is most likely the new portal. The new homepage uses 27 queries into the database (it was 26, but I added one a few minutes ago due to performance reasons). After those queries, the homepage gets built, compressed with gzip and finally transfered to your browser.

At least one of these 26 queries (see 1st post) was a real monster, here's why: In order to determine how many new posts have been made since your last visit, you need to query the posts table for all posts with a timestamp greater than the timestamp of your last visit. That's easy. For a query like this you should have an index on the timestamp column of the posts table, because otherwise MySQL (the database server) builds a temporary copy of the affected table, sorts it, returns your query and finally deletes the temporary table. Our posts table has a size of approx. 50 megabytes. Now, if you have 20 concurrent users accessing the homepage, the server has to build 20 temporary copies of the post table. Do the math, 20 x 50 megabytes = 1 gigabyte. That's one gigabyte of data which has to get moved and queried within seconds and every click on the "Home" link would add another 50 megabytes. That's why this query was a monster. We have an index on the timestamp column now, problem solved.

But, obviously (because of yesterday's bumpy ride) there have to be more sucky queries like this in the new portal software. I have split the complex query which is constructing the newsbits (by using the posts table) on the frontpage into two simple ones. The simple queries don't do any temporary tables and thus are significantly faster.

While analysing the queries, I found another two dealing with temp tables. One query affects the pollbox on the frontpage and the other one affects the "you have posted in this thread" hint (

the green dot) in the forum display. If we run into severe performance problems again, I'll turn this hint off. If that doesn't help, I'll start messing with the poll queries.

Patience, folks. We'll get this sorted.

Quote:

Originally Posted by Marcellus Wallace

Chill out folks, I'm on the mother****er.

Tim · 09 May 2005, 23:51

Still do find it strange that a page with pictures will load faster then the forum on some times. Even with your explenation R. Watched it now a couple of times, but it is not a special time that it is slow, or the number of people that are on the forums.

Know you are doing a great job with the forum. So don't see this as critizism!

Tim

dottie · 16 May 2005, 21:12

Is it me or in answering a topic, the reply takes a while to register on the forum?

SuperLoafMan · 16 May 2005, 21:17

Some days it is the same for me Dottie...but others it is fine....

Tim · 16 May 2005, 22:59

Only loading the front page will take me over a minute, then loading the forum also.... So it is a heavy site!

05 May 2005, 14:46	#6
Cathie Sweet But Sadistic Join Date: 18.06.2003 Location: Lincolnshire/Essex depending on the time of year! Posts: 4,098	Everything's working fine now here Last edited by Cathie; 05 May 2005 at 15:00.

05 May 2005, 02:02	#2
SuperLoafMan Meat Loaf Rocks! Join Date: 25.01.2005 Location: Barnsley, UK Posts: 3,887	wondered what was hapening....I have been experiencing the slow loading pages but since the forum has come back up tonight the pages are loading fine...and fast...what ever you did R it seems to have worked.....so far...

05 May 2005, 10:03	#3
Tim The Butcher Join Date: 14.04.2002 Location: Westervoort, The Netherlands Posts: 10,321	Last night i couldn't not come on the forums, and when i did, it was so slow, the internet explorer told me he couldn't load the page... But now it is fixed.

05 May 2005, 11:24	#4
Bren The Bat Join Date: 23.03.2003 Location: Roosting on a Silver-Black Phantom Bike. Posts: 10,017	likewise, was going so icredibly slow...but works just fine at moment

05 May 2005, 14:43	#5
Chris Mega Loafer Join Date: 14.04.2002 Location: Somewhere in the real world. Posts: 7,527	Ditto!!! Although i got a very worried text from someone at almost midnight worrying that she ahd pressed something to make it all break down!

05 May 2005, 15:37	#7
Tim The Butcher Join Date: 14.04.2002 Location: Westervoort, The Netherlands Posts: 10,321	indeed, at the moment it is, i think, a bit faster then it was before. Or maybe itis my connection

06 May 2005, 23:04	#8
Chris Mega Loafer Join Date: 14.04.2002 Location: Somewhere in the real world. Posts: 7,527	Seems to be chugging a bit tonight!!!

06 May 2005, 23:15	#9
R. You dig. Join Date: 02.04.2002 Location: On the fothermucker Posts: 7,179	Honestly, I don't know why. As far as I know, I have eliminated all "bad" queries. Sometimes it looks like that this server simply cannot execute 10 simple(!) database queries simultaneously. Makes me wonder how this iron survived the migration.

07 May 2005, 11:48	#10
Tim The Butcher Join Date: 14.04.2002 Location: Westervoort, The Netherlands Posts: 10,321	It is strange. The last couple of evenings the forum is so slow. Maybe it has to do with the latest adjustments. But it is getting slower, then you post something, it will take ages to load, if it will load. And eventually the forum crashes over here. Totally dead. After a while i'm able to go on the forums again. Very strange.

09 May 2005, 23:51	#12
Tim The Butcher Join Date: 14.04.2002 Location: Westervoort, The Netherlands Posts: 10,321	Still do find it strange that a page with pictures will load faster then the forum on some times. Even with your explenation R. Watched it now a couple of times, but it is not a special time that it is slow, or the number of people that are on the forums. Know you are doing a great job with the forum. So don't see this as critizism! Tim

16 May 2005, 21:12	#13
dottie queen post- whore extraordinaire Join Date: 29.01.2003 Location: Still Rockin' at "72" Posts: 13,181	Is it me or in answering a topic, the reply takes a while to register on the forum?

16 May 2005, 21:17	#14
SuperLoafMan Meat Loaf Rocks! Join Date: 25.01.2005 Location: Barnsley, UK Posts: 3,887	Some days it is the same for me Dottie...but others it is fine....

16 May 2005, 22:59	#15
Tim The Butcher Join Date: 14.04.2002 Location: Westervoort, The Netherlands Posts: 10,321	Only loading the front page will take me over a minute, then loading the forum also.... So it is a heavy site!