Chiffboard Slowdown 2010-12-09

Board policies and official announcements about Chiff and Fipple and the forums. Please read!
User avatar
MTGuru
Posts: 18663
Joined: Sat Sep 30, 2006 12:45 pm
antispam: No
Please enter the next number in sequence: 8
Location: San Diego, CA

Chiffboard Slowdown 2010-12-09

Post by MTGuru »

You may have noticed that Chiffboard response has been sluggish for the past few days. We're working with our hosting provider to identify and fix the source of the problem (usually an issue with the shared back-end mySQL server). We'll keep you posted. Thanks!

MTGuru / Moderator
Vivat diabolus in musica! MTGuru's (old) GG Clips / Blackbird Clips

Joel Barish: Is there any risk of brain damage?
Dr. Mierzwiak: Well, technically speaking, the procedure is brain damage.
User avatar
I.D.10-t
Posts: 7657
Joined: Wed Dec 17, 2003 9:57 am
antispam: No
Location: Minneapolis, MN, USA, Earth

Re: Chiffboard Slowdown

Post by I.D.10-t »

Thanks for all of your work.
"Be not deceived by the sweet words of proverbial philosophy. Sugar of lead is a poison."
User avatar
MTGuru
Posts: 18663
Joined: Sat Sep 30, 2006 12:45 pm
antispam: No
Please enter the next number in sequence: 8
Location: San Diego, CA

Re: Chiffboard Slowdown

Post by MTGuru »

You're very welcome. An hour with tech support on Tuesday, a couple of hours today. And things do seem better now. Fingers crossed ...
Vivat diabolus in musica! MTGuru's (old) GG Clips / Blackbird Clips

Joel Barish: Is there any risk of brain damage?
Dr. Mierzwiak: Well, technically speaking, the procedure is brain damage.
User avatar
Denny
Posts: 24005
Joined: Mon Nov 17, 2003 11:29 am
antispam: No
Location: N of Seattle

Re: Chiffboard Slowdown

Post by Denny »

better for a bit....seems to be backsliding again
Picture a bright blue ball just spinning, spinning free
It's dizzying, the possibilities. Ashes, Ashes all fall down.
User avatar
MTGuru
Posts: 18663
Joined: Sat Sep 30, 2006 12:45 pm
antispam: No
Please enter the next number in sequence: 8
Location: San Diego, CA

Re: Chiffboard Slowdown

Post by MTGuru »

Yes, it's worse today, very erratic. We're working on it.
Vivat diabolus in musica! MTGuru's (old) GG Clips / Blackbird Clips

Joel Barish: Is there any risk of brain damage?
Dr. Mierzwiak: Well, technically speaking, the procedure is brain damage.
User avatar
Nanohedron
Moderatorer
Posts: 38212
Joined: Wed Dec 18, 2002 6:00 pm
antispam: No
Please enter the next number in sequence: 8
Tell us something.: Been a fluter, citternist, and uilleann piper; committed now to the way of the harp.

Oh, yeah: also a mod here, not a spammer. A matter of opinion, perhaps.
Location: Lefse country

Re: Chiffboard Slowdown

Post by Nanohedron »

Makes for a real chore when you're trying to go about the business of oppressing folks, let me tell you.
"If you take music out of this world, you will have nothing but a ball of fire." - Tribal musician
User avatar
MTGuru
Posts: 18663
Joined: Sat Sep 30, 2006 12:45 pm
antispam: No
Please enter the next number in sequence: 8
Location: San Diego, CA

Re: Chiffboard Slowdown

Post by MTGuru »

Sorry folks, still no joy. And apparently, our live tech support staff go home at 5 PM PST. I guess that's when the Internet closes down for the night. :evil:
Vivat diabolus in musica! MTGuru's (old) GG Clips / Blackbird Clips

Joel Barish: Is there any risk of brain damage?
Dr. Mierzwiak: Well, technically speaking, the procedure is brain damage.
User avatar
I.D.10-t
Posts: 7657
Joined: Wed Dec 17, 2003 9:57 am
antispam: No
Location: Minneapolis, MN, USA, Earth

Re: Chiffboard Slowdown

Post by I.D.10-t »

Nanohedron wrote:Makes for a real chore when you're trying to go about the business of oppressing folks, let me tell you.
I just go to Wikileaks to see the non-censored version.
"Be not deceived by the sweet words of proverbial philosophy. Sugar of lead is a poison."
User avatar
MTGuru
Posts: 18663
Joined: Sat Sep 30, 2006 12:45 pm
antispam: No
Please enter the next number in sequence: 8
Location: San Diego, CA

Chiffboard Outage

Post by MTGuru »

Folks, we're sorry about the unplanned outage this evening, Thursday, from around 1800 to 2100 PST.

To address the recurring slowdown problem, our host decided to move our databases to a different server. Assurances that this would entail no Chiffboard downtime were apparently not so assured, or we would have alerted everyone in advance.

Anyway, we're running on a new SQL server now, and we'll keep an eye on the performance issues. Thanks for your patience!
Vivat diabolus in musica! MTGuru's (old) GG Clips / Blackbird Clips

Joel Barish: Is there any risk of brain damage?
Dr. Mierzwiak: Well, technically speaking, the procedure is brain damage.
User avatar
kkrell
Posts: 4831
Joined: Mon Jul 29, 2002 6:00 pm
antispam: No
Please enter the next number in sequence: 8
Tell us something.: Mostly producer of the Wooden Flute Obsession 3-volume 6-CD 7-hour set of mostly player's choice of Irish tunes, played mostly solo, on mostly wooden flutes by approximately 120 different mostly highly-rated traditional flute players & are mostly...
Location: Los Angeles
Contact:

Re: Chiffboard Slowdown

Post by kkrell »

Still slow.
User avatar
Hotblack
Posts: 471
Joined: Thu Jan 28, 2010 9:48 am
antispam: No
Location: Upstairs in the spare room, Oxfordshire

Re: Chiffboard Slowdown

Post by Hotblack »

^WHS^

It had seemed to improve just recently but since yesterday (23rd) it's gone slow again.

Just sayin'.
Cheers

David

I can resist everything except temptation - Oscar Wilde.
User avatar
kkrell
Posts: 4831
Joined: Mon Jul 29, 2002 6:00 pm
antispam: No
Please enter the next number in sequence: 8
Tell us something.: Mostly producer of the Wooden Flute Obsession 3-volume 6-CD 7-hour set of mostly player's choice of Irish tunes, played mostly solo, on mostly wooden flutes by approximately 120 different mostly highly-rated traditional flute players & are mostly...
Location: Los Angeles
Contact:

Re: Chiffboard Slowdown

Post by kkrell »

Example times as of now (all other out-of-cache sites I visit responding promptly, so unlikely to be my connection):

From "View new posts" - click on Board index: 40 seconds to reach Board index.

From Board index to "Mark forums read": 25 seconds.

From "View new posts" - click on topic: 20 seconds to begin switch to topic thread (then dependent on content, linked pictures, etc.)

Read PM messages - click on message: 50 seconds before display.

Note that I was logged in at all times. Wonder if that is a factor (tracking viewed messages).

(edited, after a 50-second wait, to answer NO). Same performance in Firefox and IE in Windows XP, 100-base-T, DSL)
User avatar
MTGuru
Posts: 18663
Joined: Sat Sep 30, 2006 12:45 pm
antispam: No
Please enter the next number in sequence: 8
Location: San Diego, CA

Re: Chiffboard Slowdown

Post by MTGuru »

Thanks for the feedback, everyone.

We're pretty sure that the slowdown, when it occurs, has very little to do with any normal activities or operations of the board. With our current configuration, when things are working, they're working just fine.

Without too much detail ... The Chiffboard runs on 2 different servers: 1. A front-end http server for phpBB, implemented as a virtual private server (VPS) hosted on a shared CPU; and 2. A back-end mySQL server, also shared, which holds the databases. The board data is fairly large by now, around 3 GB.

There's a correlation between the responsiveness of the board and CPU memory usage, which skyrockets to 2 or 3 times "normal" during slowdowns. Ordinarily, this might suggest a software memory leak. But re-booting our servers has no effect, so it's unlikely that's the root of the problem.

More likely, we think, the cause lies with our shared resources. That is, other users or sites sharing our CPUs and sucking up all available memory, cycles, and bandwidth. This would account for the seemingly random nature of the problem. If another site experiences excessive usage, or their software begins firing off rogue threads and processes that are left unchecked on the physical CPU, the Chiffboard grinds to a crawl. It's like a DOS (denial of service) attack as a side effect of our VPS configuration.

Yesterday we moved the databases to a different back-end SQL server with smaller network latency (faster response).

Dale and I have been discussing other steps we hope to take after the holidays to address the problem, and we'll keep you posted. Meanwhile, keep thinking happy, happy thoughts. :wink:
Vivat diabolus in musica! MTGuru's (old) GG Clips / Blackbird Clips

Joel Barish: Is there any risk of brain damage?
Dr. Mierzwiak: Well, technically speaking, the procedure is brain damage.
User avatar
kkrell
Posts: 4831
Joined: Mon Jul 29, 2002 6:00 pm
antispam: No
Please enter the next number in sequence: 8
Tell us something.: Mostly producer of the Wooden Flute Obsession 3-volume 6-CD 7-hour set of mostly player's choice of Irish tunes, played mostly solo, on mostly wooden flutes by approximately 120 different mostly highly-rated traditional flute players & are mostly...
Location: Los Angeles
Contact:

Re: Chiffboard Slowdown

Post by kkrell »

If the other sites are shopping sites, maybe the excess usage will go away after the holidays.

Or Julian Assange's friends have mistaken us for Amazon or Bank of America :lol: .
User avatar
Dale
The Landlord
Posts: 10293
Joined: Wed May 16, 2001 6:00 pm
Please enter the next number in sequence: 1
Location: Chiff & Fipple's LearJet: DaleForce One
Contact:

Re: Chiffboard Slowdown

Post by Dale »

Our old friend Rich Lafferty has agreed to look into our situation. He's on it, but may not be able to devote much time to it until after the holidays.

Thank you, Rich!
Post Reply