[cobalt-security] RaQ3/4 + glibc Update (+ ChiliSoft) + MySQL = Problem

[Michael Stauber <cobalt@xxxxxxxxxxxxxx>]

Hi all,

I just had a little chitchat in with a customer who at the moment is in a 
very tight spot: He has Cobalt Professional Support on one side and a broken 
RaQ4 on the other.

All those who haven't yet installed RaQ4-All-Security-2.0.1-13453.pkg  (or 
its counterpart for the RaQ3: RaQ3-All-Security-4.0.1-13453.pkg) should 
better wait with that patch for a little longer.

Topic basically says it all: If you got a RaQ4 with glibc Update 2.0.1 
installed (RaQ4-All-Security-2.0.1-13453.pkg), have MySQL and possibly 
ChiliSoft running, then its a good idea to make sure that your backups are up 
to date, as you *eventually* might need 'em. 

Which would be worst case. Best case is that you're either not affected or 
that SUN/Cobalt comes up with a fix real soon.

The problem is this: The glibc replacement causes trouble in regards to 
memory allocation. Every time a process related to MySQL and/or ChiliSoft 
starts (or any process for that matter), then memory is allocated - or in 
other words: reserved so that only this single process can access this memory 
area.

Once the process ends, then the memory is released back to the memory pool so 
that other applications can use it.

However, since the glibc update not the entire allocated memory is relased. A 
small chunk of it remains unuseable for all other applications. 

This kind of memory leakages sometimes happen on a small scale, but since the 
glibc patch they can grow to dramatic proportions under certain circumstances.

If you got a heavily utilized MySQL database, which frequently spawns childs 
and terminates them soon thereafter, then your availably memory decreases 
steadily up to a point where there is no more memory available for any 
process. At that time the system gets instable and crashes.

Who is affected? As far as I know after some toying around with it:

All RaQ4's with RaQ4-All-Security-2.0.1-13453.pkg and a utilized MySQL 
database.

All RaQ3's with RaQ3-All-Security-4.0.1-13453.pkg and a utilized MySQL 
database.

Why I think RaQ3's are affected as well? RaQ3 with 384MB of memory and a 
130MB MySQL database, all patches installed. 

After 2 hours of uptime:

MemFree:     184032 kB
SwapFree:    131024 kB

After one day of uptime:

MemFree:       2840 kB
SwapFree:    131024 kB

Restarting MySQL & Apache and INETd usually freed up 174 MB of memory in one 
go. Now it releases only 40-43 MB. Only chance to free up more is now to go 
for a reboot.

Solution to the problem? Pray. For a better glibc patch.

-- 

With best regards,

Michael Stauber
mstauber@xxxxxxxxxxxxxx
Unix/Linux Support Engineer