[Linux-ha-dev] LRM logging

[Andrew Beekhof]

On 8/9/06, Lars Marowsky-Bree <lmb at suse.de> wrote:
> On 2006-08-09T16:02:32, Andrew Beekhof <abeekhof at suse.de> wrote:
>
> > Its time to bring this up again... can someone please do some work on
> > the LRM logging?
> >
> > There is verbose and then there is:
> >
> > Aug 09 15:53:30 Running test SimulStop (c001n06)        [7]
> > Aug 09 15:56:53 BadNews: Aug  9 15:54:19 c001n03 lrmd: [28984]:
> > ERROR: cl_log: 293 messages were dropped
> > Aug 09 15:56:53 BadNews: Aug  9 15:54:19 c001n02 lrmd: [31993]:
> > ERROR: cl_log: 141 messages were dropped
> > Aug 09 15:56:55 BadNews: Aug  9 15:54:53 c001n07 lrmd: [11102]:
> > ERROR: cl_log: 108 messages were dropped
> > Aug 09 15:56:55 BadNews: Aug  9 15:54:53 c001n06 lrmd: [11166]:
> > ERROR: cl_log: 70 messages were dropped
> > Aug 09 15:56:56 BadNews: Aug  9 15:54:54 c001n07 lrmd: [11102]:
> > ERROR: cl_log: 106 messages were dropped
> > Aug 09 15:56:56 BadNews: Aug  9 15:54:54 c001n04 lrmd: [12569]:
> > ERROR: cl_log: 18 messages were dropped
> > Aug 09 15:56:56 BadNews: Aug  9 15:54:54 c001n07 lrmd: [11102]:
> > ERROR: cl_log: 153 messages were dropped
> > Aug 09 15:56:56 BadNews: Aug  9 15:54:54 c001n04 lrmd: [12569]:
> > ERROR: cl_log: 124 messages were dropped
> > Aug 09 15:56:56 BadNews: Aug  9 15:54:54 c001n04 lrmd: [12569]:
> > ERROR: cl_log: 493 messages were dropped
> > Aug 09 15:56:56 BadNews: Aug  9 15:54:54 c001n06 lrmd: [11166]:
> > ERROR: cl_log: 849 messages were dropped
> > Aug 09 15:56:56 BadNews: Aug  9 15:54:55 c001n04 lrmd: [12569]:
> > ERROR: cl_log: 104 messages were dropped
> > Aug 09 15:56:56 BadNews: Aug  9 15:54:55 c001n04 lrmd: [12569]:
> > ERROR: cl_log: 156 messages were dropped
> > Aug 09 15:56:57 BadNews: Aug  9 15:54:55 c001n06 lrmd: [11166]:
> > ERROR: cl_log: 22 messages were dropped
> > Aug 09 15:56:57 BadNews: Aug  9 15:54:55 c001n06 lrmd: [11166]:
> > ERROR: cl_log: 5 messages were dropped
> > Aug 09 15:56:57 BadNews: Aug  9 15:54:55 c001n06 lrmd: [11166]:
> > ERROR: cl_log: 245 messages were dropped
> > Aug 09 15:56:57 BadNews: Aug  9 15:54:56 c001n04 lrmd: [12569]:
> > ERROR: cl_log: 22 messages were dropped
> > Aug 09 15:56:57 BadNews: Aug  9 15:54:56 c001n04 lrmd: [12569]:
> > ERROR: cl_log: 208 messages were dropped
> > Aug 09 15:56:57 BadNews: Aug  9 15:54:56 c001n06 lrmd: [11166]:
> > ERROR: cl_log: 16 messages were dropped
>
> Omg, that's horrible. Do we have any idea what kind of messages cause
> this?

Not really, the messages are "lost" remember ;-)

But really, take a look at the LRM logs even when its not loosing
messages and its pretty obvious that its just horribly verbose at
everything.  The only thing that has changed is that there are more
resources in a 6 node cluster.

>
> (ie, this is a polite question about where the logfile is laying around.
> ;-)

been there done that

> > And thats only with 6 CTS nodes.
>
> Hrm. A work-around might be to try and run CTS without debug logging.

these messages dont impact the tests at all, they just show up as BadNews.
pretending the problem doesnt exist isnt going to help much.

> > This is a real problem.  The CRM is horribly verbose and still
> > doesn't deluge the logging queue the way the LRM does.
>
> Nod. Any bugzilla I should be aware of?

The most recent incarnation is 1380