Re: [cooker-server] openldap package

[Buchan Milne]

On Tue, 17 Feb 2004, Tibor Pittich wrote:

> On 17. February 2004 at 11:54, Buchan Milne wrote:
> 
> > On Tue, 17 Feb 2004, Tibor Pittich wrote:
> 
> > > i'm using at both side openldap-2.1.26-1mdk (sic).
> > 
> > Ahh, no, that might be the cause ... you may want to revert to 2.1.25, or 
> > apply a patch from CVS for 2.1.26 (I have one somewhere ...0.
> 
> arrgh. ok, my fault that i'm risking update to this release, thanks
> for your explaintation, i will try revert to 2.1.25 tonight.
> 
> is there possible problems due to usage of db 4.2 in your latest
> package, when i'm now using 2.1.26 with db 4.1 ?

The only issue is the log format has changed, so it's safest to stop, 
db_recover, then upgrade, possibly removing old logs, but I have done the 
db_recover in the %pre of the new packages, so it should just work fine. 
I'm not going to do anything about the logs (you will just get a warning 
about old log format when running db_recover again.

> 
> > > maybe, when we ship openldap with preconfigured bdb database add some
> > > conservative options to improve bdb. i mean checkpoint and for
> > > perfomance goal cachesize. what do you think?
> > 
> > I think default checkpointing should be ok (but I would like to hear other 
> > opinions)
> 
> by default both checkpoint arguments are set to zero (slapd-bdb man page
> say):
> checkpoint <kbyte> <min>
>        Specify the frequency for checkpointing the database transaction
>        log.   A  checkpoint  operation  flushes the database buffers to
>        disk and writes a checkpoint record in the log.  The  checkpoint
>        will occur if either <kbyte> data has been written or <min> min-
>        utes have passed since  the  last  checkpoint.   Both  arguments
>        default to zero, in which case they are ignored.  See the Berke-
>        ley DB reference guide for more details.
> 
> i think, that good default conservative setting should be:
> checkpoint 512 720

12 hours? I think it's probably worthwhile checkpointing more often than 
that. The people at Standford I think are checkpointing every 5 minutes. 
512kb is maybe a bit big? Maybe:

256 60
?

> 
> > I am only just now experimenting with cache sizes
> 
> hm, i don't have idea about how large usage of ldap database we can
> suppose, but maybe default bdb cache size 1000 entries will be good, but
> we can add this into config file as comment to easy end-user tunning.
>

Here's a good post fom one of the OpenLDAP developers on cache sizing:
http://www.openldap.org/lists/openldap-software/200311/msg00469.html

Note that for bdb, cache settings are now in DB_CONFIG in the daabase 
directory, I have added a default one, but you will see the cache settings 
are currently commented.

I woner aobut putting some of the thumb-rule sizings in the init script 
for convenience, the user can be warned when stopping the LDAP server 
whether their cache is too small?
 
> > > > I'm considering adding a weekly and a monthly cron job which removes old 
> > > > transaction logs,
> > > 
> > > sorry? can you explain this please.
> > 
> > The transaction logs are only necessary until you do the next full 
> > databse archive/backup (unless you want to restore to an older point).
> > Once you  have a new full backup, you can remove the transaction logs that 
> > were not in use just before you did the backu.
> 
> ah, you mean as full backup copying data files from /var/lib/ldap
> directory? hmm, this is imho not common way to backup ldap data. more
> common way is ldapsearch and save data in ldif format, imho.
> 

But, it doesn't allow hot backups , and restoring from ldif is really slow 
(approx 2 hours for 150000 entries).

> > It may be an idea to rotate them into a seperate directory, and remove 
> > them on the next cron job.
> 
> i think, that we don't touch to these files, espec with cron jobs.
> 

Instead, we could leave them and have the partition overflow (on an 
import, transaction logs are bigger than the db itself, by a few times 
...).

I was thinking of having  it do nothing by default, and configurable to 
move/remove them by configuration.

Regards,
Buchan