database locks

[Database Administrator]

I have 2 applications, one that inserts records into a database and
the other that gets and deletes records from the same database.

There can be many instances of each application. My problem is that
frequently the database will run out of locks and everything comes to
a halt. I'll end up with hundreds of applications waiting for a lock.
I believe anyways. db_stat -c will show all locks used, db_stat -d
{database} won't even run.

I generally have to kill all instances of the application an
db_recover. Could some please look and perhaps tell me what might be
causing my issue.

This is the code for the insert application but the get/delete
application is exactly the same in terms of the database setup and
connection.

Thanks, the Berkeley DB is perfect for the most part, I'd really hate
to switch to something more heavy because of a dumb problem of mine.


/* To compile this application I .....
gcc -o dirqueuei -ldb dirqueuei.c
*/

#include
#include
#include
#include
#include
#include
#include

#define DEBUG 0
#define DATABASE "/home/imaging/db/queue.db"
#define MAXDATALENGTH 256
#define DB_HOME_DIR "/home/imaging/db"


int main(int argc, char *argv[]) {

DB *db;
int ret, i;
DB_ENV *dbenv;

DBT key, data;

struct stat fileinfo;

long stamp;
char file[MAXDATALENGTH];

char message[255];


if(!argc>1) {
sprintf(message, "usage: %s {dicom file name}\n",
argv[0]);
syslog(LOG_NOTICE, message);
exit(0);
}

if((dbenv=db_setup(DB_HOME_DIR))==NULL) {
sprintf(message, "db_setup error");
syslog(LOG_NOTICE, message);
exit(0);
}

if((ret=db_create(&db,dbenv,0))!=0) {
sprintf(message,"db_create: %s",db_strerror(ret));
fprintf(stderr, "%s\n", message);
syslog(LOG_NOTICE, message);
exit(0);
} else if(DEBUG) {
fprintf(stderr, "db_create:success");
}

#if DB_VERSION_MAJOR == 4 && DB_VERSION_MINOR != 3
db->set_errpfx(db, argv[0]);
db->set_errcall(db, error_handler);
#endif

if((ret=db->set_flags(db, DB_DUP))!=0) {
db->err(db, ret, "%s", "set_flags");
exit(0);
}
if((ret=db->set_bt_compare(db,compare_keys))!=0) {
db->err(db, ret, "%s", "set_bt_compare");
exit(0);
}

if((ret=db->open(db,NULL,DATABASE,NULL,DB_BTREE,DB_CREATE,0664 ))!=0)
{
db->err(db, ret, "%s open failed", DATABASE);
exit(0);
} else if(DEBUG) {
fprintf(stderr, "db_open: success\n");
}

for(i=1; i
memset(&key, 0, sizeof(DBT));
memset(&data, 0, sizeof(DBT));

if(stat(argv[i], &fileinfo)!=0) {
sprintf(message, "error: stat(%s)", argv[i]);
syslog(LOG_NOTICE, message);
continue;
}

stamp = (long)fileinfo.st_mtime;

key.data = &stamp;
key.size = sizeof(long);

if(DEBUG) printf("processing:%s\n", argv[i]);

data.data = argv[i];
data.size = strlen(argv[i])+1;

if((ret=db->put(db, NULL, &key, &data, 0))!=0) {
db->err(db, ret, "Put failed:%s", argv[i]);
} else if(DEBUG) {
fprintf(stderr, "Record Inserted\n");
}
}
(void)db->close(db, 0);
dbenv->close(dbenv, 0);
return(0);
}
int compare_keys(DB *db, const DBT *val1, const DBT *val2) {
long a, b;
memcpy(&a, val1->data, sizeof(long));
memcpy(&b, val2->data, sizeof(long));
if(DEBUG) { fprintf(stderr, "compare_keys: %ld - %ld\n", a,
b); }
return (a - b);
}
void error_handler(const char *errpfx, char *msg) {
char message[255];
sprintf(message, "%s:%s", errpfx, msg);
fprintf(stderr, "%s\n", message);
syslog(LOG_NOTICE, message);
}
DB_ENV *db_setup(char *home) {
int ret;
DB_ENV *dbenv;
if((ret=db_env_create(&dbenv, 0))!=0) {
fprintf(stderr, "db_env_create:%s\n",
db_strerror(ret));
return(NULL);
}
if((ret=dbenv->set_cachesize(dbenv, 0, 5*1024*1024, 0))!=0) {
fprintf(stderr, "set_cachesize:%s\n",
db_strerror(ret));
return(NULL);
}
if((ret=dbenv->open(dbenv, home,
DB_CREATE|DB_INIT_LOCK|DB_INIT_MPOOL,0))!=0) {
fprintf(stderr, "environment open:%s\n", home);
return(NULL);
}
return(dbenv);
}

[Database Administrator]

> There can be many instances of each application. My problem is that
> frequently the database will run out of locks and everything comes to
> a halt. I'll end up with hundreds of applications waiting for a lock.
> I believe anyways. db_stat -c will show all locks used, db_stat -d
> {database} won't even run.

Are you sure you are running out of locks, and not out of lockers?

[Database Administrator]

No actually. I'm using my memory and now that I look again at the
results of a working db_stat -c I believe all the numbers in the
section below grow very large. I wait for it to happen again and grab
some exact information.

Sorry

0 Number of current locks
7 Maximum number of locks at any one time
0 Number of current lockers
8 Maximum number of lockers at any one time
0 Number of current lock objects
5 Maximum number of lock objects at any one time


On Sat, 27 Jan 2007 13:06:46 +0100, Florian Weimer
wrote:

>> There can be many instances of each application. My problem is that
>> frequently the database will run out of locks and everything comes to
>> a halt. I'll end up with hundreds of applications waiting for a lock.
>> I believe anyways. db_stat -c will show all locks used, db_stat -d
>> {database} won't even run.
>
>Are you sure you are running out of locks, and not out of lockers?