unable to allocate space in the buffer cache

suk926@yahoo.com (Bryan Suk) wrote in message news:<95c94ee7.0404082231.1db87c66@posting.google.com>...

> I am using bdb 4.2
> I get "unable to allocate space in the buffer cache".
>
> I set the cache size to 512M.
> The total size of uncomitted data may be over the cache size.
>
> Does bdb hold uncommitted data in the cache?

Berkeley DB does not hold uncommitted data in the cache.

Generally, this only happens when the page size is large, and
the application has enough threads of control in the database
at the same time, that all of the cache pages are pinned.

Would it be possible for you to post a test program that displays
this failure?

Regards,
--keith

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Keith Bostic bostic@sleepycat.com
Sleepycat Software Inc. keithbosticim (ymsgid)
118 Tower Rd. +1-781-259-3139
Lincoln, MA 01773 http://www.sleepycat.com

There were 60 threads running at the same time, each opening its own transaction.

30 threads were simply receiving data and putting them into bdb.

30 other threads were getting data from bdb and consuming them.

I checked the cache with db_stat.

There were multiple entries for file. Of them, there were multiple dead files.
What are those dead files?
Also, does each file entry occupy 8192 bytes in the cache?

Thank you.

suk926@yahoo.com (Bryan Suk) wrote in message news:<95c94ee7.0404111941.5d54053a@posting.google.com>...

> There were 60 threads running at the same time, each opening its own
> transaction.
>
> 30 threads were simply receiving data and putting them into bdb.
>
> 30 other threads were getting data from bdb and consuming them.

That data access pattern should fit into a cache size of 512MB
without any problem, I would think.

> I checked the cache with db_stat.
>
> There were multiple entries for file. Of them, there were multiple
> dead files. What are those dead files?

Dead files are files that are no longer opened by any thread
of control, but for which there are still pages in the cache.

> Also, does each file entry occupy 8192 bytes in the cache?

There is per-file information in the cache, I don't know
off-hand how much room it takes up. It's not fixed-size in
nature, however, for example, there are file names being
stored.

FWIW, you're approaching the problem in the right way --
you should be able to tell from a display of the cache
statistics (perhaps using the -MA option) exactly what is
taking up the space in the cache.

Regards,
--keith

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Keith Bostic bostic@sleepycat.com
Sleepycat Software Inc. keithbosticim (ymsgid)
118 Tower Rd. +1-781-259-3139
Lincoln, MA 01773 http://www.sleepycat.com

Hello.

I am using Win 2000.
I am using default option for shared memory region(not private, not system_mem).

In this case, where does the data in the cache go?

Does it go into one of the __db files?

If that is the case, if the cache filled up,
would the file be over 512M?

My question is, where does the cache reside?

Thank you very much.

bostic@sleepycat.com (Keith Bostic) wrote in message news:...
> suk926@yahoo.com (Bryan Suk) wrote in message news:<95c94ee7.0404111941.5d54053a@posting.google.com>...
>
> > There were 60 threads running at the same time, each opening its own
> > transaction.
> >
> > 30 threads were simply receiving data and putting them into bdb.
> >
> > 30 other threads were getting data from bdb and consuming them.
>
> That data access pattern should fit into a cache size of 512MB
> without any problem, I would think.
>
> > I checked the cache with db_stat.
> >
> > There were multiple entries for file. Of them, there were multiple
> > dead files. What are those dead files?
>
> Dead files are files that are no longer opened by any thread
> of control, but for which there are still pages in the cache.
>
> > Also, does each file entry occupy 8192 bytes in the cache?
>
> There is per-file information in the cache, I don't know
> off-hand how much room it takes up. It's not fixed-size in
> nature, however, for example, there are file names being
> stored.
>
> FWIW, you're approaching the problem in the right way --
> you should be able to tell from a display of the cache
> statistics (perhaps using the -MA option) exactly what is
> taking up the space in the cache.
>
> Regards,
> --keith
>
> =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
> Keith Bostic bostic@sleepycat.com
> Sleepycat Software Inc. keithbosticim (ymsgid)
> 118 Tower Rd. +1-781-259-3139
> Lincoln, MA 01773 http://www.sleepycat.com

suk926@yahoo.com (Bryan Suk) wrote in message news:<95c94ee7.0404121721.f57e87e@posting.google.com>...

> I am using Win 2000. I am using default option for shared memory
> region(not private, not system_mem).
>
> In this case, where does the data in the cache go?
>
> Does it go into one of the __db files?

Yes, the cache will be instantiated in one of the __db.XXX
files.

> If that is the case, if the cache filled up, would the file
> be over 512M?

The default cache size is 256KB, and may not be specified as less than
20KB. Any cache size less than 500MB is automatically increased by 25%
to account for buffer pool overhead; cache sizes larger than 500MB are
used as specified. The current maximum size of a single cache is 4GB.
(All sizes are in powers-of-two, that is, 256KB is 2^32 not 256,000.)
For information on tuning the Berkeley DB cache size, see Selecting a
cache size:

http://www.sleepycat.com/docs/ref/am...cachesize.html

> My question is, where does the cache reside?

In your case, the cache resides in filesystem-backed shared
memory.

Regards,
--keith

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Keith Bostic bostic@sleepycat.com
Sleepycat Software Inc. keithbosticim (ymsgid)
118 Tower Rd. +1-781-259-3139
Lincoln, MA 01773 http://www.sleepycat.com