virtual address space size of a proces

[Database Administrator]

Hi,

I am looking to work on unix systems. I want to get the total virtual
address space of a process, the used virtual memory of a process i am
able to get without any problem. I have tried using getrlimit but that
gives only RLIMIT_INIFINITY or RLIMIT64_INFINITY if i use the
getrlimit64.

Can you give me an idea on how much virtual addressable space is there
in each respective unix system for a user address space as i need to
do my memory allocations based on that and show a warning in an
application when the memory is about to get depleted?


Best regards,
RUI

[Database Administrator]

On Nov 4, 9:25*am, guidevelo...@gmail.com wrote:
> Hi,
>
> I am looking to work on unix systems. I want to get the total virtual
> address space of a process, the used virtual memory of a process i am
> able to get without any problem. I have tried using getrlimit but that
> gives only *RLIMIT_INIFINITY or RLIMIT64_INFINITY if i use the
> getrlimit64.
>
> Can you give me an idea on how much virtual addressable space is there
> in each respective unix system for a user address space as i need to
> do my memory allocations based on that and show a warning in an
> application when the memory is about to get depleted?
>
> Best regards,
> RUI

I'm not sure you can do much better than 2^sizeof(void *).

DS

[Database Administrator]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

David Schwartz wrote:
> I'm not sure you can do much better than 2^sizeof(void *).

Even that is wrong:
1. sizeof returns size in bytes. To get amount of memory that *CPU* can
access you have to get pointer's size in bits,
2. memory accessible for *CPU* (and kernel in that case) != memory
accessible for user space program.

Pawel Dziepak
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org

iEYEARECAAYFAkkQvWMACgkQPFW+cUiIHNo6ZwCghDNSnUXy83 feFL8jsjyALuBm
bKEAn0wDCnGPaKlY6c3aELk6kZV4LCAF
=Qkry
-----END PGP SIGNATURE-----

[Database Administrator]

On Nov 4, 9:47*pm, and...@cucumber.demon.co.uk (Andrew Gabriel) wrote:
> In article <53671ae0-eeb2-4d28-83f6-4073a5742...@w1g2000prk.googlegroups.com>,
> * * * * guidevelo...@gmail.com writes:
>
> > Hi,
>
> > I am looking to work on unix systems. I want to get the total virtual
> > address space of a process, the used virtual memory of a process i am
> > able to get without any problem. I have tried using getrlimit but that
> > gives only *RLIMIT_INIFINITY or RLIMIT64_INFINITY if i use the
> > getrlimit64.
>
> > Can you give me an idea on how much virtual addressable space is there
> > in each respective unix system for a user address space as i need to
> > do my memory allocations based on that and show a warning in an
> > application when the memory is about to get depleted?
>
> For a 32 bit process, it's going to be 4GB minus an ammount
> which depends on the OS and CPU. Some CPU's run the kernel
> in the same address space although hidden from the user-land
> process (e.g. sun4c, sun4m, sun4d, x86) and have to allow
> space for the kernel (256MB or 512MB, or 1GB in the case of
> Windows). Others have a completely separate kernel address
> space (sun4u, sun4v) so you've got the full 4GB available.
> There may still be some OS/CPU combinations which impose
> smaller address spaces due to limitations in MMU's page
> tables, etc, but with most OS/CPU's being 64 bit nowadays,
> they don't normally have any problem providing the full
> address space of a 32 bit application, as what's small
> compared with what they're expected to deal with.
>
> In principle, the same applies for 64 bit processes (but with
> 1.8E9 GB address space) but in practice I doubt any OS/CPU can
> setup tables to address that, so the limit is going to be down
> to the particular OS/CPU combination.
>
> --
> Andrew Gabriel
> [email address is not usable -- followup in the newsgroup]

Hi,

I understand that the address space is 4 GB which could be used by
kernel space or maybe completely used by user space, however i would
like to know for solaris 2.4 ownwards, how can i programatically get
the virtual memory available to the process so that i can show a
warning to the user when further allocation is not possible.

Same goes for hp ux 10 if somebody can help?
And also for suse linux 9 and redhat 9 ownwards?

Isn't there a common api or a configuration file where i can retrieve
this information from, for most of the unixes?

Best regards,
RUI

[Database Administrator]

guideveloper@gmail.com wrote:
> the used virtual memory of a process i am
> able to get without any problem.

How can you get this for HP-UX?

> I have tried using getrlimit but that gives only RLIMIT_INIFINITY

getrlimit(2) works for HP-UX. It returns either ulimit or maxdsiz(5).

> Same goes for hp-ux 10 if somebody can help?

This is no longer supported. Only 11.11 and up.

[Database Administrator]

guideveloper@gmail.com writes:

> Hi,
>
> I understand that the address space is 4 GB which could be used by
> kernel space or maybe completely used by user space, however i would
> like to know for solaris 2.4 ownwards, how can i programatically get
> the virtual memory available to the process so that i can show a
> warning to the user when further allocation is not possible.
>
> Same goes for hp ux 10 if somebody can help?
> And also for suse linux 9 and redhat 9 ownwards?
>
> Isn't there a common api or a configuration file where i can retrieve
> this information from, for most of the unixes?

On a modern operating system, there really isn't a good way to find out
"how much more memory can I allocate before I run out". The amount of
memory available to your process depends on many factors, and is
constantly changing based on what else is happening in the system.

Really, about the only thing you can reliably do is to try the
allocation and see if it succeeds. You can try to write the program in
such a way that it will behave gracefully when this occurs, perhaps
ask the user to free up some memory and then try to continue.

Perhaps you are trying to tune the amount of memory used by your program
based on how much is available. A better way is to ask the user to
specify how much to use, since the user might have a better idea of how
much memory is present, and how much is needed for other tasks, present
and future. In my experience, programs that try to auto-detect the
right thing to do usually get it wrong, are hard to understand because
their behavior keeps changing, and are generally annoying to use.

[Database Administrator]

On 2008-11-05 00:58:48 +0000, Nate Eldredge said:

> guideveloper@gmail.com writes:
>
>> Hi,
>>
>> I understand that the address space is 4 GB which could be used by
>> kernel space or maybe completely used by user space, however i would
>> like to know for solaris 2.4 ownwards, how can i programatically get
>> the virtual memory available to the process so that i can show a
>> warning to the user when further allocation is not possible.
>>
>> Same goes for hp ux 10 if somebody can help?
>> And also for suse linux 9 and redhat 9 ownwards?
>>
>> Isn't there a common api or a configuration file where i can retrieve
>> this information from, for most of the unixes?
>
> On a modern operating system, there really isn't a good way to find out
> "how much more memory can I allocate before I run out". The amount of
> memory available to your process depends on many factors, and is
> constantly changing based on what else is happening in the system.
>
> Really, about the only thing you can reliably do is to try the
> allocation and see if it succeeds. You can try to write the program in
> such a way that it will behave gracefully when this occurs, perhaps
> ask the user to free up some memory and then try to continue.

That is usually the right approach, but AIUI it falls apart slightly on
systems which overcommit VM, like Linux. That is, they return memory to
the process even if there isn't enough VM. When you try to use the
returned memory... either you get lucky and the system's got enough VM
again or the OS has to kill some other poor process to make some VM for
you.

I've a feeling HP-UX overcommits VM too.
--
Chris

[Database Administrator]

Chris Ridd wrote:
> I've a feeling HP-UX overcommits VM too.

No, not by default.

[Database Administrator]

Chris Ridd writes:
> On 2008-11-05 00:58:48 +0000, Nate Eldredge said:

[...]

>> Really, about the only thing you can reliably do is to try the
>> allocation and see if it succeeds.

[...]

> That is usually the right approach, but AIUI it falls apart slightly
> on systems which overcommit VM, like Linux. That is, they return
> memory to the process even if there isn't enough VM. When you try to
> use the returned memory... either you get lucky and the system's got
> enough VM again or the OS has to kill some other poor process to make
> some VM for you.

The default is to kill the process which tried the memory access that
could not succeed because there wasn't enough virtual memory
available. But any other behaviour would not 'magically' cause this
set of processes whose attempts to use the ressources of the system in
particular way at that time to be more successful.

[Database Administrator]

guideveloper@gmail.com writes:

[...]

> I understand that the address space is 4 GB which could be used by
> kernel space or maybe completely used by user space, however i would
> like to know for solaris 2.4 ownwards, how can i programatically get
> the virtual memory available to the process so that i can show a
> warning to the user when further allocation is not possible.

The answer is 'you cannot'.