Memory Limits for Applications on Windows

When building and running an application under Windows*, you need to be aware of the limitations on the size of code and data your program can declare and use.  This article applies to Intel C++ and Intel Fortran compilers.
In this article, "32-bit" will refer to an application built with a compiler "for applications running on IA-32", even if it is run on a 64-bit version of Windows.  In the Visual Studio* environment, this is a "Win32" platform target.  "64-bit" refers to an application built with a compiler "for applications running on Intel 64" and run on an "x64" variant of Windows. In the Visual Studio environment, this is an "x64" platform target.  Note that 64-bit applications will not run on a 32-bit variant of Windows.
There are three kinds of memory limits:
  • Static code and data - this is all the compiled code plus all static data.  In C or C++, static data is generally created by variables and structs declared at "file scope", outside of a procedure.  In Fortran, COMMON and module variables are always static.  Variables declared in main programs and in procedures may or may not be static - by default, non-ALLOCATABLE/POINTER arrays are static and scalars are not.  Any variable marked as SAVE or data-initialized is static.
  • Dynamic data - this is memory that is allocated during program execution.  In or C or C++ this is usually done with malloc or new; in Fortran it is done with ALLOCATE or malloc.
  • Stack data - this is memory that is allocated as a procedure is entered and deallocated when the procedure exits. In C or C++, most routine local variables and variables declared inside blocks are allocated on the stack. In Fortran, automatic arrays (with dimensions based on a dummy argument, common or module variable) are stack-allocated.  Fortran code, in particular, may create temporary copies of arrays on the stack as needed to satisfy language semantics.  In Fortran, a routine declared RECURSIVE uses stack allocation for all local variables (other than POINTER and ALLOCATABLE), and enabling OpenMP also results in variables being allocated on the stack by default.
Given these definitions, the following lists the limits on 32-bit and 64-bit variants of Windows:
32-bit
  • Static data - 2GB
  • Dynamic data - 2GB
  • Stack data - 1GB (the stack size is set by the linker, the default is 1MB.  This can be increased using the Linker property System > Stack Reserve Size)
Note that on 32-bit Windows, the sum of all types of data must be 2GB or less.  The practical limit is about 1.75GB due to space used by Windows itself
64-bit
  • Static data - 2GB
  • Dynamic data - 8TB
  • Stack data - 1GB (the stack size is set by the linker, the default is 1MB.  This can be increased using the Linker property System > Stack Reserve Size)
Note that the limit on static and stack data is the same in both 32-bit and 64-bit variants.  This is due to the format of the Windows Portable Executable (PE) file type, which is used to describe EXEs and DLLs as laid out by the linker. It has 32-bit fields for image section offsets and lengths and was not extended for 64-bit variants of Windows. As on 32-bit Windows, static data and stack share the same first 2GB of address space.
If you exceed the limit on static code and data you may see the following error when the application is linked:
LNK1248: image size ('size') exceeds maximum allowable size (80000000)
Another possibility is that the application will appear to link successfully, but when run you will see:
is not a valid Win32 application
It is important to note that these are Windows operating system limits and are not constraints imposed by the compilers.
Resolving the Problem
If you are running a 32-bit application, you will need to switch to a 64-bit build and run on a 64-bit variant of Windows.  But even on a 64-bit variant, you must reduce the size of static data.  The best way to do that is to change to dynamic allocation for large arrays.  In C and C++ this is done with pointers and malloc.  In Fortran, replace large static arrays with ALLOCATABLE arrays and allocate them to the desired size at the start of the program. In many cases, you can replace COMMON declarations with variables declared in a module, and USE that module where you would have included the COMMON statements.  Note that you cannot use data-initialization for ALLOCATABLE arrays - such initialization must be in executable code.

Source

Comments

Popular Posts