Ramdisk and (Non)Paged Memory External links Options

[franks59]

I have a few questions about imdisk and paged memory.

If I understand correctly, when a virtual disk is created without a backing file, a Ram disk is created. I would like the memory allocated to this ramdisk not to be swapped to disk.

So question #1 is: Does executing the command

imdisk -a -s 10M -o hd -m r: -p "/fs:ntfs /q /y"

create a Ramdisk with paged or non-paged memory?

I see in awealloc.c, the function AWEAllocCreate appears to have the proper call - FsContext = ExAllocatePoolWithTag(NonPagedPool, sizeof(OBJECT_CONTEXT), POOL_TAG) - to do this but I cannot confirm that this is called for the above imdisk command.


Question #2 is: Is AweallocCreate used to create the ramdisk in the above imdisk command?

I saw in another thread ( http://www.boot-land.net/forums/index.php?...alloc&st=30 ) some advice about using awealloc as a backing "file" to create a ramdisk as follows:

imdisk -a -s 2G -f \\.\awealloc -m S: -p "/fs:ntfs /q /y"

Question #3: will using this 2nd imdisk command create a Ramdisk with non-paged memory?

Thanks,
Frank

[Olof Lagerkvist]

I have a few questions about imdisk and paged memory.

If I understand correctly, when a virtual disk is created without a backing file, a Ram disk is created. I would like the memory allocated to this ramdisk not to be swapped to disk.

So question #1 is: Does executing the command

imdisk -a -s 10M -o hd -m r: -p "/fs:ntfs /q /y"

create a Ramdisk with paged or non-paged memory?

There are no guarantees for that. The above command allocates memory using ZwAllocateVirtualMemory().

I see in awealloc.c, the function AWEAllocCreate appears to have the proper call - FsContext = ExAllocatePoolWithTag(NonPagedPool, sizeof(OBJECT_CONTEXT), POOL_TAG) - to do this but I cannot confirm that this is called for the above imdisk command.

The awealloc driver allocates physical memory through MmAllocatePagesForMdl() and associated functions. This memory should never be swapped out. The quoted line above just allocates a few bytes from kernel non-paged pool for holding a data structure with fields about current size, pointer to allocated memory etc.

Question #2 is: Is AweallocCreate used to create the ramdisk in the above imdisk command?

No. The imdisk.sys driver allocates memory by calling ZwAllocateVirtualMemory().

I saw in another thread ( http://www.boot-land.net/forums/index.php?...alloc&st=30 ) some advice about using awealloc as a backing "file" to create a ramdisk as follows:

imdisk -a -s 2G -f \\.\awealloc -m S: -p "/fs:ntfs /q /y"

Question #3: will using this 2nd imdisk command create a Ramdisk with non-paged memory?

Yes, because in that case the awealloc driver is used to allocate memory. Actually, for the imdisk.sys driver's point of view, it just opens an image file and change size of it to the requested disk size. The awealloc.sys driver emulates files by allocating memory when size change is requested and then redirects file I/O to this allocated memory. So, for imdisk.sys this is exactly like working with an image file on disk.

[franks59]

Yes, because in that case the awealloc driver is used to allocate memory. Actually, for the imdisk.sys driver's point of view, it just opens an image file and change size of it to the requested disk size. The awealloc.sys driver emulates files by allocating memory when size change is requested and then redirects file I/O to this allocated memory. So, for imdisk.sys this is exactly like working with an image file on disk.

Ok, I thought everything was good until that last part, "exactly like working with an image file on disk".

Maybe I should re-phrase the question: How can I use Imdisk to create a Ramdisk where all of the files are kept in memory and at no time is any of the contents (files) stored on the hard drive, even temporarily?

Thanks,
Frank

[Olof Lagerkvist]

Ok, I thought everything was good until that last part, "exactly like working with an image file on disk".

Okay, just to clarify, it is always kept in physical memory only, it can never be swapped to disk. I/O requests to the awealloc "file" is handled by the awealloc.sys driver and this driver redirects file I/O to the memory block it has allocated. The imdisk.sys driver does not "see" that this is a RAM disk, from the imdisk.sys point of view it is exactly like working with an image file on disk. Therefore all information that imdisk.sys returns about such a virtual disk indicates that it is backed by an image file while in reality it is in memory all the time because awealloc "fools" imdisk.sys to believe that it is a disk file.

Maybe I should re-phrase the question: How can I use Imdisk to create a Ramdisk where all of the files are kept in memory and at no time is any of the contents (files) stored on the hard drive, even temporarily?

Using the awealloc method.

[franks59]

Okay, just to clarify, it is always kept in physical memory only, it can never be swapped to disk. I/O requests to the awealloc "file" is handled by the awealloc.sys driver and this driver redirects file I/O to the memory block it has allocated. The imdisk.sys driver does not "see" that this is a RAM disk, from the imdisk.sys point of view it is exactly like working with an image file on disk. Therefore all information that imdisk.sys returns about such a virtual disk indicates that it is backed by an image file while in reality it is in memory all the time because awealloc "fools" imdisk.sys to believe that it is a disk file.



Using the awealloc method.

Ok, thanks.

Frank

[MedEvil]

I'm a bit confused now. What exactly is the difference between creating a RAMDISK with and without awealloc?

[Olof Lagerkvist]

I'm a bit confused now. What exactly is the difference between creating a RAMDISK with and without awealloc?

Without awealloc the imdisk.sys driver allocates memory itself using a function called ZwAllocateVirtualMemory. This function has been around since first version of Windows NT. With this allocation all memory for the RAM disk is accessible immediately in one piece in the virtual address space of the System process. This makes it easy for the driver code to access the RAM disk data like any other memory. The disadvantages are that it needs a free range of virtual address space the size of the RAM disk in the System process (you can see the memory usage for the System process grow in Task Manager when you create a RAM disk) and that this memory cannot be allocated from "lost memory" above 4 GB on 32-bit systems.

The awealloc.sys driver allocates memory using a function called MmAllocatePagesForMdl. This is a fairly new function. It has been around in the Windows kernel since Windows 2000 (I think) and it is there to provide a way to allocate a physical memory range. In later versions it can be used to allocate memory from "lost memory" above 4 GB on Windows versions where this is supported, like Windows Server 2003 Datacenter Edition etc. With this kind of allocation only a small portion of this physical memory is mapped into virtual address space of the System process when needed. This means that there is no need for a large free range in virtual address space of the System process, neither will all memory for the RAM disk be displayed in memory usage column for the System process in Task Manager. But this also means that memory remapping is needed when different parts of the RAM disk are accessed.

A problem with awealloc is that allocation above 4 GB could on some computers interfere with BIOS address translations for high range memory and a few other ugly things. Therefore I decided to put all this in a separate driver that could be developed and tested independently from the other pieces of ImDisk project.

That's the main differences.

[MedEvil]

Thanks you very much Olof, for this nice thorough explaination.

[franks59]

Ok, one more question on the awealloc method.

How can I use the awealloc method with a mountpoint other than a drive letter (NTFS mountpoint)?

I can create the mountpoint just fine, but it is unusable since it cannot be formatted, and the awealloc "image file" is not really an image of a disk.

I'm not really interested in a reparse point per se, I don't want a drive letter assigned at all, just the NTFS mountpoint.

If this cannot be done as is, could you please point me to a place in the code (cli\imdisk.c - ImdiskCliCreateDevice ?), and what needs to be done conceptually to make it work, i.e. create a DosDevice or Volume so that it can be formatted like "format \\?\Volume{f0a157ce-1cc8-4a05-ad8f-a0ad2bebade2}\" ?

Thanks,
Frank

[Olof Lagerkvist]

Ok, one more question on the awealloc method.

How can I use the awealloc method with a mountpoint other than a drive letter (NTFS mountpoint)?

I can create the mountpoint just fine, but it is unusable since it cannot be formatted, and the awealloc "image file" is not really an image of a disk.

Oh, I agree that is kind of awkward...

I'm not really interested in a reparse point per se, I don't want a drive letter assigned at all, just the NTFS mountpoint.

Volume mount points are implemented as NTFS reparse points. What do you mean that you want and what is it that you do not need?

The easiest workaround as it works right now is probably to create a temporary drive letter, format using this drive letter and then remove the temporary drive letter.

Alt 1: use the -m T: parameter and when the disk is created and formatted remove it with dosdev -d T: and create an NTFS reparse point that links to the disk instead using for example junc C:\ramdrive \Device\ImDisk0\ or something like that.

Alt 2: use syntax to create a subdirectory mountpoint, for example -m C:\ramdrive parameter but without formatting. Then create a temporary drive letter using dosdev -r T: \Device\ImDisk0 and use the format command on it and when formatted remove the drive letter again with dosdev -d T:.

(junc.exe and dosdev.exe are on my website.)

If this cannot be done as is, could you please point me to a place in the code (cli\imdisk.c - ImdiskCliCreateDevice ?), and what needs to be done conceptually to make it work, i.e. create a DosDevice or Volume so that it can be formatted like "format \\?\Volume{f0a157ce-1cc8-4a05-ad8f-a0ad2bebade2}\" ?

I agree that ideally this could be automated into imdisk.exe by adding this temporary drive letter logic around the call to format.com when a new disk is created without a drive letter and the -p switch is also specified to format the disk.

Technically speaking, this logic could be isolated to the ImDiskCliFormatDisk function and applied only when it is called with something else than a drive letter followed by a colon as the MountPoint parameter (but I suspect you would need an extra parameter DeviceNumber to this function so that it knows where to point the temporary drive letter it might have to create). You are more than welcome to write something like that!

...and thanks a lot for posting ideas like this!