How to solve the "The Win 16 Subsystem has insufficent resources to continue running" problem on Windows XP


Windows NT 4.0 and Windows 2000 have a limit of 16384 GDI objects per session. This means that the only way to get around this is to log on in a different new session using Terminal Services/Remote Desktop.

The console session (session 0) controls the actual physical display and input devices (keyboard mouse), and there is no way to get around this limit. Please note that this has nothing to do with the per-process limit for GDI objects, which can be tuned from the registry. As far as I know, there is no way to get around this per-session limit for GDI objects, other than recompiling Windows from source code ;-)

In Windows XP and later versions (2003, etc.) the limit was extended to 65536 GDI objects per session.

This poses a serious problem for the Win16 subsystem, because, for some reason, it always shifts the GDI handles to the right by two bits, when converting them from 32 to 16 bits. This means that the actual handle cannot be larger than 14 bits.

When the limit was updated for Windows XP, rather than eliminate this bizarre shift to the right, Microsoft programmers decided to simply crash the 16-bit application that happenes to be served a handle larger than 16384 by the GDI system.

We know this was a deliberate decision because of this error message, "The Win 16 Subsystem has insufficent resources to continue running", which always accompanies this type of crash and nothing else (it is specific to this type of crash), and is a new error message in Windows XP.

When does the problem occur:

The problem occurs when a 16-bit application is served a handle larger than 16384 by the GDI.

Because the GDI allocated handle numbers sequentially, and always prefers to reuse a freed handle rather than allocate a never-used higher handle, this can only happen if at some point during the current session the GDI handle count exceeded 16384. Please note that usually you are using session 0, the same that displays the logon screen when you start your computer, and so the only way to reset it is to reboot the computer (just like the second part of the error message says).

The error message is wrong when it says you can solve the problem by closing applications. Though closing applications could theoretically reduce the probability of hitting this problem, in practice it doesn't. The only thing that works is to reboot the computer (or, maybe, the patch below).

An illegal, dangerous, patch that could solve the problem for you

I made a version of wow32.dll that doesn't shift the handles to the right. You can download it from here. It is based on, and should work with, Windows XP Service Pack 2.

Installing it is quite tricky, because wow32.dll is both a KnownDll and WFP protected. I believe the simplest idea is to use some sort of parallel installation of Windows, and using it to replace both copies of wow32.dll in your main installation (the first in windows\system32, the second in windows\system32\dllcache) with the one provided here, making sure you rename it to wow32.dll first.

Having a parallel installation of Windows is always a good thing to have around; for instance, you can use it in a disaster situation to try and repair your main installation of Windows. Of course, having a second computer and moving harddrives from one another is almost as good as a parallel installation of Windows. Microsoft recommends using parallel installations in many circumstances, but (correctly) warns people to install on a separate partition, so that the parallel installation will have its own "Documents and Settings" and "Program Files" folders, safely isolated from those of the main installation.

There are more advanced ways to install this file, without using a parallel installation or even rebooting your computer; I used a combination of a simple, one byte write in live kernel memory, to get around KnownDlls protection, and wfpreplace to get aroung WFP protection.

The only danger with using this file, besides the fact that it is illegal, is that it could, sometimes, crash your 16-bit programs, though I have never seen this. But if you came here, your 16-bit programs are probably crashing all the time anyway, or refusing to run at all, so this is certainly an improvement.

There are some serious dangers with installing the file however; I think that Windows will refuse to boot if it can't find wow32.dll where it expects it. As I said, installing the file is tricky. Having a parallel installation will help in that you can boot from it and put any version of wow32.dll in its correct place inside the main installation.