PRB: Cannot Load <exe> Because NTVDM Is Already Running

• Microsoft Win32 Software Development Kit (SDK), on platform(s):
  • Microsoft Windows NT, versions 3.1, 3.5, 3.51

SYMPTOMS

WinDbg can debug 16-bit Windows-based applications running on Windows NT, under the Win16 VDM (virtual MS-DOS machine), NTVDM. By default, each 16- bit Windows-based application runs as a thread in NTVDM. On Windows NT 3.5, each application can be run in a separate address space. Note that WinDBG does not work on 16-bit applications under Windows NT version 4.0 and later.

If NTVDM is running when the debugger tries to start the application, you get the following error message:

Cannot load <exe> because NTVDM is already running

RESOLUTION

Under Windows NT 3.5, to work around this problem, go to the WinDbg Options menu, choose Debug, and check Separate WOW VDM, to allow the debuggee to be run in a different address space.

Alternatively, you could terminate NTVDM. On Windows NT 3.1, your only choice is to terminate NTVDM, because separate address spaces for 16-bit Windows-based application are not supported.

To terminate NTVDM, run PView, select NTVDM, and choose "Kill Process." Note that there may be two NTVDM processes. The one that you want to terminate has one thread for each Windows-based application (plus a few more).

The Windows NT WinLogon is set up to automatically start WoWExec, which automatically starts the Win16 VDM. This behavior can be changed by removing WoWExec from:

   HKEY_LOCAL_MACHINE\ 
             Software\ 
            Microsoft\ 
           Windows NT\ 
       CurrentVersion\ 
             Winlogon\ 
                Shell 

Additional query words:

Keywords          : kberrmsg kbtool TlsWindbg 
Version           : WINDOWS:
Platform          : WINDOWS 
Issue type        : kbprb