Tutorial: Debugging OLE Applications
  
PSS ID Number: Q122680
Article last modified on 04-20-1996
 
2.00
 
WINDOWS NT
 

---------------------------------------------------------------------
The information in this article applies to:
 
 - The Visual Workbench Integrated Debugger, included with:
   Microsoft Visual C++, 32-bit Edition, version 2.0 on the
   following platform: x86
---------------------------------------------------------------------
 
SUMMARY
=======
 
This article provides a tutorial to assist you in learning how to use the
Visual Workbench Integrated Debugger.
 
The Visual C++ integrated debugger supports simultaneous debugging of OLE
client and server applications. You can seamlessly step across and into OLE
clients and servers, with the ability to step across OLE Remote Procedure
Calls. A second instance of the debugger is automatically spawned the first
time an OLE client calls into an OLE server.
 
When building OLE servers, you may want to debug them in the context of
being activated by an OLE container, thus debugging both the container and
the server at the same time. This tutorial provides an example of how to
debug an OLE server when the main debuggee is an OLE container. It shows
how a second instance of the debugger is automatically spawned when an OLE
client calls into an OLE server. The tutorial is designed to lead you step
by step through the code that creates the OLE server object and establishes
the connection between the OLE container and the OLE server. This is useful
in tracking down problems that occur when the OLE server does not get
created or initialized correctly.
 
All regular debugging facilities are available as you debug your OLE
application.
 
MORE INFORMATION
================
 
Preparing to Debug
------------------
 
To prepare for OLE Client/Server debugging:
 
1. Open the project for the OLE application and build a version with
   symbolic debugging information.
 
2. Choose Options from the Tools menu.
 
3. Select the Debug tab.
 
4. Ensure that the OLE RPC Debugging check box and the Just-In-Time (JIT)
   Debugging boxes are checked. (You must have Windows NT administrator
   privileges to enable OLE RPC Debugging.)
 
5. Choose OK. The information is now stored with your project.
 
6. Set breakpoints at the points in the source files for your OLE
   application where you want to determine the state of the application.
 
7. From the Debug menu, choose Go or press the F5 key to start the
   debugger.
 
Viewing Derived-Most Types
--------------------------
 
The QuickWatch dialog box provides support for the automatic downcast of
pointers in OLE and MFC debugging. The debugger automatically detects when
a pointer is pointing to a subclass of the type it is required to point to.
When the pointer is expanded, Visual C++ will add an extra member, which
looks like another base class and indicates the derived-most type. For
example, if you are displaying a pointer to a CObject and the pointer
really points to a CComboBox, the QuickWatch expression evaluator will
recognize this and introduce a pseudo CComboBox member so you can access
the CComboBox members.
 
The rest of the tutorial takes you through a debug session, using MFC
sample code.
 
Creating the Object
-------------------
 
1. Build debug versions of the Microsoft Foundation Classes (MFC)
   samples in the development environment:
 
    - MSVC20\SAMPLES\MFC\CONTAIN\STEP2
 
    - MSVC20\SAMPLES\MFC\SCRIBBLE\STEP7
 
2. Run the SCRIBBLE.EXE file, built in step 1, to update the registry to
   point to this executable file.
 
3. Load the CONTAIN\STEP2 project into the Visual Workbench.
 
4. Choose Options from the Tools menu, select the Debug tab, and make sure
   OLE RPC Debugging is enabled, as described in step 4 of the "Preparing
   to Debug" section in this article.
 
5. Open the CONTRVW.CPP file in CONTAIN\STEP2, and set a breakpoint on line
   139, which contains a call to CreateItem.
 
6. Press the F5 key to run CONTAIN.EXE. In the CONTAIN main menu, choose
   Insert New Object from the Edit menu.
 
7. From the resulting Object Type list, choose Scrib Document, and then
   choose the OK button. At this point, the debugger should stop at the
   breakpoint you set in step 5. This is the call to the Insert Object
   dialog's CreateItem function. The purpose of CreateItem is to create and
   initialize an object of the type you selected from the dialog box. The
   CreateItem function is passed a CCntrItem object and it uses the object
   to handle this process.
 
   For a brief overview of what CCntrItem does, see its class definition in
   CNTRITEM.H. Then look at the definition of COleClientItem (which
   CCntrItem is derived from) on line 399 in AFXOLE.H.
 
8. Press the F8 key to step into the call to CreateItem. You are in the MFC
   source file OLEDLGS1.CPP. Press the F10 key to step over instructions
   until you get to the call to pNewItem->CreateItem on line 104.
 
9. Press the F8 key to step into the call to pNewItem->CreateItem. Then
   press the F10 key to step over instructions until you get to line 594,
   which contains a call to OleCreate. While stepping through the code,
   read the comments and observe that storage is allocated for the object
   and its rendering format is established.
 
10. Step into the call to OleCreate. At this point, execution proceeds
    through the RPC mechanism to the server code itself. Therefore, as the
    server code begins to execute, a new instance of the debugger is
    created in which to debug the server. A pseudo project for SCRIBBLE.EXE
    is loaded (as in JIT debugging), and the instruction pointer is set at
    the call to COleServerDoc::XPersistStorage::InitNew in OLESRV1.CPP. If
    you installed the .DBG files (see the "NT System Symbols Setup" icon in
    your Visual C++ program group), your callstack will include fully
    decorated names.
 
11. Step over instructions in Scribble's InitNew until you reach the call
    to pThis->OnNewEmbedding on line 1707. Then step into OnNewEmbedding.
 
12. Step over lines until you reach the call to OnNewDocument on line 856.
    Then step into the call to COleServerDoc::OnNewDocument. You are inside
    COleLinkingDoc::OnNewDocument. (COleServerDoc is derived from
    COleLinkingDoc.) Notice the code in this small function. It creates a
    new document object and attaches it to the server (Scribble).
 
13. Press SHIFT+F7 twice to step out twice to get back into
    COleServerDoc::XPersistStorage::InitNew, where you first came into
    Scribble.
 
14. Step out one more time. This will cause the container to return from
    its call to OleCreate, the function that took you into Scribble in the
    first place. At this time, the instance of the debugger that has
    Contain loaded gets the focus and you are back in OLECLI1.CPP
    immediately following the call to OleCreate. The embedded Scribble
    object has now been created, but it is not yet fully initialized.
 
15. Step into the next line, the call to FinishCreate. Step through the
    FinishCreate code to see how OLE finalizes the connection between the
    container and the server, then step out of FinishCreate.
 
Now a Scribble object has been created and initialized in memory, but it is
not yet editable in the container; Scribble hasn't been activated. In fact,
Contain still has only an IUnknown interface to Scribble. You can see this
by expanding the lpClientSite variable in the Locals Window.
 
Activating the Object
---------------------
 
1. Step out two more times to get back to Contain's OnInsertObject function
   in CONTRVW.CPP.
 
2. In Contain's OnInsertObject function, step over five times to get to the
   call to DoVerb on line 149. This function activates Scribble.
 
3. Step into the call to DoVerb. Then step over a few lines until you come
   to the call to Activate on line 71.
 
4. Step into Activate. Now you are in OLECLI3.CPP. Before going on, scan
   through the code for Activate. Notice that a rectangle is first created
   for the embedded Scribble item to live in. Then GetClientSite is called
   to establish an interface back to the container for the Scribble server.
   Then the server's DoVerb function is called to pass both of these to the
   server.
 
5. Step to line 75. Then step into the call to DoVerb. At this point,
   execution proceeds once again through the RPC mechanism to the server
   code itself. As you would expect, the instruction pointer is pointing to
   the first instruction in the server's COleServerDoc:: XOleObject::DoVerb
   function.
 
6. Step to line 2064, the call to OnDoVerb. Then step into OnDoVerb.
   OnDoVerb consists of a switch statement that executes the command (verb)
   passed to it. In this case, the command is OLEIVERB_SHOW. Step over
   instructions until you get to OnShow.
 
7. Step into OnShow. Then step three lines to ActivateInPlace, and step
   into ActivateInPlace.
 
   The ActivateInPlace function does many things, and is worth examining
   in detail. While it is beyond the scope of this tutorial to go into all
   the details, it is worthwhile to step through the code and observe the
   comments. At this point, step over each instruction until you get to
   line 1098 in OLESRV1.CPP -- the call to OnFrameWindowActivate. Among
   other things, you will see the following tasks accomplished:
 
    - Get the document type used in SetActiveObject calls.
 
    - Get the in-place client-site interface.
 
    - See if the container wants to go in-place right now.
 
    - Get the parent window to create the COleIPFrameWnd.
 
    - Create the inplace frame window.
 
    - Send an activate notification to the container.
 
    - Get the frame and doc window interfaces as well as other information.
 
    - Set up the shared menu.
 
    - Allow the server to install frame controls in the container.
 
    - Update the zoom factor information before creating control bars.
 
    - Resize the window to match the object.
 
    - Set the active object.
 
    - Add the frame and the document-level frame controls.
 
    - Show any hidden modeless dialogs.
 
    - Attempt toolbar negotiation.
 
    - Install the menu and a hook that forwards messages from the menu to
      the inplace frame window.
 
    - Make sure the object is scrolled into view.
 
    - Show the inplace frame window and set the focus.
 
   As you can see, ActivateInPlace does a lot of work. It is also very
   RPC-intensive.
 
8. Step into the first line of OnFrameWindowActivate. Observe that this
   function sends the final notifications via the container to activate
   the server.
 
9. Press the F5 key or choose the GO button to finish executing
   OnFrameWindowActivate. Under normal circumstances, Scribble would come
   up already activated in place within Contain. However, in this case, you
   have already executed past the code that set the focus to the activated
   server. Rremember the call to pFrameWnd->Set Focus on line 1094 in the
   ActivateInPlace function in OLESRV1.CPP? By continuing to step through
   the code you have reset the focus back to the debugger. Therefore, you
   must now switch tasks manually to Contain.
 
10. Switch tasks to Contain. You will see Scribble's menu and toolbar
    within Contain, and you will be able to draw in the embedded item's
    rectangle.
 
Finishing the Debug Session
---------------------------
 
When you finish with Scribble, close the document window. Contain's menu
and toolbar reappear, and the Scribble debugging session ends within the
second instance of the debugger.
 
Although Scribble has terminated, the second instance of the debugger is
still running. To avoid complications, terminate the second instance of the
debugger. You need to do this because whenever you call into a server to
embed a new item or activate an existing one, the debugger will start
another instance to debug this server, even if it is the same server that
is already attached to another item in the document. Therefore, it is
possible (but not desirable) to have multiple instances of the debugger
debugging multiple instances of the same server all connected to the same
container.
 
Once the second instance of the debugger has been terminated, it is safe
to embed another Scrib Document object in Contain. It is not necessary to
terminate the container before doing so.
 
Trying New Things in Future Debug Sessions
------------------------------------------
 
If you have more time in a subsequent debugging session, try stepping
into (rather than over) some of the function calls. The flow of control
goes back and forth between the container and the server many times, and
the debugger will track this flow accurately, bringing you into container
code, then server code, and so forth. Functions of interest include:
 
   CanInPlaceActivate
   OnInPlaceActivate
   GetWindow
   OnUIActivate
   GetWindowContext
   SetActiveObject
   SetMenu
   ShowObject
 
The server maintains several pointers (interfaces) into the container's
code through which it calls the container's member functions. These
pointers include m_lpClientSite, lpInPlaceSite, and pFrameWnd.
 
Additional reference words: kbinf 2.00
KBCategory: kbtool kbole kbinterop
KBSubcategory: WBDebug VCx86
=============================================================================
Copyright Microsoft Corporation 1996.
