Tuesday, November 23, 2010

Integrating XNA 4.0 and WPF

Download the code for this example here.

Disclaimer

Integration of WPF and XNA is a recurring topic of interest for developers working on Windows applications that support 3D visualization.  Microsoft has repeatedly made it clear that the two technologies playing together is not [yet] supported.  Additionally, XNA remains 32-bit only, which limits the number of commercial applications that can use it as a managed graphics API.  Before you consider any solution which hosts XNA in WPF for production code, I am obligated to point you to the excellent managed wrapper for D3D: SlimDX.  It's open source, and of course, free.

Background

Microsoft supports displaying D3D images in WPF, via the D3DImage class.  They also support the use of XNA with WinForms, which XNA guru Shawn Hargreaves has covered in a blog.

That being said, if you have a use case where you need a UI around your game and only WPF will do, there are ways of violently hacking your way to success.  Many people are approaching this problem in different ways, such as the one mentioned in Nick Gravelyn's blog.  My example attempts to wrap an existing game, making as few changes to the game code as possible, without the addition of a separate thread for the game loop.  I arrived at this solution using the assistance of several blogs and forum postings.

I began with a simple "game" that uses a ContentManager to import a Model.  The game has a public Vector3 property that is used to scale the model.  I added a DrawableGameComponent to track the framerate.

TeapotGame running in an XNA window.

Creating a GraphicsDevice and initializing the game

In the WPF application, I created a user control, called GamePanel.  The GamePanel XAML contains little more than an Image that uses a D3DImage as its source.  The control registers a dependency property that can be used to bind a Game in XAML.  When the GamePanel is loaded, the bound Game is passed to my GameReflector class, where I encapsulate some of the nastier reflection details.

I used Reflector to see which methods were being called in Game.Run.  GameReflector.CreateGame is my attempt at hacking the equivalent functionality, without creating a separate window for display.  First, the GraphicsDeviceManager is retrieved from the Game object using reflection.  Calling the non-public method ChangeDevice on the GraphicsDeviceManager raises the PreparingDeviceSettings event, which allows me to specify the HWND from my WPF image as a parameter used in the creation of the game's GraphicsDevice.  Next, I call Initialize on the game, which also calls LoadContent.

public static void CreateGame(Game game, Visual visual)
{
    var deviceManager = GetGraphicsDeviceManager(game);

    deviceManager.PreparingDeviceSettings += (sender, e) =>
    {
#if RESIZABLE
        //If using a RenderTarget2D for drawing, the GraphicsDevice buffer can be whatever size it needs to be
        e.GraphicsDeviceInformation.PresentationParameters.BackBufferWidth = 4096;
        e.GraphicsDeviceInformation.PresentationParameters.BackBufferHeight = 4096;
#else
        //If using the GraphicsDevice for drawing, create with the standard XNA back buffer dimensions
        e.GraphicsDeviceInformation.PresentationParameters.BackBufferWidth = 800;
        e.GraphicsDeviceInformation.PresentationParameters.BackBufferHeight = 480;
#endif
        e.GraphicsDeviceInformation.PresentationParameters.RenderTargetUsage = RenderTargetUsage.PreserveContents;
        e.GraphicsDeviceInformation.PresentationParameters.IsFullScreen = false;
        e.GraphicsDeviceInformation.PresentationParameters.DeviceWindowHandle = (PresentationSource.FromVisual(visual) as HwndSource).Handle;
    };

    //A non-public method which creates the GraphicsDevice and performs other initializations
    var changeDevice = deviceManager.GetType().GetMethod("ChangeDevice", BindingFlags.NonPublic | BindingFlags.Instance);
    changeDevice.Invoke(deviceManager, new object[] { true });

    var initialize = game.GetType().GetMethod("Initialize", BindingFlags.NonPublic | BindingFlags.Instance);
    initialize.Invoke(game, new object[] { });
}

private static GraphicsDeviceManager GetGraphicsDeviceManager(Game game)
{
    foreach (var field in game.GetType().GetFields(BindingFlags.Instance | BindingFlags.NonPublic | BindingFlags.Public))
    {
        if (field.FieldType == typeof(GraphicsDeviceManager))
        {
            return (GraphicsDeviceManager)field.GetValue(game);
        }
    }

    throw new InvalidOperationException("Game contains no GraphicsDeviceManager");
}

Drawing to an IDirect3DSurface9 

I wanted the game window to be resizable, as with a normal WPF control.  Therefore, I created two versions of the code using a preprocessor directive, called RESIZABLE.  Defining this directive uses a RenderTarget2D for drawing the game.  By drawing to a RenderTarget2D, I can change the size of the surface I draw to, without having to recreate the GraphicsDevice.  I listen to the resize event for the WPF image and recreate the RenderTarget2D each time the size changes.

Changing the game's render target may not always be desirable.  Removing the preprocessor definition RESIZABLE will use the GraphicsDevice's back buffer for drawing, instead.  Getting the surface from the GraphicsDevice is all-or-nothing.  This means that the resolution of the images drawn with XNA will be fixed once the GraphicsDevice is created, which can lead to stretching and aliasing if you resize the window.

To show the game in the WPF window I used a D3DImage, which takes an IDirect3DSurface9 as a back buffer.  This surface can be obtained in one of two ways, depending on whether my GamePanel is resizable.

Both the GraphicsDevice and RenderTarget2D contain pointers to their underlying D3D objects.  These are COM objects that inherit from IUnknown.  GraphicsDevice contains a pointer to an IDirect3DDevice9, and RenderTarget2D contains a pointer to an IDirect3DTexture9.   The only methods I'm interested in from both of these interfaces are the ones that obtain the IDirect3DSurface9.  In the case of the device, this is the back buffer; for the texture, it's the first mip level.  I import the COM objects using the GUIDs in d3d9.h, then define the interface methods I'm interested in and use placeholders for the vtable with the rest of the methods.

public static IntPtr GetRenderTargetSurface(RenderTarget2D renderTarget)
{
    IntPtr surfacePointer;
    var texture = GetIUnknownObject<IDirect3DTexture9>(renderTarget);
    Marshal.ThrowExceptionForHR(texture.GetSurfaceLevel(0, out surfacePointer));
    Marshal.ReleaseComObject(texture);
    return surfacePointer;
}

public static IntPtr GetGraphicsDeviceSurface(GraphicsDevice graphicsDevice)
{
    IntPtr surfacePointer;
    var device = GetIUnknownObject<IDirect3DDevice9>(graphicsDevice);
    Marshal.ThrowExceptionForHR(device.GetBackBuffer(0, 0, 0, out surfacePointer));
    Marshal.ReleaseComObject(device);
    return surfacePointer;
}

private static T GetIUnknownObject<T>(object container)
{
    unsafe
    {
        //Get the COM object pointer from the D3D object and marshal it as one of the interfaces defined below
        var deviceField = container.GetType().GetField("pComPtr", BindingFlags.NonPublic | BindingFlags.Instance);
        var devicePointer = new IntPtr(Pointer.Unbox(deviceField.GetValue(container)));
        return (T)Marshal.GetObjectForIUnknown(devicePointer);
    }
}

[ComImport, Guid("85C31227-3DE5-4f00-9B3A-F11AC38C18B5"), InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
private interface IDirect3DTexture9
{
    void GetDevice();
    void SetPrivateData();
    void GetPrivateData();
    void FreePrivateData();
    void SetPriority();
    void GetPriority();
    void PreLoad();
    void GetType();
    void SetLOD();
    void GetLOD();
    void GetLevelCount();
    void SetAutoGenFilterType();
    void GetAutoGenFilterType();
    void GenerateMipSubLevels();
    void GetLevelDesc();
    int GetSurfaceLevel(uint level, out IntPtr surfacePointer);
}

[ComImport, Guid("D0223B96-BF7A-43fd-92BD-A43B0D82B9EB"), InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
private interface IDirect3DDevice9
{
    void TestCooperativeLevel();
    void GetAvailableTextureMem();
    void EvictManagedResources();
    void GetDirect3D();
    void GetDeviceCaps();
    void GetDisplayMode();
    void GetCreationParameters();
    void SetCursorProperties();
    void SetCursorPosition();
    void ShowCursor();
    void CreateAdditionalSwapChain();
    void GetSwapChain();
    void GetNumberOfSwapChains();
    void Reset();
    void Present();
    int GetBackBuffer(uint swapChain, uint backBuffer, int type, out IntPtr backBufferPointer);
}

Once I have the drawing surface, I lock the D3DImage and set the surface as the back buffer.  I register for the CompositionTarget.Rendering event when the GamePanel loads, and call Tick on my Game object when the event is raised.  Tick is a public method which handles Update and Draw in the game (and game components).

Putting it together with MVVM

I created a GameViewModel that has TeapotGame and Scale properties.

class GameViewModel
{
    public GameViewModel()
    {
        Game = new TeapotGame();
    }

    public TeapotGame Game { get; private set; }

    public Vector3 Scale
    {
        get
        {
            return ((ITeapotService)Game.Services.GetService(typeof(ITeapotService))).Scale;
        }
        set
        {
            ((ITeapotService)Game.Services.GetService(typeof(ITeapotService))).Scale = value;
        }
    }
}

The GameView binds the TeapotGame to a GamePanel and binds the Scale to a text box.
<DockPanel>
    <StackPanel DockPanel.Dock="Left">
        <TextBox Text="{Binding Scale}" />
        <TextBox />
    </Stackpanel>
    <local:GamePanel game="{Binding Game}">
        <local:Gamepanel.ContextMenu>
            <ContextMenu>
                <MenuItem>Click Me!
                <MenuItem>Don't Click Me!
            </ContextMenu>
        </local:GamePanel.ContextMenu>
    </local:GamePanel>
</DockPanel>

You can change the scale using the text box (just click in the second text box below it to validate).  Right clicking on the GamePanel shows the ContextMenu I defined, showing that the control successfully circumvents any airspace issues.

TeapotGame running in a WPF window

As a final note, if you only want to use XNA as a managed front-end to D3D, you don't have to use a Game object.  Just create a GraphicsDevice and handle drawing yourself in the Rendering event callback.

22 comments:

  1. Thanks, dude! Works like a charm, and no perf issues like the buffer-copy approach.

    ReplyDelete
  2. Integrated your XNA in WPF solution in my game editor and have the strange behavior.

    When the XNA render is created (only 2 triangles are rendered) the app is laggy and Perforator shows only about 20 fps.

    But if I minimize the app and get it up again the fps jumps to vsync 60 fps.

    ReplyDelete
  3. I haven't seen that issue, though I have experienced some performance issues that I suspect are due to fill rate of the D3DImage. Let me play with it and I'll see if I can reproduce the problem you're having.

    ReplyDelete
    Replies
    1. This is a problem i also encountered, after 2 days of work i found this one. The IsActive boolean of Game isn't set to true at initialisation.

      I've extended the reflector class with the following method.

      public static void ActivateGame(Game engine)
      {
      var activateHost = typeof(Game).GetMethod("HostActivated", BindingFlags.NonPublic | BindingFlags.Instance);
      activateHost.Invoke(engine, new Object[] { engine, new EventArgs() });
      }

      And i call it in the onrender method when the game is inactive.

      Delete
  4. More info on that:
    Using the XNA render in DocumentPane on AvalonDock.
    Tried profiling, but nothing comes in function calls. Seems smth with fill rate maybe.

    ReplyDelete
  5. Ok, found the following.
    The strange behavior of WPF tabs ui, which causes the user controls in it to receive Loaded event twice.
    Fixed that with private bool in GamePanel to prevent 2nd init of Game.
    The laggy fps jumped to 40.
    Again minimize/maximize and smooth 60 comes :)

    ReplyDelete
  6. Interesting, I haven't tried running it in AvalonDock. I found references in forum posts to the behavior you describe, as far as the Loaded event being raised more than once. It makes sense that if the game were being created more than once, it would be laggy while trying to update both instances.

    Perhaps minimizing and maximizing the window causes the GC to clean up dead game references that are still updating? That's a complete guess without seeing the code, of course. If you can upload to a public location, I'll be glad to take a look at it. Maybe two sets of eyes can figure it out?

    ReplyDelete
  7. Hey Justin,
    whenever I try to do this I end up with a transparent image and nothing else.

    Any ideas? Been trying to solve this for hours now.

    ReplyDelete
  8. Ah found the issue, it has todo with the inhertiance of the RibbonWindow. By some reason it does not render correctly in such a window.

    ReplyDelete
  9. Hey, thanks for this! I am having an issue though: the game is updating but not drawing. Any idea's?

    ReplyDelete
  10. AHA! I forgot to call the base.Update() in my override so the Game's private doneFirstUpdate member was never set to true (this is done in Game.Run originally, but that is not called by the GamePanel usercontrol)

    ReplyDelete
  11. Hey Simon, I'd be interested to see what you're trying to do using a RibbonWindow. Have you made any progress?

    ReplyDelete
  12. Thanks for the great example although i've been trying to put 2 GamePanels into a Windows Control but haven't got any success.

    I tried putting the GamePanels in Canvas' but then the canvas disappear, does anyone know the reason of this?

    ReplyDelete
  13. This is just what I've been looking for! Thanks so much!

    ReplyDelete
  14. Cool, glad it worked for you. If you discover any "gotchas" while working with it, please post your findings.

    ReplyDelete
  15. This comment has been removed by the author.

    ReplyDelete
  16. Hello Justin i would say your solution is great!!! i just use it for my xna game and its quite fine with it
    thank you very much for the great post have a nice day, Imal hasaranga

    ReplyDelete
  17. So glad to hear this solution is still working for people, glad to be of help.

    ReplyDelete
  18. Using the RESIZABLE flag, I needed to override Game.EndDraw() to block the GraphicsDevice class from trying to present the backbuffer.

    ReplyDelete
  19. Hey, is your code licensed under a particular license?

    ReplyDelete
  20. No, it's public domain as far as I'm concerned.

    ReplyDelete
  21. Thanks for the clarification. One more question...

    Have you noticed any issues with the GamePanel or GameReflector not releasing resources when you swap out games in the GamePanel? Even when I exit/dispose of a game after using it in the GamePanel and swapping into another game (or another instance of the same game), I notice that the memory doesn’t seem to be freed up. I’ve got a game with a lot of big assets and even though I’m calling Dispose, UnloadContent, etc. on the game, I’m still running out of memory after starting the third instance of the game.

    I have a version of the control in a solution with your teapot game that shows the memory add up if you would like a repro case (though it does take a lot more instances of the teapot game notice the issue in memory).

    Thanks!

    ReplyDelete