Tuesday, November 23, 2010

Integrating XNA 4.0 and WPF

Download the code for this example here.


Integration of WPF and XNA is a recurring topic of interest for developers working on Windows applications that support 3D visualization.  Microsoft has repeatedly made it clear that the two technologies playing together is not [yet] supported.  Additionally, XNA remains 32-bit only, which limits the number of commercial applications that can use it as a managed graphics API.  Before you consider any solution which hosts XNA in WPF for production code, I am obligated to point you to the excellent managed wrapper for D3D: SlimDX.  It's open source, and of course, free.


Microsoft supports displaying D3D images in WPF, via the D3DImage class.  They also support the use of XNA with WinForms, which XNA guru Shawn Hargreaves has covered in a blog.

That being said, if you have a use case where you need a UI around your game and only WPF will do, there are ways of violently hacking your way to success.  Many people are approaching this problem in different ways, such as the one mentioned in Nick Gravelyn's blog.  My example attempts to wrap an existing game, making as few changes to the game code as possible, without the addition of a separate thread for the game loop.  I arrived at this solution using the assistance of several blogs and forum postings.

I began with a simple "game" that uses a ContentManager to import a Model.  The game has a public Vector3 property that is used to scale the model.  I added a DrawableGameComponent to track the framerate.

TeapotGame running in an XNA window.

Creating a GraphicsDevice and initializing the game

In the WPF application, I created a user control, called GamePanel.  The GamePanel XAML contains little more than an Image that uses a D3DImage as its source.  The control registers a dependency property that can be used to bind a Game in XAML.  When the GamePanel is loaded, the bound Game is passed to my GameReflector class, where I encapsulate some of the nastier reflection details.

I used Reflector to see which methods were being called in Game.Run.  GameReflector.CreateGame is my attempt at hacking the equivalent functionality, without creating a separate window for display.  First, the GraphicsDeviceManager is retrieved from the Game object using reflection.  Calling the non-public method ChangeDevice on the GraphicsDeviceManager raises the PreparingDeviceSettings event, which allows me to specify the HWND from my WPF image as a parameter used in the creation of the game's GraphicsDevice.  Next, I call Initialize on the game, which also calls LoadContent.

public static void CreateGame(Game game, Visual visual)
    var deviceManager = GetGraphicsDeviceManager(game);

    deviceManager.PreparingDeviceSettings += (sender, e) =>
        //If using a RenderTarget2D for drawing, the GraphicsDevice buffer can be whatever size it needs to be
        e.GraphicsDeviceInformation.PresentationParameters.BackBufferWidth = 4096;
        e.GraphicsDeviceInformation.PresentationParameters.BackBufferHeight = 4096;
        //If using the GraphicsDevice for drawing, create with the standard XNA back buffer dimensions
        e.GraphicsDeviceInformation.PresentationParameters.BackBufferWidth = 800;
        e.GraphicsDeviceInformation.PresentationParameters.BackBufferHeight = 480;
        e.GraphicsDeviceInformation.PresentationParameters.RenderTargetUsage = RenderTargetUsage.PreserveContents;
        e.GraphicsDeviceInformation.PresentationParameters.IsFullScreen = false;
        e.GraphicsDeviceInformation.PresentationParameters.DeviceWindowHandle = (PresentationSource.FromVisual(visual) as HwndSource).Handle;

    //A non-public method which creates the GraphicsDevice and performs other initializations
    var changeDevice = deviceManager.GetType().GetMethod("ChangeDevice", BindingFlags.NonPublic | BindingFlags.Instance);
    changeDevice.Invoke(deviceManager, new object[] { true });

    var initialize = game.GetType().GetMethod("Initialize", BindingFlags.NonPublic | BindingFlags.Instance);
    initialize.Invoke(game, new object[] { });

private static GraphicsDeviceManager GetGraphicsDeviceManager(Game game)
    foreach (var field in game.GetType().GetFields(BindingFlags.Instance | BindingFlags.NonPublic | BindingFlags.Public))
        if (field.FieldType == typeof(GraphicsDeviceManager))
            return (GraphicsDeviceManager)field.GetValue(game);

    throw new InvalidOperationException("Game contains no GraphicsDeviceManager");

Drawing to an IDirect3DSurface9 

I wanted the game window to be resizable, as with a normal WPF control.  Therefore, I created two versions of the code using a preprocessor directive, called RESIZABLE.  Defining this directive uses a RenderTarget2D for drawing the game.  By drawing to a RenderTarget2D, I can change the size of the surface I draw to, without having to recreate the GraphicsDevice.  I listen to the resize event for the WPF image and recreate the RenderTarget2D each time the size changes.

Changing the game's render target may not always be desirable.  Removing the preprocessor definition RESIZABLE will use the GraphicsDevice's back buffer for drawing, instead.  Getting the surface from the GraphicsDevice is all-or-nothing.  This means that the resolution of the images drawn with XNA will be fixed once the GraphicsDevice is created, which can lead to stretching and aliasing if you resize the window.

To show the game in the WPF window I used a D3DImage, which takes an IDirect3DSurface9 as a back buffer.  This surface can be obtained in one of two ways, depending on whether my GamePanel is resizable.

Both the GraphicsDevice and RenderTarget2D contain pointers to their underlying D3D objects.  These are COM objects that inherit from IUnknown.  GraphicsDevice contains a pointer to an IDirect3DDevice9, and RenderTarget2D contains a pointer to an IDirect3DTexture9.   The only methods I'm interested in from both of these interfaces are the ones that obtain the IDirect3DSurface9.  In the case of the device, this is the back buffer; for the texture, it's the first mip level.  I import the COM objects using the GUIDs in d3d9.h, then define the interface methods I'm interested in and use placeholders for the vtable with the rest of the methods.

public static IntPtr GetRenderTargetSurface(RenderTarget2D renderTarget)
    IntPtr surfacePointer;
    var texture = GetIUnknownObject<IDirect3DTexture9>(renderTarget);
    Marshal.ThrowExceptionForHR(texture.GetSurfaceLevel(0, out surfacePointer));
    return surfacePointer;

public static IntPtr GetGraphicsDeviceSurface(GraphicsDevice graphicsDevice)
    IntPtr surfacePointer;
    var device = GetIUnknownObject<IDirect3DDevice9>(graphicsDevice);
    Marshal.ThrowExceptionForHR(device.GetBackBuffer(0, 0, 0, out surfacePointer));
    return surfacePointer;

private static T GetIUnknownObject<T>(object container)
        //Get the COM object pointer from the D3D object and marshal it as one of the interfaces defined below
        var deviceField = container.GetType().GetField("pComPtr", BindingFlags.NonPublic | BindingFlags.Instance);
        var devicePointer = new IntPtr(Pointer.Unbox(deviceField.GetValue(container)));
        return (T)Marshal.GetObjectForIUnknown(devicePointer);

[ComImport, Guid("85C31227-3DE5-4f00-9B3A-F11AC38C18B5"), InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
private interface IDirect3DTexture9
    void GetDevice();
    void SetPrivateData();
    void GetPrivateData();
    void FreePrivateData();
    void SetPriority();
    void GetPriority();
    void PreLoad();
    void GetType();
    void SetLOD();
    void GetLOD();
    void GetLevelCount();
    void SetAutoGenFilterType();
    void GetAutoGenFilterType();
    void GenerateMipSubLevels();
    void GetLevelDesc();
    int GetSurfaceLevel(uint level, out IntPtr surfacePointer);

[ComImport, Guid("D0223B96-BF7A-43fd-92BD-A43B0D82B9EB"), InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
private interface IDirect3DDevice9
    void TestCooperativeLevel();
    void GetAvailableTextureMem();
    void EvictManagedResources();
    void GetDirect3D();
    void GetDeviceCaps();
    void GetDisplayMode();
    void GetCreationParameters();
    void SetCursorProperties();
    void SetCursorPosition();
    void ShowCursor();
    void CreateAdditionalSwapChain();
    void GetSwapChain();
    void GetNumberOfSwapChains();
    void Reset();
    void Present();
    int GetBackBuffer(uint swapChain, uint backBuffer, int type, out IntPtr backBufferPointer);

Once I have the drawing surface, I lock the D3DImage and set the surface as the back buffer.  I register for the CompositionTarget.Rendering event when the GamePanel loads, and call Tick on my Game object when the event is raised.  Tick is a public method which handles Update and Draw in the game (and game components).

Putting it together with MVVM

I created a GameViewModel that has TeapotGame and Scale properties.

class GameViewModel
    public GameViewModel()
        Game = new TeapotGame();

    public TeapotGame Game { get; private set; }

    public Vector3 Scale
            return ((ITeapotService)Game.Services.GetService(typeof(ITeapotService))).Scale;
            ((ITeapotService)Game.Services.GetService(typeof(ITeapotService))).Scale = value;

The GameView binds the TeapotGame to a GamePanel and binds the Scale to a text box.
    <StackPanel DockPanel.Dock="Left">
        <TextBox Text="{Binding Scale}" />
        <TextBox />
    <local:GamePanel game="{Binding Game}">
                <MenuItem>Click Me!
                <MenuItem>Don't Click Me!

You can change the scale using the text box (just click in the second text box below it to validate).  Right clicking on the GamePanel shows the ContextMenu I defined, showing that the control successfully circumvents any airspace issues.

TeapotGame running in a WPF window

As a final note, if you only want to use XNA as a managed front-end to D3D, you don't have to use a Game object.  Just create a GraphicsDevice and handle drawing yourself in the Rendering event callback.

Saturday, November 6, 2010

Stick a fork in me...

Yesterday, at 2:00 PM, in College Station, Texas, after four and a half years of work, I successfully defended my thesis, Perspective-Driven Radiosity on Graphics Hardware.

Four and a half years is a long time to work on anything, especially when it's something that should have been completed in two.  So, why the delay?  The superficial reason is that I've been working while I finished my MS, so I haven't been able to devote my undivided attention to the thesis.  But, the underlying cause is that I just made some bad decisions.

Graduate school got off to a rough start in Fall 2004.  My dad was diagnosed with cancer the week before I started and died six weeks later.  It took me a while to recover from that experience, and my education suffered because of it.  I wound up with a 3.5 for my overall GPA, but that was with the help and guidance of some great people: Dr. Glen Williams, my committee chair and former employer; Dr. John Keyser, a professor for many of my best classes, and a member of my committee; Dr. Jianer Chen, who was my professor and graduate advisor; my mom and the memories of my dad; and, most especially, my wife, Marie.

During my two years of graduate classes, from 2004 to 2006, I was a research graduate assistant on the Texas A&M University Autonomous Ground Vehicle (AGV) project.  I worked on the collision avoidance system which used a SICK lidar and directed a drive-by-wire truck.  It was fun, and I learned a lot doing it.  But, it was not something I wanted to work on for the rest of my life.  My interests were, and remain, computer graphics.  I could have completed a thesis related to the AGV during my two years of classes.  Instead, I chose a thesis topic that truly interested me, and to which I would enjoy devoting so much of my time.  Since I couldn't remain in school forever, and funding for the AGV was gone, I decided to finish my thesis while I worked as a professional.

After working for many nights and weekends on my research, from 2006 to 2010, I could finally say, "It's done."  I took two weeks off from work this past summer, went to College Station, and lived in the library for 16 days straight.  Which brings me to today.

I have regrets regarding my graduate school experiences, but one thing I didn't want to regret was never finishing my degree.  I can't do anything to change the past, but I can drive my future the direction I want it to go.  I should count myself lucky that I have chosen a career path that not only interests me, but, in my opinion, is exactly what I'm supposed to do while I'm here.

So, what's next?

There are many things on which I want to work.  Computer graphics, gaming, visual effects: these are not jobs, these are hobbies for me.  Coincidently, I can also get paid to do them!  I have lots of projects outside of work, it's just a matter of completing them, one at a time.  As I go along, I want to share what I learn, and document what I've accomplished.  I did many projects in graduate school, all of which are lost to the sands of time (and a fried hard disk).  I don't want the same thing to happen again.

I'm also preparing to look for my next employer.  I enjoy my job, but the challenges are no longer the kind that interest me.  Lately it seems that I have to work on projects at home to keep my brain occupied.  Ideally, I'd like an entry-level position in the graphics-related entertainment industry: gaming or animation.  I'm ready to devote my time to a product that excites me and fuels my passions, as well as those of the customer.  How fulfilling it must be to complete a project and enjoy it just as much as your target audience!

Whatever I find next, yesterday was a good day, and I'm glad it finally arrived...delayed or not.