Compile times: Visual Studio 2008 /MP and RAMDISK

I wanted to test if a RAMDISK could speed up my build process. And it did.
For everybody who is interested in this as well, I’m publishing my result.

Software Configuration

For the following Tests I used OGRE 1.7.0 RC1 and Visual Studio 2008 9.0.21022.8 RTM on Windows 7 Professional x64.

My tests were run on my default development PC with the following specs:

Hardware Configuration

CPU: Core 2 Quad Q9300 @ 2.50 GHz
RAM: 8 GB DDR2 @ 667 MHz
Hard-Disk: Western Digital 720RPM 750 GB

Test Results

As previously stated I’m testing a full rebuild of OgreMain in the debug configuration:

Following are the build times for a full clean build (debug). The times are always the average of two rebuilds. The build times were very stable between the two builds with very little variation between the builds.

Clean Build	no /MP	/MP	Speedup
no RAMDISK	8:20	4:49	3:31
RAMDISK	7:37	4:08	3:29
Speedup	0:43	0:41

Following table contains just the link times:

Link Times	no /MP	/MP
no RAMDISK	0:50	0:46
RAMDISK	0:36	0:34
Speedup	0:14	0:12

/MP seems to make linking faster, but this shouldn’t be the case because the Linker in vs 2008 is always single threaded.

Using a RAMDISK resulted in around 0:40 minutes less build time.

The /MP option was the biggest speedup with a ~ 3:30 minutes speedup.

Using /MP + RAMDISK for the intermediate files made the build 4:12 minutes faster. This is nearly half the time of the original build.

Conclusion

I will continue using a RAMDISK to store the intermediate build files. It comes of course with the cost of a full rebuild every time you have to reboot your PC. But for me, this is just the case once a day.
Or you copy the content of your RAMDISK to your hard drive on shutdown and restores it after a reboot.

OgreMain (debug build) generates 1.6 GB of intermediate files, so you better have a lot RAM for your RAMDISK.

I decided to do a full rebuild once a day. This has the nice side effect that every problem will immediately surface itself, not later than the next day, and I will keep an eye on  the build time for a full rebuild.

Some Books I’ve been reading

Over the last couple of weeks I’ve been reading two books I want to talk about, because I liked them very much.

The first book is “Game Engine Architecture” by Jason Gregory.

Game Engine Architecture

Jason Gregory, is currently working at Naughty Dog Inc. Before that, he worked at Midway and Electronic Arts Los Angeles, resulting in a lot of years of practical experience. And this is what makes the book such a great one.

The book starts of with a short description what a game engine actually is, what the requirements for the different types for games and how the architecture of a game engine has to reflect this. No engine can be used for every type of game to archive maximum level of performance and playability.

One of the most astonishing graphics is on page.26, showing all major runtime components typically found in a game engine. Without accounting for all the tools required.  I don’t think that I’m allowed to post this figure here, but think of it as one page full of boxes. Starting at the bottom with the Hardware, Drivers and the OS. Going further up through the Platform Independence Layer, Core Systems Layer … finally at the Top are the Game Specific Subsystems.  15 Layers with more than 100 components.

Just a short note , If you are looking for book with a lot of ready to run code, this book is not for you. It operates at a much higher level talking about what a game engine is.
The algorithms are discussed in detail enough that you can implement them on your own. Including links to further sources of information. 
He also never present just one way. The author always bring up different ways what possible problems and what the strength of this way are.

No game is equal, every game, every platform has different requirements. So know your game.

One chapter which especially profits from the way, is the chapter about containers. Jason shows that seemingly easy construct like Linked Lists are actually not so easy. Extrusive Lists or Intrusive Lists … even when not dealing with multiple threads it can have a lot what can go wrong.

All 800 pages are filled with up-to-date information, practical information about seemingly everything a game engine is made up from the Resource Manager, over seemingly simple topics like time(which aren’t so easy in reality), debugging (runtime and compile time), game loops, animation, rendering, memory management … physics … message passing … and it doesn’t seem to ever end. Take a look at the Table of Contents

My most favorite chapters are the chapter 6, talking about the file-system and the the resource managers, the Chapter talking about all the different possible ways you could architect the Runtime Object Model Architecture and the Chapter about the Animation System, is an especially nice chapter, starting at a simple overview going deep into the details of, as usual, different ways of architect this system.

Even on the 800 pages not all system could be covered. Missing are Audio, Camera, Networking and AI Systems. But just take a look at the book shelves, there are a lot of books about those topics taking hundreds of pages alone.

Every chapter includes links to further resource and discussions. Reading those links gives you a lot of insight, so always read them. They mostly contain some code and give a just an other idea how to implement the discussed components.

Just look how long this post got … This wasn’t my plan, but hey there is a lot more to find in the book,

My advice, do yourself a favor and buy this great book. I think everybody can find some interesting chapters to read and learn.

 

Game Coding Complete, Third Edition

Game Coding Complete, is a completely different book than the previously presented “Game Engine Architecture”.

Game Coding Complete is written by Mike Mcshaffry, currently an Executive Producer at Red Fly Studio. He started working on Ultima in 1991 and since then had worked on numerous games over the years. Just take a look at his about page and you will see how much experience “Mr. Mike” has.

Game Coding Complete is a complete different style of book, than the previously discussed “Game Engine Architecture”. Through the course of the book you will be presented with a lot of code. All building up a small, but functional game engine. This game engine can be downloaded from the following page. The resulting engine is surprisingly easy to get started with and extend it further. But it’s not optimized for performance, but Mike will give hints in the book where performance could be improved and more clever algorithms would be needed.

This book never dos such a high level discussion about all the different architectures and choices you can possibly make. It stays to the implementation he chooses for the “GameCode” Engine.

Starting with what game programming is actually like and why it is hard. It goes over setting up a strong directory structure for you game builds. Discussion of Version Control including setting up branches.

This book contains three special inset blocks.

• “Gotcha” where he describes common mistakes and how to avoid them.
• “Best Practice” some tricks the Author learned over the years to avoid programming trouble.
• “Tales from the Pixel Mines” brings some real stories/anecdotes from real world game projects into the book.

The Math 101 chapter is a nice introduction into the Vector and Matrix Math. But the chapters about 3D programming don’t really go into depth, but 3D Programming is such a big discipline you can never handle it in just to chapters. But there a a lot of great books out there about 3D Programming.

Some topics missing from “Game Engine Architecture” can be found in this book. Network Programming, Introduction to Game AI and an Introduction to Game Audio programming.

The Author shows to to Integrate a Physics Engine into your architecture, he is using the open source Bullet Physics SDK in this revision of the book.

Lua scripting is a very nice chapter. He shows to to build a bride between your C++ and your Lua Scripts. Including how to handle Events send by C++ code and events send by Lua code.

The Chapter “Debugging your Game” is one of the best chapters in the whole book. Crammed with a lot of knowledge from years of practical game programming, it talks about the basics about how the visual studio debugger is organized over symbol level debugging all the way down to assembly level debugging. Debugging tips for graphics and physics. How to create your own mini-dumps and debugging them.

To conclude I also enjoyed this book, especially after reading the more theoretical “Game Engine Architecture”. It was really nice to have working code so I could quickly fire up the compiler and test some things. Through the flexibility in the presented engine you can easily change things and test it.

Thanks to both Authors, co-Authors and everybody who also worked on both books and made them possible. Thanks, you made a great job.

Using the DIA SDK (Debug Interface Access SDK) from .Net

I’m currently working on a new MemorySystem. So I just wanted to access some Debugging Information out of the .pdb file … But this, turned into quite some time spent . The Documentation isn’t really a great help, when you are starting to use the DIA SDK for the first time.

When using the DIA SDK from .Net, even less documentation is available on the Web.

The DIA SDK is a COM component so we can just Add a Reference to it in our C# project.

Under the COM Tab you should find the “dia 2.0 Type Library”, this is the Component you have to add.

If you don’t have the component in the list, as I had. You aren’t lost. The dll is just not registered. We have to do this manually. Normally this should be done by the installer, but the newer versions seem to be not doing this.

Register the dll manually

So start by searching for the DIA SDK directory. This is normally placed under your Visual Studio install directory. When you have multiple version of Visual Studio installed on your computer (as I have) you can choose which version you want to use. I choose to use the Version installed with VS 2010. In most cases you should be using the newest version available to you.

If you have found the directory it’s time to open cmd.exe and move to the \bin folder in the DIA SDK you choose to use.

For me the Path is: D:\Program Files (x86)\Microsoft Visual Studio 10.0\DIA SDK\bin

When you are in the directory, execute the following command, to register the dll.

regsvr32 msdia<VersionNumber>.dll

(For me the file was named: msdia100.dll)

You will now be presented with a window telling you that the dll was registered correctly. If not, you will receive an error.

If you have successfully registered the dll, close Visual Studio and reopen it, so it will reload the references. You should now find the “dia 2.0 Type Library” under the COM components Tab under Add References.

Just add the reference.

Using the DIA SDK to load a .pdb

The following section is not mean to be a complete tutorial, just should show how to use SDK from .Net. From now on always have a look at the Documentation.

The first thing you have to create is a to create a source where the data is coming from.

IDiaDataSource source = new DiaSource();

So now we have to load the actual pdb file. There are multiple possibilities to do this. We can explicitly tell which pdb file to load:

source.loadDataFromPdb(@"D:\Jendrik Illner Projects\C++\TestApplication1\Debug\TestApplication1.pdb");

Or we let the SDK decide which one to use:

source.loadDataForExe(@"D:\Jendrik Illner Projects\C++\TestApplication1\Debug\TestApplication1.exe", @".\Debug", null);

Now we told where the data coming from, but we are not able to access it. For this we have to define and open a IDiaSession.

IDiaSession session;

source.openSession(out session);

Now we are able to access all information available in the PDB file, which can be a huge amount of data.

List all functions from the PDB

This is just a example of what you can do with a PDB file. This will just list all functions listed in the PDB file.

IDiaEnumSymbols results;

session.findChildren(session.globalScope, SymTagEnum.SymTagFunction, null, 0, out results);

foreach (IDiaSymbol symbol in results)
{
      Console.WriteLine(symbol.name);
}

Hope this small Article helps getting you started with the DIA SDK. If you have any questions left unanswered, feel free to ask them.

How to make use of Property Sheets for better solution management for C++ projects in Visual Studio

Setting up a C++ project can take a lot of time and often we are setting the same properties for a lot of projects, so isn’t there a solution to this problem?

Maybe create a custom Project-Template and always create the projects from this template?

Would be feasible but we would need a lot of templates. But what happens if we have to change a property after the project was already created?
We would have to change it for all project by hand …

But there is a a better solution in visual studio for these kind of problems, Property Sheets.

This can help with managing large solutions, but a lot developers don’t know about this feature, because it’s hidden away under the “Other Windows” menu in the shell.

With this post I want to let more people know about this great “hidden” feature of Visual Studio for C++ projects.

What is a Property Sheet?

A Property sheet simply is a .xml file with the file extension .props in Visual Studio 2010 and .vsprops in older versions (2005 and 2008). These file contain property definitions which can than be added to a project so the Properties are automatically set on all projects using the Property Sheet.

But Property Sheets do not contain different configurations, because this would destroy the ease of use.
If you need different settings for different configuration just create multiple Property Sheets.

A Project can use any number of Property Sheets.

What can they be used for?

Property sheets can be used to solve a lot different problems.

I’m using them to place the settings for the different configurations (Debug, Release, Production) outside of the actual projects so I have only one file containing the debug properties, one for release and one for production.

Now every project just has to add these Property Sheets and they are ready to be compiled. With the same settings as every other project in the solution. Using the same Compiler Warning Level, optimization settings, Linker settings and so on.

An other possible use case would be one property sheet for every  external dependency your code is relying on.
For example when we are using boost, we could create a property sheet which contains just the path to header and library directory. So every project which would like to use boost just has to add this property sheet and not have to define the path itself.

Defining a path can take some time, keep in mind if you are compiling for x86 and x64 you have to use different libraries.
So your path has to be defined in a way, it will link to the correct location for the chosen platform which can easily contain an error. If this is the case and you are using a Property Sheet, there is just one place you need to change whether it’s being used in 1 or 20 projects.

These are just some examples, surely there are more problems they can help, just be creative.

How to use?

We talked about the what and why, but now it’s actually time talk about how we actually can use Property Sheets. I will show how to use them in the Visual Studio 2010 Beta 1, but nothing really changed since Visual Studio 2005( in this area) so they versions are mostly equal.

Everything starts with finding the correct window. It’s hidden away under
“View” –> “Other Windows”. There you will find the “Property Manager”.

You will now see this unimpressive window. This is the Property Manager.

It contains every project in the Solution with a “Folder” for every Platform/Configuration Combination. Like Debug/x68 and Debug/x64.

There are some buttons, the first one (from left to right) opens the properties for the currently selected item in the list. If the project is selected, the configuration window for the project will be opened.
If a property Sheets is selected, the properties for the selected Sheet will be opened.

When opening the Properties of a Property Sheet you  have to pay attention that just the values which are bold are actually defined in the sheet. The inherited values are also visible but they are not bold.

The second button adds a new, empty Property Sheet. The button right from it, adds a already existing Sheet to the project.

With the arrays you can move the priority. The Property sheets inherit values from each other. The Property Sheet which is placed at the top has the highest priority and will override the settings from Property Sheets placed below.

My advise would be to open the Property Sheet in an editor after you have made changes and see if just the value you wanted to set are actually set.

If you have changed any values in a Property Sheet, my advice would be to open the file in an xml editor and have a look at the properties you have changed. So you can be sure that have set the correct values.

Make sure to always save the changes, as they will not be saved automatically.

Some Pitfalls

There are some pitfalls you have to keep an eye on.

The first pitfall is that changes to the Property Sheets are not saved automatically. You manually have to save every Property Sheet you have changed. If you are changing a lot Property Sheets it can happens quickly that you have forgotten to save the changes and you are left wondering, why nothing changed.

My Solution for this problem is: I always close the solution and reopen it after I made any changes. This way Visual Studio will ask, If you want to save you changes and will save all Property Sheets automatically. This takes some seconds but it’s a lot faster than recompiling you project just to find out that you have forgotten to save the changes.

The other one is actually a problem with the UI.
It’s really hard to see how the values for the final project are brought together.
There is no other way to see if a Property Sheet overrides settings from any other Sheet without opening both. It’s also not possible to have two Property Windows open at the same time. Which doesn’t make it any easier, as well.

Conclusion

So i hope I could show the potential of Property Sheets for better Project Management in C++ projects and I also could warn you about the potential problems you may encounter when you are using Property Sheets for the first time.

I think this is a great feature and deservers a better handling in the UI than it currently has. Nothing has changed in the Property windows since Visual Studio 2005 and I think it’s time for a rework of those windows.

Thanks for reading and I hope you learned something new.

DirectX August 2009 SDK is released

Also today is the 10.September 2009, Microsoft released the August SDK today.  The 550MB download is available here

The first thing you will see when you try to open the documentation is that the Documentation is now longer one single file. It’s now placed in two different files and it looks like the Documentation isn’t really completely organized by now. It looks like they wanted to get the SDK out.

You will find the Documentation in the \Documentation\DirectX9 folder,all documentation is placed here. I think the location will change in the next release. This doesn’t look right.

One file is called directx_sdk.chm and contains the documentation for the following technologies

• DirectX Audio
• DirectX Input (XInput and DirectInput)
• Windows Games Explorer
• Technical Articles
• Samples and Tutorials

The Samples and Tutorials section also contains the samples for XAudio2, XInput, DirectInput, Direct3D 11, Direct3D 10 and Direct3D 9.

The documentation for the Graphics Technologies are now placed in there own files. In the same \Documentation\DirectX9  folder, you can find the windows_graphics.chm file. This is the documentation for the Graphics Technologies with following content

• Direct3D 11 Graphics
• Direct3D 10 Graphics
• Direct3D 9 Graphics
• DXGI
• HLSL
• Tools for DirectX Graphics

If your are using windows Vista keep an eye on the following site , when it’s available. http://go.microsoft.com/fwlink/?LinkId=160189

This release, is build for the DirectX 11 Runtime. This runtime is installed on Windows 7 RTM versions but has to be installed on Vista machines via a system update. The link will provide more information about it.
I also tested some samples on the Windows 7 Beta and they worked. I don’t have a Vista system to test on but the documentation says some sample may crash on vista without installing the DirectX 11 Runtime.

Interesting new and updated Samples:

The SubD11 Sample once again got updated.  It now contains this nice looking human-model. This samples shows the use of the new Tessellation stage in D3D11, the model also shows skeletal-animation.
Because you have to use the reference device, I really doesn’t make fun to play around with the different settings. Hopefully ATI and NVIDIA will release a D3D11 graphics card quickly.

VarianceShadows11 Sample demonstrates how to integrate cascaded shadow maps with variance shadow maps.

Compute Shader

Also interesting are the new samples for DirectCompute but they have nothing interesting to show on pictures. When you are running Windows 7 with the newest NVIDIA (ATI currently doesn’t have a driver to support it) You can run those samples using DirectCompute using the 4.0 level.

That’s enough for now. Have fun exploring the SDK on your own, I will have it

VertexDataAccessor, a helper to access data in a XNA VertexBuffer

On the XNA-forums somebody asked how-to extract the vertices from a XNA Model. Before answering I firstly searched if there was already a article on the Internet I could link to, but  to my amazement I couldn’t find a real solution which would work for all VertexBuffers.

So I thought that shouldn’t be hard to write a general and more easy way to solve the problem. The old way, which I also have written as answer to the question, involved knowing the vertex declaration and creating a struct based on this. But this isn’t the most elegant way.

A better way is quite simple to implement. I want to describe it in this post and provide my implementation as well.

The Result

Before starting, the Resulting API will look like this:

VertexDataAccessor vertexData = new VertexDataAccessor( mesh.VertexBuffer, mesh.MeshParts[0].VertexDeclaration);

Vector3[] positions = vertexData.GetData<Vector3>(VertexElementUsage.Position);

Introduction; The old way

Before we begin implementing the API shown above, I wanted to show what is necessary to retrieve data from a XNA VertexBuffer, when using the default XNA Model class. If you already know this or aren’t interested in it, feel free to skip this section.

Before we can retrieve any data from the VertexBuffer we firstly have to know what data is available. This is nicely stored in the VertexDeclaration which can be found in the ModelMeshPart contained in a mesh. From this VertexDeclaration we have the possibility to get the VertexElements. This contains information about the usage, the ordering and Types the data in the Buffer.

ModelMeshPart part = mesh.MeshParts[0];
VertexElement[] vertexElements=part.VertexDeclaration.GetVertexElements();

The Content of the VertexElements, we retrieve, could look like this.

[0]: {Stream:0 Offset:0 Format:Vector3 Method:Default Usage:Position UsageIndex:0} 
[1]: {Stream:0 Offset:12 Format:Vector3 Method:Default Usage:Normal UsageIndex:0} 
[2]: {Stream:0 Offset:24 Format:Vector2 Method:Default Usage:TextureCoordinate UsageIndex:0}

We also need the size of each vertex element. This information is also contained in the model Mesh Part.

int sizeInBytes = part.VertexStride;

We can now simply retrieve the data out of the vertex buffer:

VertexPositionNormalTexture[] vertices = new VertexPositionNormalTexture[mesh.VertexBuffer.SizeInBytes / part.VertexStride]; 
mesh.VertexBuffer.GetData<VertexPositionNormalTexture>(vertices);  

We now have the data, but maybe we just wanted the position data, all this work just to get the positions of the vertices? We are also bound to this specific VertexDeclaration. There must be a better way, as you already has seen this way exists and is quite easy to implement.

The better way

VertexDataAccessor vertexData = new VertexDataAccessor( mesh.VertexBuffer, mesh.MeshParts[0].VertexDeclaration);

Vector3[] positions = vertexData.GetData<Vector3>(VertexElementUsage.Position);

This how the result will look, as you have seen above. This is also independent of the actual VertexDeclaration, as long as the requested data is available in the VertexBuffer.

Lets begin implementing it:

Firstly we need a class that will contain the logic, this will be called VertexDataAccessor.
The Construct takes the VertexBuffer and the VertexDeclaration for the VertexBuffer.

public VertexDataAccessor(VertexBuffer vertexBuffer, VertexDeclaration declaration) 
{
    // create the array to hold the vertex buffer
    this.buffers = new VertexBuffer[1];          
    this.buffers[0] = vertexBuffer;              
    this.vertexDeclaration = declaration;         
}

As you can see we simply store the values. The Vertex Buffer will actually be stored in an array because multiple VertexBuffer’s could be used.

Now we have to implement the actual logic to retrieve the data from the Buffer. As you have seen the method for this is called GetData<T> and is a generic method. So it can handle all kinds of requests, but the generics has to be restricted to value types. Because only value types are supported as data inside a VertexBuffer.

public T[] GetData<T>(VertexElementUsage usage) where T:struct

Now to the implementation. The first step is to check if the requested usage is contained in the VertexBuffer. We check this by looking into the VertexDeclaration provided to the constructor.

// firstly check if requested data is available
VertexElement[] vertexElements = this.vertexDeclaration.GetVertexElements();
// this element will contains the requested element, if we can find it
// otherwise we will check the Stream if it is left -1 the vertexElement couldn't be found
VertexElement requestedElement = new VertexElement(-1, 0, VertexElementFormat.Unused, VertexElementMethod.Default, VertexElementUsage.Binormal, 0);

foreach (VertexElement element in vertexElements)
{
    if (element.VertexElementUsage == usage)
    {
        // we found the element describing the request data
        requestedElement = element;
    }
}

if (requestedElement.Stream == -1)
{
    throw new Exception("The requested data with the given VertexElementUsage is not present in the VertexBuffer");
}

The Rest is actually quite easy, as you will see.
We begin by collection the data for the request. The most important one is the offset. The offset define how many bytes at the beginning of each element should be skipped. XNA takes care of copying the right amount of memory into our element data.

We than handle the case that multiple vertex buffers could be used. This information is also in the VertexDeclaration structure, called Stream. So we just have to calculate the number of elements(vertices) in the buffer and create a array that for this number of elements.

Now only copy the data from the VertexBuffer using the VertexBuffer.Get(..) method defined by the XNA framework and were done.

// generate the values we need to get the data out of the vertex buffer
int offset = requestedElement.Offset;
int startIndex = 0; // always copy the complete buffer
int streamID = requestedElement.Stream; 
int vertexStride = this.vertexDeclaration.GetVertexStrideSize(streamID); // the Stream defined which vertex buffer contains the data
int elementCount = this.buffers[streamID].SizeInBytes / vertexStride;
T[] data = new T[elementCount];

// get the actual data out of the vertex buffer
this.buffers[requestedElement.Stream].GetData<T>(offset, data, startIndex, elementCount, vertexStride);
return data;

You can download my implementation, it contains some more methods than shown here, but the actual logical is nearly the same: download

Spore Creatures

The latest spore patch added a “Cheat” to export creatures to Collada. I started playing around with it.

Export a Creature

To export a creature to collada, you have go into the creature editor and bring up the cheat console. (Strg-Shift-C) and type in colladaexport.  The first time you have to accept the EULA. The path for the exported creature is shown in the console.

Importing a static mesh

I started with a creature I had made before. I just exported it and got a 5.3mb colllada file plus a diffuse, normal and specular texture. I now started to import it into the various 3d packages I have access to, but I couldn’t get it into Blender at all. In the Softimage Mod Tool it looked better but just the mesh was imported. It has lost the uv mapping and rigging information, yes the creatures are also rigged, when exported. Which makes it especially interesting.

So now it just tried to export it as .fbx using the 2010.0 FBX Converter. The conversion was successful without any warnings or errors.
So just added it to a xna project and the pipeline imported it. When rendering the model I got this result.

The model is imported correctly, just the texture is missing. SO I had to manually set the diffuse texture and everything looks like it should look.

I tried it with all kind of creatures and everything worked great.

Importing as skinned mesh

As I said before all exported creatures are completely rigged, so I now tried to import and render the meshes as skinned meshes. But as I opened the collada file to see how many bones a creature has, I was a little bit shocked. This mesh had 192 bones.

So I created a new mesh, a lot simpler with just 52 bones. This is ok. My existing skinning solution supported up 80 bones.

I just imported it and everything worked, without any problems.  You can see the skinned model below on the left in the bind pose and on the right with a big toes, just one bone scaled up.

More complex creatures

But now back to my previous test creature,  which has 192 bones. But this creature isn’t the most complex you can build in spore. After some more testing I had to see that creature can get a lot more than 200 bones. The Question, how should I handle those large number of bones? Implement a mesh splitting algorithm or do the skinning on the CPU?
I decided to choose the CPU way, after I couldn’t find any resources about a mesh splitting algorithm on the web.
I just want to play with the creatures for now,  if the performance would be to bad I always could go back and implement skinning on the GPU at a later time.

Current State

I have now implemented skinning on the CPU and I’m able to import and use every creature from the spore universe.

What’s next?

Next step is to implement the creation of animations directly into xna, as I’m not able to get the creatures into any other 3d package I own.

This will be an interesting experiment to get this working.

Using Windows Management Instrumentation(WMI) to get Hardware Information in .Net

I’m currently working on some new projects in .Net which use the new addition for Parallel Programming in the .NET Framework. While developing I noticed that my simple-log really isn’t that great in a Multithreaded Project. I was using a simple text file. I’m now switching to an new log-system, I’m currently working on, but this is a different post.

As I was designing the new log-system I saw that I don’t had any information about the system the application was running on. I was dependent that everybody who send me a log for a bug report also tells me their system information. This wasn’t that clever. So I started to look for a way to get all those data in .Net and I found out that WMI also can be easily be accessed from .Net.

The code to access WMI is in the System.Management namespace. This is in the Assembly with the same name. Which is referenced by default, so don’t forget to add the reference.

When you have found all the different pieces needed to query all the different data you have done the task which takes most of the time. Sadly I haven’t found any resource which had all the links in one place. I will post the links to the documentations you need at the end of the post.

When we have all the documentation open it’s really easy to get a lot of information from the hardware. So lets start.

The first step is to create a ManagementObjectSearcher. A ManagementObjectSearcher returns a collection of ManagementObject based on the query defined when creating the Searcher. The ManagementObjectSearcher overload I will be using here takes two strings. The first argument contains the scope in which the query should be executed and the second argument is the actual query. But lets look at the code:

ManagementObjectSearcher searcher = new ManagementObjectSearcher(“root\\CIMV2″, “SELECT * FROM Win32_Processor”);

This ManagementObjectSearcher queries all the Processors in the “root\\CIMV2” scope, this is the default scope, it contains all the hardware.

Now we have to get the actual processors from the searcher.
foreach (ManagementObject wmiObject in searcher.Get())
{
// [Add Code here]
}
The searcher.Get() methods returns a ManagementObjectCollection so we use a foreach to get the data. Maybe the PC has multiple elements of this kind as well so keep in mind to handle those cases as well.

Now it’s time again to look in the documentation, in this case we have to look at the WIN_32_Processor documentation. At this page you get a look at all the different properties you have access to.

If you have found the name of the property you want to get the data from you just have to call the following code. You just have to replace the string with the name the property you want to get the value from.
object clockSpeedObject = wmiObject.GetPropertyValue(“CurrentClockSpeed”);
uint clockSpeed = (uint) clockSpeedObject;
GetPropertyValue(..) will always return a object so you have to cast it your self, always check the types the Property values is in.

This is everything needed you need to know. Below are some samples showing how to retrieve data from GPU, CPU and RAM.

Hope you find this useful. This article just scratches the surface of what WMI is able to do.

GPU

ManagementObjectSearcher searcher = newManagementObjectSearcher(“root\\CIMV2″, “SELECT * FROM Win32_VideoController”);

foreach (ManagementObject wmiObject in searcher.Get())
{
object memoryInBytes = wmiObject.GetPropertyValue(“AdapterRAM”);

// some video controllers are listed without RAM
// those aren’t real cards, so can we ignore them
if (memoryInBytes != null)
{
// calculate Memory in MB
uint memoryInMB = (uint)memoryInBytes / 1024 / 1024;

string cardName = (string)wmiObject.GetPropertyValue(“Description”);
string driverVersion = (string)wmiObject.GetPropertyValue(“DriverVersion”);
string manufacturer = (string)wmiObject.GetPropertyValue(“AdapterCompatibility”);
}
}

CPU

ManagementObjectSearcher searcher = new ManagementObjectSearcher(

 "root\\CIMV2", "SELECT * FROM Win32_Processor");

foreach (ManagementObject wmiObject in searcher.Get())
{
      object clockSpeedObject = wmiObject.GetPropertyValue("CurrentClockSpeed");

      // get all the informations about the processor
      uint clockSpeed = (uint)wmiObject.GetPropertyValue("CurrentClockSpeed");
      uint physicalCoreCount = (uint)wmiObject.GetPropertyValue("NumberOfCores";
      uint logicalCoreCount = (uint)wmiObject.GetPropertyValue("NumberOfLogicalProcessors");
      string manufacturer = (string)wmiObject.GetPropertyValue("Manufacturer");
      string name = (string)wmiObject.GetPropertyValue("Name");
}

RAM

ManagementObjectSearcher searcher = new ManagementObjectSearcher(

                     "root\\CIMV2", "SELECT * FROM Win32_PhysicalMemory");

int totalMemory = 0;

// calculate the RAM capacity we have
//  we have to add every RAM-Stick Capacity to get the fincal value
foreach (ManagementObject wmiObject in searcher.Get())
{
     ulong capacityInBytes = (ulong)wmiObject.GetPropertyValue("Capacity");

     // calculate in MB
     ulong capacityInMB = capacityInBytes / 1024 / 1024;

     totalMemory += (int)capacityInMB;
}

Links

• WMI MSDN Documentation Windows Management Instrumentation
• Overview about the different Classes dealing with System Hardware: Computer System Hardware Classes
• Overview of the namespace to work with WMI in .Net System.Management Namespace
• ManagementObjectSearcher
• ManagementObject

Opera Unite shareFile problem and solution

WARNING: I’m not very experienced with Opera Unite, the technique presented in this post works for me and doesn’t has to be the best way possible. If you find a better way please let me know.

Opera Unite Version: first release

In Opera Unite all files are on the server and can not be accessed without specify allowing it. But when you share a file using the given APIs opera.webserver.webserver.shareFile(..) method this breaks and becomes unavailable. This seems to be somehow caused be loading the file again while the last download was still running.

As described on http://unitehowto.com/. (Thanks for the great work you are doing)
He found a way to solve this problem, he reloads the scripts on each request but I have thought there has to be an other way.

And I found one while looking through the different examples opera provides with unite. The solution for this problem seems that you have to share and unshare the file on each request. I don’t know if this is a bug or by indent. Opera can you tell us more?

But know I want you to show the solution I came up with.

Everything starts in the window.onload method, this method is called when the service is started. So everything the server has to initialize should be done here.

The first step is to get the webserver and filesystem object. This has nothing to-do specifically with this problem.

var webserver = opera.io.webserver;
var filesystem = opera.io.filesystem;

But the next line is important. You have to mount the the ‘application’ directory.So you have access to the application files.

this.application = filesystem.mountSystemDirectory('application');

webserver.addEventListener('_request', request, false);           

You also have to event Listener for the ‘_request’ event. This event will always be called when a site is requested. In this method we will do the work to make the files available.

And now lets start the most work.

function request(evt) {

    // make alias

    webserver = opera.io.webserver
    var request = evt.connection.request;
    var path = request.uri;

    path = path.substr(webserver.currentServicePath.length, path.length);
    path = path.replace(/\?.*$/, ''); // remove the query path

 
    if (path != "") {
        if (staticFiles[path]) {
            var file = self.application.resolve(staticFiles[path]);
            opera.io.webserver.shareFile(file, path);
            response.closeAndRedispatch();
            opera.io.webserver.unshareFile(file);
            return;
        }
    }

   if (response){

    response.closeAndRedispatch();

   }
}

This is a lot of code so lets get started. We begin the class by getting get webserver, request and the path which was requested.

We now have to remove the current service name from the string. The path has the format

/[serviceName]/[request].

After the service name and query path is removed we have to heck if the path is even contains any data now. If we call the default start site this string will be empty because we haven’t’ requested anything. If the file exists I check in the staticFiles directory if we have a static file defined with the given path. The directory looks like the following,

var staticFiles = { 'imgs/img.jpg': 'sharedFiles/imgs/img.jpg' }

The first element is the path the file should be accessed by the user and the value define the actual path of the file.

If we have defined a static file under the given path we will continue and call application.resolve(..) on the value of the staticFile directory. This gives us the the file we can now share.

for this we call webserver.shareFile(file, path) and feed in the file we just received and the path we generated before.

After the file is shared we call webserver.closeAndRedispatch(). This closes the connection and redispatch the request to the Web server.

After that we just have to unshare the file again. webserver.unshareFile(file) and call return;

But we also have to handle the case if we haven’t defined the static-file the user requested. This is done in the last if(response) block. Here we just webserver.closeAndRedispatch() and let the webserver handle what to do, normally show an 404 Not found error page.

This is everything necessary to get static-files shared without getting the resource broken after some time.

That’s it hope this information is helpful.

My Top-3-Developer Tools

The German msdn blog “Softwarehersteller in Deutschland” is asking what are your favorite 3 developer tools and here are mine:

1. Visual Studio + Add-Ins

The first tool I cannot live without is Visual Studio. I always try to use it but sometimes,  in school, I have to use different IDEs .. and then I know again why I love Visual Studio.
Also the Add-Ins available for Visual Studio are so great. I could have listed them as tools as well, but I don’t count them here. Some Add-Ins I’m using a lot are GhostDoc, TortoiseSVN, TestDriven.Net … just to mention some examples.

2. ToDoList – free

The second tool I have chosen is ToDoList. This is a simple but great todo-list management applications.
It has all the features a lot of the expensive tools have, but for free.
The UI isn’t something special but it’s functional and that’s what count.

3. XmlMarker – free

XmlMarker is the third tool I have chosen, as I work quite often with xml files.
When the xml-files are small and simple you can manage them in Visual Studio. But when the files get complex and large, as Collada files you can’t deal with them in Visual Studio.
Visual Studio is not designed for those long lines and that much data.

But XmlMarker really handles this great.  This is the best xml-tool I have found and it’s free.