Performance: C++/CLI vs COM

edit comments

In my current project, we are going to require a great deal of native code due to interation with drivers and hardware. The application is going to be based in .NET, at least the UI portion, but the heart has to live in native C++.

I began investigating different ways to interop with .NET and unmanaged code. There seems to be three approaches.

  1. PInvoke
  2. COM/COM+/Interop
  3. C++/CLI

I did not look into the PInvoke method because I know that it will not be suitable for my needs. Things get complicated really quick with PInvoke. It is ideal for small/quick access to Win32, but not beyond that. With that said, I produced some metrics about the performance between COM+ and C++/CLI.

I have one class (unmanaged) that I use in every metric.

#pragma once
__declspec(dllexport) class NativeClass
	__declspec(dllexport) NativeClass(void);
	__declspec(dllexport) ~NativeClass(void);
	__declspec(dllexport) int GetWindowsVersion(int numberOfExecutions);

There are duplications of this class but they all do the same thing. The method we are testing is a simple GetVersionEx from the WinAPI.

int NativeClass::GetWindowsVersion(int numberOfExecutions)

	for(int x = 0; x < numberOfExecutions; x++)
		ZeroMemory(&osvi, sizeof(OSVERSIONINFO));
		osvi.dwOSVersionInfoSize = sizeof(OSVERSIONINFO);

	return osvi.dwBuildNumber;

The parameter “numberOfExecutions” tells the unmanaged class how many times it should invoke GetVersionEx. This native class is written/duplicated in multiple projects to similate the same process. I have 4 projects.

  1. PerformanceTest - This is a console application that runs the performance in C#/.NET
  2. CLITest - This is a C++/CLI project that contains two classes. The first class is a wrapper around an unmanaged class that is compiled in the same C++/CLI project. The second class is a wrapper around an unmanaged class that is compiled in a seperate native dll (not C++/CLI).
  3. COMTest - This is a project that has a COM wrapper around an unmanaged class.

I tested two scenarios.

  1. Calling an unmanaged class from .NET multiple times.
  2. Calling an unmanaged class from .NET a single time while doing a large task (numberOfExecutions = 10000000).

Here are the results from the console application for the 3 situations.

  1. .NET to CLI calling an unmanaged class compiled in C++/CLI.
  2. .NET to CLI calling an unmanaged class compiled in native C++.
  3. .NET to COM calling an unmanaged class compiled in native C++.

Here is the output of the test.

         Calls: 100000
         Executions: 1
         Result: 00:00:00.0190000
         Calls: 1
         Executions: 100000000
         Result: 00:00:15.7790000
CLI to native................
         Calls: 100000
         Executions: 1
         Result: 00:00:00.0180000
         Calls: 1
         Executions: 100000000
         Result: 00:00:04.8020000
         Calls: 100000
         Executions: 1
         Result: 00:00:02.6120000
         Calls: 1
         Executions: 100000000
         Result: 00:00:04.7310000


Native code compiled and running under C++/CLI is a great deal slower than running pure native code. COM would be the obviouse choice due to it running pure unmanaged code, but C++/CLI can give the same performance if used as a wrapper around a pure native assembly. In addition, COM/ATL is very difficult to learn, however, Visual Studio’s class wizards make it easier. Also, it seems that, making a great deal of calls to native code is MUCH more efficient through C++/CLI as opposed to COM, but this may be an unlikely scenario. Given these results and given the fact that this native code I am writing will be written primarily for .NET (as opposed to COM/activex/etc), I would recommend using the C++/CLI wrapper approach. Same performance, more flexible interop.

The project used for running these tests can be found on github here.

Why you should avoid Ektron

edit comments

The first Ektron project I worked on, I had no experience with it. I am a very quick learner and dive into new things head first, so I gave Ektron and honest chance. After setting Ektron up out of the box, I immediately started feeling uncomfortable. The first thing I noticed were the tables.

Alt text

This immediately gave me the feeling that there are separate divisions within Ektron that live by their own rules, sorta like the wild west. “Maybe this will be the extent of my discomfort.” I told my self. Then, I saw that the product was a mix of C#/VB. This goes against best practices, especially since the code in each language serves the same problem domain! Edit: Latest release now includes only C#.

In addition, the product only supports websites (not web applications). You will be hard-pressed to find any website now-a-days (excluding Ekton websites) that still use the website project template. This has introduced many pain points. For one, you are unable to “exclude” the Workarea from Visual Studio. This is a problem because the workarea is very large. It ruins the performance of Visual Studio due to intellisense namespace/object caching because Visual Studio has to inspect that directory. People have suggested “hiding” the workarea and a filesystem level, but this doesn’t work in development due to widgets referencing user contols that exist in the workarea (which can’t be located). Also, websites don’t work well with a build process of any kind. It increased my build times on TFS to almost 30 minutes! There are some workarounds, but none that don’t involve re-deploying the workarea every time (which is very large). Their latest versions claim to support web applications, but there is no template provided. You must use the “Convert to web application” command in visual studio. There is stil the problem of uncompiled source code that shouldn’t be built/available in implementations. At my last company, I wanted to have everything compiled in my implementation. So, I decided to publish the website, perform an aspnet_merge to create one (very big) EktronWeb.dll, and add the output to my implementation. I was notified by support that the solution would be unsupported. That is ridiculous to so say that a published version of your website (which is essentially what I had) would be unsupported. That is like saying you do not support any deployed instances of Ektron! I was forced to waste a great deal of time building/deploying. Long story short, web applications are the way to go and avoid any project that is a website (including Ektron). Ektron is the only CMS I have seen that uses a website project template. There are good reasons for this Ektron! Take note!

Also, the usability of Ektron is horrible. They almost refuse to adhere to any kind of UI/layout/style standard. This leads to a large amount of HTML/CSS that is required to style forms that adhere to their own standards (or lack thereof). It also forces the end users to constantly rethink what is going on in each area/page, therefore, ruining usability.

The use of XML/XLST is largely outdated and Ektron insists on being centered around this idea. Their SmartForm is very unusable in real world situations. It is entirely xml based and corrupts very easily. Support recommended that I remove each field 1-by-1 to determine what was corrupting it, which is unacceptable. You cannot change the smartform after content has already been applied to it. Collections cannot be reordered without running the risk of losing content. Working with the data that the smartform generates (in xml) is also verify difficult, however, recent versions have allowed you to map the xml to a .NET object. Also, since that database stores the content in one huge xml field (that isn’t even an xml data type), it is almost impossible to query smartforms based on criteria without loading all the items in memory. I saw somewhere that they support linq, but further investigation revealed that they support linq-to-objects that are in memory. This is a out-of-the-box .NET feature, not an Ektron feature! They recently introduced support for criteria seaching, but it is still unacceptable, due to the fact you can only query standard fields in Ektron. This is not valid in the real world due to most implementations will want to filter/sort based on custom fields specific to the implementation.

The editor for Ektron insists (no way around it) on putting ‘&#160’ everywhere which makes css break and html bloated. Developers are forced to add .Replace(“&#160”,””); everywhere, which is also unacceptable. Edit: The latest version has a new editor which doesn’t do this.

Also, MVC will never be supported. I have made MVC work inside of Ektron, but Ektron itself will never be centered around the pattern. This is a feature I like to see in the products I choose. Now, I can’t really say for certain they will never support it, but given what I know about how page builder, embedded widgets and the workarea, it is a safe bet to assume so. A rewrite is needed (for more than 1 reason).

Every CMS has support for building custom forms, but Ektron’s “HTML Forms/Surveys” is not for the standard business user. It requires an advanced knowledge of XHTML. My client needed to build custom forms, but they weren’t a web dev shop, so they couldn’t use the feature. Unless, of course, they paid $180 an hour for my previous job’s time. They were extremely upset with it. Yet another Ektron site that the client later regrets. I will say it again, Ektron must move from being XML centered.

The Ektron API’s are unusable from non-web applications which make one-off tasks impossible without the use of hand-crafted ADO.NET code.

Ektron also uses many 3rd party libraries, but they refuse to keep them updated, which forces implementors to use outdated libraries. My project required the latest version of Netwon’s JSON.NET, but Ektron had a very outdated version of it in use. I was force to download JSON.NET, add a wrapper namespace and rename the assembly to avoid breaking Ektron (which the new version did). This is unacceptable Ektron. Before each release, you should ensure all 3rd party libraries are up to date.

Long story short, I was forced into many situations where I had to explain to my clients that things were not possible. For example, I had to tell a client that they couldn’t generate a sitemap for google. There response was an awkward “Well, that is a standard feature of CMS system.” I was inclined to agree and was forced to defend a product that I strongly oppose.

After my experiences with Ektron, I realized that I am not the only one. Sure, every product will have its naysayers, but I believe (as well as others) that there is an alarming amount of negative press surrounding Ektron. Ektron must shift its focus from developing features to improving that architecture of Ektron, which I don’t think is possible without doing a complete rewrite. It is unfortunate that they do infact have a great marketing/sales team because they are polluting the CMS industry with a horrible product. Clients must stop assuming that “This product is expensive, so it MUST be good!” Have your software architects review the product before shelling out a great deal of money for licenses/support!