You should use Heat (WIX 3.0) located in the bin directory of the version you are using.
Have a look at this blog post, we use it here to register all our COM objects, by creating a wix fragment…
something like
heat file MyComExposedLibrary.dll -out MyComExposedLibrary.wxs
After, reading your edit, I would create a basic msi with wix that installs the com object only, see if that works … then you’ll know which battlefield to attack …