VMCA Intermediate CA

In vCenter 6 (and maybe some earlier versions) it's possible to configure the vCenter server to act as a subordinate CA to your existing PKI and issue all certs as part of a trusted chain. There's lots of instructions on the VMware knowledge base, but there are at least two critical errors in the CLI guide.

On this page that describes the process of compiling the VMCA Chained Cert that includes your root (and any other intermediate certs) it clearly shows the order as:

-----BEGIN CERTIFICATE-----
Certificate of VMCA
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
Certificate of intermediary CA
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
Certificate of Root CA
-----END CERTIFICATE-----

But if you try this you will receive errors regarding an invalid cert chain. Specifically you will see:

Error Code : 70063
Error Message : Invalid Certificate Chain was gives as input

That's because this is exactly backwards! The new cert needs to be at the top, followed by the intermediate, then the root. I learned that on a couple of different blogs. I was hesitant to believe it at first, but once I found it in multiple sources I gave it a shot and sure enough it worked.

The second error relates to the vpxd service not restarting in time, leading the certificate-manager to attempt to rollback the changes (which fails). This situation is described in this forum posting, but no answer is given. After many hours of testing, and digging through logs, etc, a possible solution was discovered on this only tangentially related forum thread: nowhere in the documentation is there any mention of required OU entries for any of the certificates, but this blog post states that as of 6.0u1b and 6.0u2 there are in fact distinct requirements. They are as follows

For the MACHINE CSR use "Root" for Organizational Unit (OU)

For User Solution User Certificate CSRs:
For Machine, use "Machine" as OU
For vsphere-webclient, use "WebClient" as OU
For vpxd, use "VPXD" as OU
For vpxd-extension, use "VPXD-EXT" as OU

After changing these values the process ran through to completion.

My last headache was cause by my reading "stopping services" and "starting services" to mean that the services had been restarted. In fact it was necessary to restart all services (or the vcenter instance in my case) before all of the new certs took effect.

I hope this helps someone in the future avoid some of the pain I've experienced for the past two days.