HTTPS and certificates in lower Cobalt environments

Unknown

I've encountered an issue where certain Cobalt modules wish to call each other's APIs via HTTPS. The problem, though, is that in lower environments, the machine's certificates don't match the host they're running on. For example:
next-client.demo.westlaw.com uses an invalid security certificate. The certificate is only valid for *.next.demo.westlaw.com (Error code: ssl_error_bad_cert_domain) I should underscore that these aren't problems visiting the host in my browser (it's easy enough to add security exceptions), but where a module is calling another module's endpoint via HTTPS and the Apache HTTP client rejecting the connection due to certificate errors. There are a few approaches to this, ranging from hackish (ignore certificate errors based on a CMDB value) to "correct" but possibly a bunch of work (make every host have a properly matching SSL certicate). Is there a standard approach to solving this in the Cobalt environment that I should try first before experimenting?

Benjamin Fairchild

The following email may have some bearing: Potential for Breaking Tests!! All, There are some changes coming through iRules that will prohibit the ability to direct target WLN using HTTPS. If any of your tests direct target using HTTPS, switch them to be HTTP or else your tests will start failing. This includes post deploy and uptime tests. Most tests should already be using HTTP, but this is a heads up that HTTPS will no longer be supported. If you have any questions feel free to ask. Thanks, ~Grant. From: Ohrt, Greg (Legal) Sent: Tuesday, October 28, 2014 3:21 PM To: Fundaun, Iver (Legal); Hanson, Grant (Legal) Cc: Co-iRules-Tech Subject: FW: Direct Targeting Cleanup Iver and Grant, Mark had recommended that we reach out to alert QED testers/Scale testers, SWAT, AMS, Release Management that we were scheduling (see below) the removal of the http direct targeting support for WLN. Basically eliminating the capability to do https (we will still support http calls – as we always have) calls using direct targeting domains E.G.: 163.231.66.11 (bigipcobaltqedb)
cobaltigroute.int.next.qed.westlaw.com
qed-a.m.next.qed.westlaw.com
qed-a-1.m.next.qed.westlaw.com
qed-b.m.next.qed.westlaw.com
qed-b-1.m.next.qed.westlaw.com
qed-b-2.m.next.qed.westlaw.com
qed-b-3.m.next.qed.westlaw.com
qed-a.next.qed.westlaw.com
qed-a-1.next.qed.westlaw.com
qed-b.next.qed.westlaw.com
qed-b-1.next.qed.westlaw.com
qed-b-2.next.qed.westlaw.com
qed-b-3.next.qed.westlaw.com Port 80: 167.68.88.43 (bigipcobaltb)
cobaltigroute.ha.next.westlaw.com
cobaltigroute.int.next.westlaw.com
prod-a.m.next.westlaw.com
prod-a-1.m.next.westlaw.com
prod-a-2.m.next.westlaw.com
prod-a-3.m.next.westlaw.com
prod-b.m.next.westlaw.com
prod-b-1.m.next.westlaw.com
prod-b-2.m.next.westlaw.com
prod-b-3.m.next.westlaw.com
prod-a.next.westlaw.com
prod-a-1.next.westlaw.com
prod-a-2.next.westlaw.com
prod-a-3.next.westlaw.com We had started adding https servers - more recently - to eliminate a problem in the core search module – which has now been corrected. We want to adhere to our standard of http only support for this type of routing. We don’t want anyone to be surprised by this change (elimination of the https servers). How would we want to communicate this type of change (thought you folks might have some ideas)? Greg

craig.wageman

We are in the process of introducing HTTPS for platform services, starting with the security module to meet increased data security requirements. For certificate management, we have not finalized our approach, but the options we are considering are: 1) purchase a wildcard certificate or SAN certificate at F5 and use self-signed certificates on the individual servers. F5 standard configuration does not validate the certificate received from the server. 2) create and manage an internal CA to generate valid certificates 3) use self-signed certificates at F5 and have clients ignore certificate validation errors The most significant obstacle is that we have multiple domain names sharing the same virtual server at F5, which can only be handled with a SAN cert if clients are going to validate the certificate host name. It isn't feasible to separate them and set up at F5 to have matching certificates installed. The cost could be quite high to purchase a SAN certificate, given the number of hostnames. For services (e.g., Data Room, DocCompare, VirusScan, etc) that use a common domain name where a common domain name is used with single level of subdomains, a wildcard certificate makes it fairly simple.

Unknown

Thanks for the information. In the near-term, then, is it simply suggested that applications using HTTPS for endpoints relax their certificate validation? From your reply here, it sounds like getting all the hostnames to validate with proper certificates might be a ways off. Should we expect the production environment to be any different than pre-prod now? For example, should I expected
foohost-somevariant.int.westlaw.com to present a valid certificate like
foohost.int.westlaw.com would in production?

craig.wageman

I would expect that prod and pre-prod would be set up following the same approach. If you are asking whether or not the same certificate would be used across all environments, I would say not (although that approach could be taken, if validation will not be done on the hostname). If you look at the hostnames configured in zipper at the supervip for WLNext, a wildcard cert (*.
int.next.westlaw.com) could be used and clients could validate, and likewise in WLNext non-prod (*.int.next.{ci|demo|qed}.
westlaw.com). However, in the shared environment, a wildcard certificate would not work, because there are different domain names, so assuming not purchasing a SAN certificate and adding all of the names, the certificate used will not match the requested hostname, so whether the same certificate or per env certificate were installed for the VIP would not really matter (just client must not validate). In the exact scenario outlined with
next-client.demo.westlaw.com, it seems like you just need to conform to the standard subdomain naming of *.
next.demo.westlaw.com so that you can take advantage of the wildcard certificate. So, use
client.next.demo.westlan.com, for example.