Threat Report: Exposing Malware in Linux-Based Multi-Cloud Environments | Download Now

Carbon Black Server Configuration - Using a Reverse Proxy

Carbon Black Server Configuration - Using a Reverse Proxy

Communication between the Carbon Black Enterprise Server and its sensors is handled via HTTPS and validated using both server-side SSL certificates and sensor-side client certificates. If you want to use a Reverse Proxy in your Carbon Black environment, special steps must be taken to allow validation of communications with the proxy if SSL termination will be performed. This document provides the configuration necessary to set up the Carbon Black Server for use with a Reverse Proxy.

Labels (1)
Attachments
Comments

Looks relatively simple. However, it does leave the task of working out how to set those header variables on your chosen reverse proxy. I wonder whether it'd be useful to pick a popular open source proxy and provide a sample config for those adding a proxy specifically for this purpose (or where their current proxy isn't able to perform the necessary modifications).

This header seems was customized for Cb Response.

For Apache, I believe the following configuration can insert the custom header to the Reversed-proxy request.

Header set X-Client-Cert-ID "expr=%{X-Client-Cert-ID}"

However, I could not retrieved the value from this variable "%{X-Client-Cert-ID}".

Does anyone know how to dump SSL Session Variable in Apache?

Reference link. This is a good example for configuring the Reverse Proxy.

Tricks to do client certificate authentications behind a reverse proxy

Hi Yves,

I am going to setup a lab to explore this subject further this week, but from my initial optics it seems the expected value for that header is a serial number of Cert sent by Client - in that case maybe usage of SSL_CLIENT_M_SERIAL would work instead (documentation has it as string for "The serial of the client certificate" ) ?

The link you provided has an example around that as well: SSL_CLIENT_M_SERIAL "%{SSL_CLIENT_M_SERIAL}s" (not sure what is the role of the s part in this configuration, I need to explore it further in the lab)

Hi Lucas,

Thanks for your information.

I did the testing. I record the variable with the CustomLog directive in Apache.

I think the variable "%{SSL_CLIENT_M_SERIAL}x" contains the values we need to pass to the backend server.

Then I tried to add this values to a custom Request Header. However, the variable "%{SSL_CLIENT_M_SERIAL}x", can't be used in the Request Header. This is my reverse proxy configuration.

Screen Shot 2018-03-07 at 01.18.24.png

Hi Yves,

I am quite new to Apache, but reading the reference guide (unless I am totally reading it wrong) it seems:

A)

SSLOptions:

StdEnvVars

is what contains the SSL_CLIENT_M_SERIAL environmental variable, ExportCertData has only

:

"ExportCertData

When this option is enabled, additional CGI/SSI environment variables are created: SSL_SERVER_CERT, SSL_CLIENT_CERT and SSL_CLIENT_CERT_CHAIN_n (with n = 0,1,2,..)." as per mod_ssl - Apache HTTP Server Version 2.4

Might be it would be worth trying to modify your SSLOptions line to also include +StdEnvVars

I know per guide it also says:

" The %s format specifier is only available in Apache 2.1 and later; it can be used instead of %e to avoid the overhead of enabling SSLOptions +StdEnvVars. If SSLOptions +StdEnvVars must be enabled anyway for some other reason, %e will be more efficient than %s.

"

therefore it might be non-issue...

B) given the above^ can you please let me know your Apache version?

C) also worth considering is setting a blank header first before your current RequestHeader line:

RequestHeader set X-Client-Cert-ID ""

D)

I was also digging through other bits and pieces and if after modifying the above options the header is still not passed maybe you can try RequestHeader set X-Client-Cert-ID "%{SSL_CLIENT_M_SERIAL}e"

In the Apache guide I found that for 2.1+ versions "s" should do just fine after the variable just to find some obscure comment somewhere where a person was successful with both e and +StdEnvVars

I was short on time to set my own environment today to test all of the above, but will also try to explore it from my own side soon - are there any log files you can post btw?

Ps: Please keep in mind any advice on non-CarbonBlack appliance is not part of official CarbonBlack support)

Hi Lucas,

Appreciate for your help. I did the test last night. It is working fine.

This configuration to insert the custom header for Carbon Black Response.

RequestHeader set X-Client-Cert-ID "%{SSL_CLIENT_M_SERIAL}s"

Hi Yves,

That is great to hear!

Can you share your config the same way you shared it in a screenshot before? It might benefit somebody else in the future

HostnameLookups Off

NameVirtualHost *:443

####

#

#    This is the building block of one master or minion server.

#   Please copy this block <Virtualhost *:443> ... </Virtualhost> for each master or minion servers.

#    The Agent need to check-in with Master server. Then the data will submit to the minion.

#    All Master and Minion need to connect to the Internet.

#

####

<VirtualHost *:443>

   ServerName cbr.carbonblack.local

   ServerAlias cbr-master.carbonblack.local

   SSLProxyEngine on

   SSLEngine on

   ##

   ## Optional: Control the SSL Protocol. Cb Reponse enabled TLS1.2 by default.

   ##

   #SSLProtocol all -SSLv2

   #SSLCipherSuite HIGH:MEDIUM:!aNULL:!MD5:!SEED:!IDEA

   ##

   ## This is the certificate of your carbon black server. This ceritificate applied for API / HTTPS traffic.

   ## The Sensor will validate the server with this certificate.

   ##

   SSLCertificateFile /etc/httpd/conf.d/carbonblack.crt

   SSLCertificateKeyFile /etc/httpd/conf.d/carbonblack.key

   ##

   ## This the CA cert which can be used to validate the Sensor certificate.

   ## You can copy this certificate from any server in the Cb Response cluster member or standalone server.

   ##

   SSLCACertificateFile /etc/httpd/conf.d/cb-master-ca.crt

   SSLVerifyClient optional

   SSLVerifyDepth 2

   SSLOptions +ExportCertData +FakeBasicAuth +StdEnvVars

   ##

   ## This is the logs configuration of apache.

   ## You can record the HTTP/HTTPS parameters submit from the sensor.

   ## It is useful when you do trobleshooting.

   ##

   ErrorLog logs/cbr_ssl_error_log

   TransferLog logs/cbr_ssl_access_log

   LogLevel warn

   CustomLog logs/cbr_ssl_request_log \

             "%t %h %{SSL_PROTOCOL}x %{SSL_CIPHER}x SN %{SSL_CLIENT_M_SERIAL}x \"%r\" %b"

   ##

   ## This parts is the reverse proxy settings

   ## Replace the following IP address with your server IP.

   ## The Request is terminate in the Apache. Then the Apaache will create a new request to the backend server.

   ## The last configuration key will inject a custom header into the Cb Response Request.

   ## You can inject other header which is supported / required by the Cb Response.

   ##

   ProxyPreserveHost On

   ProxyPass "/"  "https://10.10.50.241/"

   ProxyPassReverse "/"  "https://10.10.50.241/"

   RequestHeader set X-Client-Cert-ID "%{SSL_CLIENT_M_SERIAL}s"

</VirtualHost>                                 

<VirtualHost *:443>

   ServerName cbrm1.carbonblack.local

   ServerAlias cbr-minion-1.carbonblack.local

   SSLProxyEngine on

   SSLEngine on

   SSLCertificateFile /etc/httpd/conf.d/carbonblack.crt

   SSLCertificateKeyFile /etc/httpd/conf.d/carbonblack.key

   SSLCACertificateFile /etc/httpd/conf.d/cb-master-ca.crt

   SSLVerifyClient optional

   SSLVerifyDepth 2

   SSLOptions +ExportCertData +FakeBasicAuth +StdEnvVars

   ErrorLog logs/cbr_ssl_error_log

   TransferLog logs/cbr_ssl_access_log

   LogLevel warn

   CustomLog logs/cbr_ssl_request_log \

             "%t %h %{SSL_PROTOCOL}x %{SSL_CIPHER}x %{X-Client-Cert-ID}s %{SSL_CLIENT_M_SERIAL}x\"%r\" %b"

   ProxyPreserveHost On

   ProxyPass "/"  "https://10.10.50.242/"

   ProxyPassReverse "/"  "https://10.10.50.242/"

   RequestHeader set X-Client-Cert-ID "%{SSL_CLIENT_M_SERIAL}s"

</VirtualHost> 

The IP address shown in the above example is my LAB's IP.

Please replace the IP address with your production IP in the above examples.

Question please: how are the sensors pointed to the reverse proxy? I assume the sensors are initiating the connection back to the CB R server through reverse proxy. I don't see any configuration examples for the sensors. is it done in the sensor config or at the o/s level?

thanks

The proxy based on the request URL to determined and forward the traffic to the back end server.

Therefore, it is transparent to the sensor. No need to change the settings of the sensor.

I think what you're saying is reverse proxy between the sensor and the CB R Server is unsupported.

Only web traffic from from the CB R UI users to the CB R Server is supported.

Got it, thanks.

The sensor can send the data back to minion server through the reverse proxy. No only web traffic.

if the sensor can use the reverse proxy please explain the configuration for the sensor. There must be some sort of configuration or the sensor would simply send to the assigned CB server for it's sensor group, yes?

My setup is one Master and Two Minion servers. The master is dedicated. In the sensor group setting, the URL is pointing to the Master. 

When the sensor register to the CBR, the master will assigned an Minion server to the sensor. When the sensor check-in to the master next time, the sensor will check-in the master first and then upload the data to the minion server. Once the sensor assigned to minion server, it won’t change until you change it manually.

So there is nothing special in the sensor group settings / sensor.


The sensors communicate with the Master and the minion servers with HTTPS port. Normally, you required a public IP for each servers. With the Reverse Proxy, you can use one IP address to redirect the traffic to the backend servers.

When you migrate the existing sensor URL to the reverse proxy, you just need to configure all the master and minion server URL in the reverse proxy and then update the DNS recorded of the master and minion URL to the reverse proxy server IP address. Also, no need to change any settings on your Sensors or the sensor groups.



ok so that was the piece I was missing - when you configure the CB Server url in the sensor group you are using the reverse proxy IP/hostname, not the minion ip/hostname.

Thanks.

so the traffic goes: sensor > reverse proxy > CB server/minion.

Hi guys,

And thanks for sharing the config yves_lam. 

Are you guys all using the self signed certificates created by CB during the install?  And do you do reverse-proxying for _all_ sensors, or just a subset of clients? 

I've set up a test environment using the self signed certificates. However, due to the CA cert being seen as "invalid" by Apache, it fails to authenticate the client.

Any specific configuration options that's needed in order to get this to work?

$ openssl verify -verbose -CAfile /etc/ssl/certs/cb-client-ca.crt extracted-client-sensor-cert.crt
CN = server
error 24 at 1 depth lookup: invalid CA certificate
error client-sensor-test-cert.crt: verification failed

From apache error log:

"(..) Certificate Verification: Error (24): invalid CA certificate [subject: (..)"
..
"(..) SSL routines:tls_process_client_certificate:certificate verify failed (..)"

 

Anonymous

Hello,

I am using new self-signed cert and my CBR is on port 8443. I am trying to configure Apache reverse proxy:

on cb.conf my reverse proxy IP is 

ReverseProxyIP="::ffff:192.168.230.203"

I have also tried EnforceClientCerts=False

Here is my Virtual Host config:

<VirtualHost *:443>

 

   ServerName cb.testdomain.com:8443

SSLProxyEngine on

SSLProxyVerify none

#SSLProxyCheckPeerCN off

#SSLProxyCheckPeerName off

#SSLProxyCheckPeerExpire off

 

ProxyRequests Off

<Proxy *>

Require all granted

</Proxy>

   

   SSLEngine on

   SSLCertificateFile /etc/httpd/conf.d/cbserver.crt

   SSLCertificateKeyFile /etc/httpd/conf.d/cbserver.key

   SSLCACertificateFile /etc/httpd/conf.d/cb-client-ca.crt

   SSLVerifyClient optional

   SSLVerifyDepth 2

   SSLOptions +ExportCertData +FakeBasicAuth +StdEnvVars

   RequestHeader set X-Client-Cert-ID "%{SSL_CLIENT_M_SERIAL}s"

   ErrorLog logs/cbr_ssl_error_log

  # TransferLog logs/cbr1_ssl_access_log

#CustomLog logs/cbr_ssl_request_log \

#"%t %h %{SSL_PROTOCOL}x %{SSL_CIPHER}x %{X-Client-Cert-ID}s %{SSL_CLIENT_M_SERIAL}x\"%r\" %b"

   LogLevel debug

   ProxyPreserveHost On

   ProxyPass "/"  "https://192.168.230.40:8443"

   ProxyPassReverse "/"  "https://192.168.230.40:8443" 

</VirtualHost>

Hi @Anonymous

What errors do you get? 

I would change SSLVerifyDepth 2 to SSLVerifyDepth 1 (since you are using self signed certs). I would also change SSLVerifyClient optional to SSLVerifyClient require, and anble EnforceClientCerts=True (cb.conf)

Other than that, I got my setup working with this virtualhost setup (one for headend and each minion):

 

<VirtualHost [Reverse Proxy IP]:443>

ServerName [Cluster minion / headend FQDN]

SSLProxyEngine on
SSLEngine on
SSLProxyVerify none
SSLProxyCheckPeerCN off
SSLProxyCheckPeerName off

SSLProtocol -all +TLSv1.2
SSLCipherSuite HIGH:!aNULL:!MD5:!SEED:!IDEA:!eNULL:!DES:!3DES

SSLCertificateFile /etc/ssl/certs/cb-server.crt
SSLCertificateKeyFile /etc/pki/tls/private/cb-server.key
SSLCACertificateFile /etc/ssl/certs/cb-client-ca.crt

SSLVerifyClient require
SSLVerifyDepth 1
SSLOptions +ExportCertData +FakeBasicAuth +StdEnvVars

ErrorLog logs/cbr_ssl_error_log
TransferLog logs/cbr_ssl_access_log
LogLevel error
CustomLog logs/cbr_ssl_request_log "%t %h %{SSL_PROTOCOL}x %{SSL_CIPHER}x SN %{SSL_CLIENT_M_SERIAL}x [HEADEND/MINION FQDN] \"%r\" %b"


ProxyPreserveHost On

ProxyPass /sensor/cblr https://[Minon/headend IP]/sensor/cblr
ProxyPassReverse /sensor/cblr https://[Minon/headend IP]/sensor/cblr

ProxyPass /sensor https://[Minon/headend IP]/sensor
ProxyPassReverse /sensor https://[Minon/headend IP]/sensor

ProxyPass /data https://[Minon/headend IP]/data
ProxyPassReverse /data https://[Minon/headend IP]/data


RequestHeader set X-Client-Cert-ID "%{SSL_CLIENT_M_SERIAL}s"

</VirtualHost>

Article Information
Author:
Creation Date:
‎09-25-2015
Views:
15893
Contributors