#Azure cloudapp ssl install#
#Azure cloudapp ssl code#
#Azure cloudapp ssl password#

# but if you are using an AD certificate authority this is where you would specify additional names # The -ext 'SAN=dns:$(hostname -f),ip:$(hostname -i) parameter is optional (our simple CA won't actually include these when signing) # Create a keystore and populate it with a new private key Now that I have a list of worker nodes, I can SSH from the head node to each of them and run the following: # Create a new directory 'ssl' and change into it Here, I've used jq to parse the API response and just show the nodes with that prefix.

HDInsight names all workers nodes with a 'wn' prefix. '.'Ĭurl -u admin:$password -sS -G ' | jq -r '.items | select(.Hosts.host_name |startswith('wn')).Hosts.host_name' # This will show a simple list of all nodesĬurl -u admin:$password -sS -G ' | jq '._name'

#Azure cloudapp ssl install#

Since I'll be working with the REST API, I'll install jq to help parse the responses. HDInsight provides the Ambari REST API by default, so I will use that to find the names of my worker nodes.

#Azure cloudapp ssl password#

I'll store my cluster name and password as variables first since I will be using them again. Now I can connect to each of my worker nodes, trust the new CA, and generate certificate requests. keytool -keystore -alias CARoot -import -file ca-cert -storepass '8t6q5>7BCAa!' -keypass '8t6q5>7BCAa!' -noprompt Whether you are using this simple CA or an enterprise CA, you will still want to store the CA certificate in a JKS trust store. This is important when configuring java applications to trust the new CA. This is a good time to store the CA certificate in a JKS trust store. The above command will create a certificate and key for signing certificate requests. Openssl req -new -newkey rsa:4096 -days 365 -x509 -subj '/CN=Kafka-Security-CA' -keyout ca-key -out ca-cert -nodes ssh Create a new directory 'ssl' and change into it In production, the CA would likely be provided by AD. For convenience, I'll use the first headnode in my cluster just like in the Microsoft tutorial. I will need a CA (Certificate Authority) to sign certificate requests and issue certificates for my Kafka Brokers.

If you already have a CA, or know how it works and want to follow the Microsoft tutorial that works just as well and you can skip right to the Kafka config and validation part. I've deployed a basic HDInsight cluster accepting defaults using the Azure Portal.

#Azure cloudapp ssl code#

That tutorial, for reference, is here:įor this post I'll focus on fixing the errors, share some extra details around creating a CA and certificates and demonstrate some code for validating the configuration.Īt a minimum, Kafka traffic should be encrypted. Microsoft provides a good tutorial for this purpose, but there are a couple of errors (as of this writing) and some of the options have changed. This post is intended to show minimal steps to encrypt Kafka traffic with SSL on HDInsight. The out of the box configuration does provide a dedicated VNET, which is good and the Enterprise Security Package offers a much more complete solution. The out of the box configuration doesn't provide much in the way of security though, and enabling SSL is a good first step. Kafka on HDInsight is an easy way to get started establishing a data integration layer and enabling analysis with modern tools. Besides offering simplified deployment, it also offers native integration with other Azure services like Data Lake Storage, CosmosDB and Data Factory. Azure HDInsight is a great way to get started with popular open source frameworks like Hadoop and Kafka.