Scripts to create a minimal mesos or nomad cluster on Azure using terraform.
Table of Contents
This creates 5
master and 10
agents in a mesos or nomad cluster. You can change the number of masters with MASTER_COUNT
and the number of agents with AGENT_COUNT
.
It also creates a "jumpbox" or "bastion host" since all the masters and agents are not publicly accessible.
If you want to ssh into the internal nodes you must first go through the bastion on node.
The username on the nodes is vmuser
.
The base image for all the virtual machines is CoreOS Container Linux.
The cloud-config.yml files defines the servers running on each of the hosts. The hosts are designed to be super minimal. This is done via the CoreOS Cloud Configuration.
All internal IPs are in the block 10.x.0.x.
These are multi-region so westus2
is 10.1.0.x and so on and so forth for the other regions.
So the first 5 in the block are the masters: 10.x.0.5-9
. And the agents follow after starting at 10.x.0.10
.
The IPs get populated programmatically in the terraform files. But they are also generated and substituted into the cloud-config*
files via the Makefile.
On each server there are helpers in the bashrc
so that you can run nomad
and consul
commands from the host. This just uses a container under the hood and you can see the alias with type nomad
.
Both consul and nomad are setup with TLS which is generated by the Makefile when the cluster is created.
On the bastion server we run:
On the masters we run:
On the agents we run:
On the bastion server we run:
On the masters we run:
On the agents we run:
Makefile
You will need to set the following environment variables:
AZURE_CLIENT_ID
AZURE_CLIENT_SECRET
AZURE_TENANT_ID
AZURE_SUBSCRIPTION_ID
See creating a service principal on how to get these values.
$ make help mesos-apply Run terraform apply for mesos. mesos-destroy Run terraform destroy for mesos. nomad-apply Run terraform apply for nomad. nomad-destroy Run terraform destroy for nomad. shellcheck Run shellcheck on all scripts in the repository. test Runs all the tests. update-terraform Update terraform binary locally from the docker container. update Run all update targets.
This is as simple as:
$ AZURE_CLIENT_ID=0000 AZURE_CLIENT_SECRET=0000 AZURE_TENANT_ID=0000 AZURE_SUBSCRIPTION_ID=0000 \ make az-apply
You need a service principal in order to use the Makefile
.
$ az ad sp create-for-rbac --role="Contributor" --scopes="/subscriptions/SUBSCRIPTION_ID"
The command will output the following:
{ "appId": "00000000-0000-0000-0000-000000000000", "displayName": "azure-cli-2017-06-05-10-41-15", "name": "http://azure-cli-2017-06-05-10-41-15", "password": "0000-0000-0000-0000-000000000000", "tenant": "00000000-0000-0000-0000-000000000000" }
These values map to the Makefile
variables like so:
appId
is the AZURE_CLIENT_ID
defined abovepassword
is the AZURE_CLIENT_SECRET
defined abovetenant
is the AZURE_TENANT_ID
defined aboveReference docs:
terraform
docs on setting up authentication: here.