HashiCorp, HashiDays

Oh Hashi Days! Yesterday I was delighted to attend the 2017 London HashiDay in Moorgate. This wonderfully organised and insightful day of “deeply technical” HashiCorp presentations from the likes of co-founder Mr Hashimoto and his customers, such as born-in-the-cloud finance disruptor Monzo, contained some very important messages for enterprise ears, which I hope to summarize in this blog.

Let’s get straight into the key take-away messages.

Individual Providers Can Release At Their Own Pace

As some readers may have noticed recently, HashiCorp tools are really starting to pick up pace with the functionality updates, especially Terraform, which now boasts capability across 75 different providers and encompasses around 500 resources. A key takeaway of yesterday's event was the separation of Terraform by versioned providers, allowing each individual provider to have its own release cadence. This is a huge benefit for organisations in need of particular, not-yet-included functionality. It means that matters can be taken into the hands of the users with greater impact: fork the repository, develop the functionality and pull-request back into the relevant Terraform provider repository and have a smoother process of getting the functionality integrated, with a lower bar to entry. It also means that certain hot and current providers can run away with capability and have frequent releases without affecting other providers. However, on the other hand, it could result in dependency management issues if particular versions of particular providers are required within an organisation’s implementation of Terraform. Therefore, implementation workflows should plan to keep up with the new releases at pace as part of the original solution. Anticipation of change is key. The way that the provider split will all be pulled together is via the ‘terraform init’ command, pulling modules, initialising the state storage backend (remote or local), and then pulling and initiating the relevant providers, recognised from the Terraform configuration files that are present - all held together using the core Terraform binary that all users will download.

Nomad Is Pushing the Boundaries and Nomad Enterprise Is Imminent

Armon Dadgar, co-founder of HashiCorp, pieced together something of a computer science lecture around workload schedulers when giving a talk around Nomad and next-generation application architectures. As some may know by now, utilising workload schedulers means that there is higher resource utilisation, a decoupling of work from resources, and generally a better quality of service.

HashiCorp have truly been pushing the limits of Nomad, reporting an experiment that involved running one million Redis containers in GCP using the tool. After hearing of this, leading global hedge fund investment company Citadel wanted to push the tool even closer to its limits (if it has any), stressing 4 million containers to test out Citadel workloads at scale, a test that was thrashed out across 18 thousand cores over 5 hours, averaging 2,200 containers per second. It’s safe to say that Citadel adopted Nomad post-experiment.

Mitchell Hashimoto, co-founder and CTO of HashiCorp, also stated that it continues to be the largest enterprises that tend to adopt Nomad first, most too big to name. Nomad Enterprise is also close to being released, providing governance and access control to rival the current commonly-present RedHat Openshift (sitting on Kubernetes under the covers). Given that rivals such as Kubernetes have a 120 thousand container limit, and the actual functionality of Nomad is closer to that of Google’s Borg project than people are aware of, all I can say is: watch this space!

On the topic of Nomad, Paddy Foran of HashiCorp also gave an interest talk yesterday around considering multi-cloud with Terraform and Nomad, a demonstration of orchestrating and managing a workload scheduler across multiple cloud-providers - run your containers anywhere and everywhere! The live demo consisted of creating a Nomad cluster across both AWS and GCP, raising the question of ‘why would you run containers in one cloud-provider when they can be ran anywhere?’.

Consul Functionality Is Maturing at Pace

A couple of points worthy of note are regarding Consul and Vault. Consul has been progressing with functionality rather nicely recently, focusing on operational interaction and maintainability. Consul Autopilot was introduced in v0.8, enabling dead-server cleanup, effective Consul server health checks out of the box and also the ability to introduce new Consul servers with assured stability. Consul also has capability to auto-join other Consul server members to form a cluster in both GCP and AWS, something that’s nice to plug into an auto-scaling group for resiliency.

Consul Enterprise adds further resilience to a Consul implementation, through the introduction of non-voting (in leader election) servers, serving as a backup server in an active/standby setup, and also the capability to enable automatic upgrades of the Consul binary within a long-running server cluster, allowing organisations to keep up with the HashiCorp update waves.

Vault Functionality Is More Accessible

There were a few interesting take-aways from the presentation on HashiCorp Vault, described by Vault development lead Jeff Mitchell as a ‘security swiss army knife’.

Vault is occasionally considered complex and overwhelming due to having many tools within a tool due to the numerous backends and capabilities, as well as generally dealing with encryption and secrets management. However, Jeff maintained that Vault allows a single workflow with a unified access mechanism, enabling applied security for ‘everyday survival in the datacentre’. As with all tools, how a tool is implemented and the workflows around it are what dictate success, or lack thereof.

Vault is also expected to add Oracle into its array, dynamically supported database backends, something that is commonly requested by our large enterprise customers to provide short-lived, revocable and machine-worthy database credentials, rather than having static credentials in application code, encrypted or not.

Enterprise Competition Is Gaining Ground

Some of the customer talks demonstrated the transformative ways in which digital disruptors are making use of HashiCorp technology to sustain digital innovation.

Monzo, a company that makes ‘banking for your smartphone’ and who have recently gained their licence declaring them a fully-fledged bank, demonstrated the effectiveness of their software delivery mindset from the word Go (please excuse the pun). Their mantra is ‘Rebuild everything: Go microservices, Git as Change management, Realtime and API’. 

If a banking-capable organisation can have that level of agility, large financial enterprises will have to catch up if they want to keep pace with the current state-of-play amongst their most threatening disruptive competition.

Elsevier also provided some battled-hardened Terraform wisdom, stating ‘everything we do in Terraform starts with a module’. For them, re-usability is key. This is a great message to evangelize as modular Terraform implementations continue to prove themselves as the most operationally effective solutions across all of our transformational engagements.

That’s all for this year's HashiDays London 2017 overview, but HashiDays NY and HashiConf, Texas have been confirmed to have even more fresh talks and insightful content. 

See you there!

  • Jordan Taylor

    DevOps Practitioner

    Jordan has a passion for being a central catalyst to organisational transformations; helping organisations achieve value through technological and process-related innovation.

    With strong understanding of the core discipline of automation, picking up any new technology at pace and delivering value with it is one of Jordan's prized skills. However, listening, understanding and interpreting conversations with clients form a combined skill that Jordan continues to demonstrate as the key to the success of any technological or organisational project delivery.

    More Articles by Jordan