Splunk Uses Puppet to Automate Cloud Provisioning, Scale Management + Create Happier Customers

Challenge: Scaling a Cloud Environment at the Speed of Business

Splunk® Cloud enables customers to quickly leverage Splunk capabilities without having to manage the underlying infrastructure or software. To scale their cloud environment at the rate customers’ businesses demand, the Splunk Cloud operations team needed to move to a deterministic, configuration-driven automation model that removed inconsistency and drift.  

Splunk started with agentless provisioning tools, but the company quickly realized it was critical to move to an agent-based automated process to better manage the company’s growing customer environments.

“To serve our customers better, we needed to have environments under constant management, rather than being managed on an ad hoc basis by provisioning tools,” said Chris Vervais, Director Site Reliability Engineering at Splunk. “Most of the work happens after provisioning, and once that is solved, the provisioning problem becomes much smaller.”

Results: Consistent, Predictable Cloud Environments at Scale

With the move to Puppet Enterprise, the Splunk team efficiently managed more customer environments and cut down provisioning time from days to minutes. Splunk was able to roll out a global change in a couple of hours, where before it would have taken several weeks of planning and a huge chunk of company resources.

"We want to make sure we scale efficiently as we scale our customer base – with Puppet, we can do that,” said Vervais. “We can focus on the larger picture and drive more scalability. We’re not mired down in toil work doing rote, repetitive, error-prone tasks. We’re focused on both the high-value problems we solve at Splunk and [the ones] we solve for our customers."

With this shift, Splunk quickly scaled consistent management to thousands of nodes, shrinking maintenance windows and helping customers scale even faster. By automating its customer cloud environments, Splunk also cut down on error rates associated with rote, repetitive tasks with a two‐person pull request model.

“Puppet helped us achieve automation at scale by managing the environment, preventing drift, and making changes easy to deploy across the entire fleet regardless of size. It keeps the environment consistent and in the intended state.”

Chris Vervais, Director, Site Reliability Engineering at Splunk

Beyond the initial goal of scalability, Splunk also wanted to provide additional value to its customers through deterministic change. With Puppet Enterprise, Splunk has been able to effectively deliver change to environments, which decreases mean time to repair (MTTR) on a fleet‐wide basis. In fact, it now takes minutes to resolve configuration changes whereas the change could have taken hours previously.

“When you have mission critical use cases you just can’t let hours go by. Automating for common error scenarios really reinforces how this service can be used to address mission‐critical customer issues,” said Vervais.

With Puppet Enterprise, environments are more consistent and predictable. For instance, when Splunk introduced change across its cloud environments, it no longer had to test for a dozen different scenarios. Splunk has since reallocated its resources to focus less on updating its customer environments manually and instead has used this time to work on more high‐value issues, such as transparency and compliance for regulatory controls.

See for yourself what Puppet can do to scale your cloud environments. Try it today or request a demo with the Puppet team.

TRY PUPPET ENTERPRISE   DEMO PUPPET