Lockless code deploys

The lockless code deploys feature within Code Manager allows deployment of Puppet code without interrupting other Puppet operations. When this feature is disabled, requests to Puppet Server are blocked during code deployments until the file sync client has finished updating the live Puppet code directory, However, when lockless code deploys are enabled, the file sync client saves newly deployed code into versioned directories, ensuring that the live code directory is not overwritten. This process allows Puppet operations to continue without interruption during code deployments.

With lockless code deploys, each new deploy writes code to versioned directories at: 
/opt/puppetlabs/server/data/puppetserver/filesync/client/versioned-dirs/puppet-code/

When the feature is enabled, the Puppet Server code directory is set up at /etc/puppetlabs/puppetserver/code and points, via symlink, to the most recent versioned code directory (at the versioned directories filepath specified above). If you disable lockless deploys, your code directory moves to /etc/puppetlabs/code.

Lockless deploys enable you to deploy a new version of code alongside an old version. When a catalog compiles starts, it uses the full path to the most recent version of code in the versioned code directory (via the /etc/puppetlabs/puppetserver/code symlink). Existing catalog compiles continue using the version they started on and new compiles use the latest code version.

To conserve disk space, code written to versioned directories is optimized to reduce duplication, and directories older than the latest and its predecessor are cleaned up after 30 minutes. If you deploy code very frequently, you might prefer to decrease the versioned-dirs-ttl setting, which is specified, in minutes, in file-sync.conf within each file sync client.

System requirements for lockless deploys

Enabling lockless deploys increases the disk storage required on your primary server and compilers because code is written to multiple versioned directories, instead of a single live code directory. Follow these guidelines for estimating your required system capacity.

You can roughly estimate your required disk storage with this equation: 
(Size of typical environment)×(Number of active environments)
For example, if your typical environment is 200 MB on disk when deployed, and you have 25 active environments, your disk storage calculation is 200 MB × 25, which equals 5,000 MB or 5 GB.
The number of times you deploy a given environment each day also impacts your disk use. Deploying multiple versions of the same environment uses approximately 25% more disk space than deploying multiple unique environments. To estimate the additional disk storage required for deploying environments multiple times a day, use this equation:
(Size of typical environment × .25)×(Number of environments deployed multiple times per day)×(Number of deployments per day)
Continuing the previous example, if 6 of your 200 MB environments are deployed up to 10 times per day, your additional disk storage calculation is (200 MB × .25) × 6 × 10, which equals 3,000 MB or 3 GB of additional disk space. In total, this example requires 8 GB available for your primary server and each compiler.
Note: If you're using the Continuous Delivery for PE impact analysis tool, you might need additional disk space beyond these estimates to accommodate the short-lived environments created during impact analysis.

Toggle lockless code deploys on or off

Use the toggle_lockless_deploys plan to turn lockless code deploys on or off across all compiler nodes, including the primary server and the replica.

Before you begin
For the toggle plan to run successfully, all compiler nodes (including the primary server and the replica) must have the same lockless code deploys status: the feature must be enabled on all or disabled on all. The plan cannot proceed if some nodes have the feature enabled while others have it disabled. To bring your compiler nodes into alignment, you can Enable or disable lockless code deploys on individual compiler nodes.
You can use the toggle_lockless_deploys plan by running these commands on the primary server:
  • To enable lockless code deploys:
    puppet infra run toggle_lockless_deploys enable=true
  • To disable lockless code deploys:
    puppet infra run toggle_lockless_deploys enable=false
    If you do not specify true or false, true is the default.
  • Optionally, you can use the old_code_directory parameter with either the save or delete values, to specify whether you want to save old code directories to ${codedir}_backup. For example:
    puppet infra run toggle_lockless_deploys enable=true old_code_directory=save
    If you do not specify the old_code_directory parameter, save is set by default.
Tip: You can optimize file sync performance for lockless code deploys by configuring the versioned_sync_pool and copy_method settings in the puppet_enterprise::profile::master::file_sync class. See Code Manager settings.

The toggle_lockless_deploys plan does not properly switch over a replica in disaster recovery to have lockless deploys because doing so requires updating Hiera data. Users who already have disaster recovery enabled and are toggling lockless deploys are recommended to update their pe.conf after running the plan.

While polling for changes, do not prematurely abort the toggle_lockless_deploys plan run if you see errors on the console. Wait until the plan fully completes before assessing whether the failures were an expected part of polling for changes or not.

Enable or disable lockless code deploys on individual compiler nodes

You can follow this process for individual compiler nodes to ensure that all your compiler nodes have the same lockless code deploy status, which is a prerequisite for running the toggle_lockless_deploys plan. Or you might want to enable lockless code deploys on an individual compiler for testing, before you enable the feature across all compiler nodes.
  1. In the compiler's node-specific Hiera file, set puppet_enterprise::profile::master::versioned_deploys to true to enable the feature, or false to disable it.
  2. Commit changes.
  3. Run Puppet on the compiler twice: puppet agent -t; puppet agent -t
  4. On your primary server, run: puppet code deploy --all --wait
    Important: You must deploy all environments (with --all) to avoid errors.
    Tip: To monitor the impact on a compiler after enabling lockless code deploys, you can analyze the Puppet Server data collected by the puppet_metrics_collector module.