PE release notes

These are the enhancements and resolved issues in this version of Puppet Enterprise (PE).

For security and vulnerability announcements, see Security: Puppet's Vulnerability Submission Process.

PE 2023.4

Released October 2023

Important: Puppet Enterprise (PE) 2023 is our leading-edge PE release stream (also referred to as STS). For important information about upgrading to 2023, see Upgrading Puppet Enterprise.

If you're on the LTS stream (2021.7), you'll find release notes and other information for that series in the 2021.7 documentation.

Customers on 2019.8.z are encouraged to upgrade to either 2021.7 or 2023.

CAUTION: PE 2023.4 includes several major component upgrades that introduce breaking changes. Before upgrading, carefully review all Upgrade cautions.

New features

PE certificate authority supports auto-renewal of agent certificates: If your installation includes puppet-agent 8.2.0 or a later version, PE is preconfigured to allow the certificate authority service to generate new agent certificates ahead of certificate expiration dates. This default functionality helps prevent disruption associated with certificate expirations. Optionally, you can turn off auto-renewal of agent certificates and customize your PE certificate authority settings.
Default timeout limits for deploy jobs: Timeout limits forcibly stop deploy jobs that run too long. This feature is useful for stopping jobs that are stuck, without requiring you to manually monitor the progress of jobs.
CAUTION: The feature for forcibly stopping deploy jobs can result in incomplete Puppet runs, partial configuration changes, and other issues. When setting timeout limits, consider the job scope, typical runtime, and your infrastructure's capacity (such as concurrency limits).; The default timeout limit is 30 minutes per node. You can change the global default limit by modifying the default_deploy_node_timeout setting in your Orchestrator and pe-orchestration-services parameters.
View and edit scheduled plans in the console: You can now view and edit scheduled plan details in the console.
View and edit scheduled jobs in the console: You can now view and edit scheduled job details in the console.

Enhancements

Puppet 8 is installed with PE 2023.4

When you install PE 2023.4, an upgraded version of Puppet is installed automatically. Puppet 8 includes several changes that can enhance PE performance capability. For example:

Starting in Puppet 8, legacy facts are replaced by structured facts.
Strict validation is enabled by default.
Ruby is upgraded to version 3.2.

Important: For information about these and other key changes in Puppet 8 that might affect your PE upgrade, see Puppet upgrade in 2023.4.

r10k upgrade

PE includes r10k version 4.0, which has been updated to enhance scalability, reduce dependency risks, and align with Git security best practices.

Important: To review information about changes introduced in r10k 4.0 that might affect your PE upgrade, see Upgrade cautions.

Task concurrency limit now pertains to individual tasks or plans

The task_concurrency setting defines the maximum number of task or plan actions that can be executed simultaneously.

Previously, the concurrency limit pertained globally to actions on any nodes in your installation that were targeted by any current task or plan jobs. Now, the concurrency limit is specific to the nodes targeted by individual task or plan jobs. This improvement significantly reduces latency when multiple task or plan jobs are run simultaneously.

For example, in a company that uses PE, four users each run a task job targeting 10 nodes. The four task jobs are similar in scope and the users initiate their jobs simultaneously. The task_concurrency parameter is set to 10. Previously, with a concurrency limit of 10, task actions would begin executing for one of the jobs and the three remaining jobs would be queued. Now in this scenario, the concurrency limit pertains to each job, so all 40 task actions are executed concurrently. Because the four task jobs are similar in scope, they can be expected to be completed in roughly the same timeframe.

The global default concurrency limit is 1000 actions per job. You can change the global default limit by modifying the task_concurrency parameter value in your Orchestrator and pe-orchestration-services parameters.

Enhanced workflow for configuring and running jobs in the console

The process of configuring and running jobs has been divided into three clear steps in the Jobs section of the console. You can now configure the job, use one of the three node-targeting methods, and review your setup before scheduling or running the job.

Classifier service automatically replaces legacy facts in node group rules

With the removal of legacy facts in Puppet 8, the PE classifier service now analyzes your node group rules and automatically replaces legacy facts with corresponding structured facts. If any of your node group rules contain legacy facts that cannot be directly mapped to structured facts, the classifier service generates warning messages in the logs, prompting you to manually remove or replace the unmappable legacy facts. For more information about the removal of legacy facts in Puppet 8, see Puppet upgrade in 2023.4.

PE installer flags unmappable legacy facts in node group rules

Because legacy facts are removed in Puppet 8, the PE installer now examines your existing node group rules and if any unmappable legacy facts are found, the installation process stops with a warning. To proceed with installation, you can replace or remove unmappable legacy facts and re-run the installer. For more information about the removal of legacy facts in Puppet 8, see Puppet upgrade in 2023.4

Session timeout warning in the PE console

Previously, whenever a console session timed out due to inactivity, users were logged out automatically and returned to the console login screen without warning. Now, whenever a session is about to expire due to inactivity, the console displays a warning modal to inform users they will be logged out soon. The warning modal includes an option to continue the session.

You can configure the behavior of the timeout modal using the following console service parameters:

puppet_enterprise::profile::console::session_timeout_polling_frequency_seconds
puppet_enterprise::profile::console::session_timeout_warning_seconds

Orchestrator HTTP-client limits can be configured to match infrastructure requirements

You can now specify HTTP-client connection limit parameters in the puppet_enterprise::profile::orchestrator class. You can set connection limits for authenticated and unauthenticated clients by specifying an integer value for the following parameters:

max_connections_per_route_authenticated
max_connections_total_authenticated
max_connections_per_route_unauthenticated
max_connections_total_unauthenticated

Orchestrator socket timeout is configurable

By default, whenever no data is available on the socket, the orchestrator waits for a maximum of 120,000 milliseconds before closing the HTTP connection. Now you can specify the maximum time before socket timeout by changing the default value of the socket_timeout parameter in the puppet_enterprise::profile::orchestrator class.

Enhanced logging of certificate authority actions

Previously, agent certificate requests were authorized using the ”pp_cli_auth”: “true” certificate extension. Now, when RBAC tokens are available, token-based authentication is used. This new default authorization method allows better auditability because user IDs that trigger certificate authority actions are reported to the audit log. If you want to configure the certificate authority service settings so that RBAC tokens are always required for authorization of agent certificate requests, you can set the value of allow_puppetlabs_certificate_authentication to false in your certificate_authority service parameters.

More efficient agent run reporting to conserve storage in PuppetDB

Previously, agent run reports submitted to PuppetDB contained significant amounts of data about unchanged managed resources. Now by default, to conserve storage space in PuppetDB, agent run reports only include data relating to changes enforced by the Puppet run. Data about the desired state of each managed resource is still available in agent catalogs. To revert to the previous behavior for agent run reporting, you can modify the puppet_enterprise::profile::agent::exclude_unchanged_resources parameter.

Improvements to error logging for the puppet backup command

Previously, error messages returned by the

puppet
                                backup

command were generic in many cases. Now, descriptive error messages are displayed both in the terminal and in the log file, and you can use a --debug flag with

puppet
                                backup

to extend error logging to all underlying Puppet commands.

Optimized translation of classifier rules in PuppetDB queries

Classifier rule translation has been optimized to produce better queries to PuppetDB when regular expressions are used in fact matching.

Restriction: This enhancement does not impact trusted facts, so suboptimal queries can still be produced when regular expressions are used against trusted facts.

Platform support

PE 2023.4 adds support for the following operating system platforms.

Added primary server platforms: Red Hat Enterprise Linux (RHEL) 9 x86_64; Ubuntu 22.04 amd64

Added agent platforms: macOS 13 ARM and x86_64

Added client tools platforms: macOS 13 ARM and x86_64

With this release, support was removed for several previously deprecated platforms. Before upgrading, review the following list of removed platforms and the important information in Platforms removed in 2023.0 and later.

Removed agent platforms: AIX 7.1; CentOS 6; CentOS 7 aarch64; macOS 10.15; Oracle Linux 6; Oracle Linux 7 aarch64; Red Hat Enterprise Linux (RHEL) 6; Red Hat Enterprise Linux (RHEL) 7 aarch64; Scientific Linux 6; Scientific Linux 7 aarch64; Solaris 10
Removed client tool platforms: CentOS 6; macOS 10.15; Oracle Linux 6; Red Hat Enterprise Linux (RHEL) 6; Scientific Linux 6

Deprecations and removals

Removed platforms: For information about platforms removed in this release, see the Platform support section.
Puppet 8 deprecations and removals: For information about deprecations and removals associated with the upgrade to Puppet 8, see Puppet upgrade in 2023.4.

Resolved issues

Installing Windows agent through the console no longer fails when option to test connections is selected: In PE 2021.2 and later, when installing Windows agents in the console’s Install agent on nodes screen, checking the Test Connections checkbox before clicking Add nodes caused the process to hang indefinitely. The issue is resolved in PE.
Security fixes: Addressed CVE-2023-5255

PE 2023.2

Released June 2023

Important: Puppet Enterprise (PE) 2023 is our leading-edge PE release stream (also referred to as STS). For important information about upgrading to 2023, see Upgrading Puppet Enterprise.

If you're on the LTS stream (2021.7), you'll find release notes and other information for that series in the 2021.7 documentation.

Customers on 2019.8.z are encouraged to upgrade to either 2021.7 or 2023.

Resolved issues

Security fix: Addressed CVE-2023-2530

PE 2023.1

Released May 2023

Important: Puppet Enterprise (PE) 2023 is our leading-edge PE release stream (also referred to as STS). For important information about upgrading to 2023, see Upgrading Puppet Enterprise.

If you're on the LTS stream (2021.7), you'll find release notes and other information for that series in the 2021.7 documentation.

Customers on 2019.8.z are encouraged to upgrade to either 2021.7 or 2023.

Enhancements

Improved performance when querying PuppetDB: This enhancement helps to improve performance for PuppetDB queries that contain large arrays, for example, if many nodes are enumerated or many terms are joined by a single "and" or "or" element.
Improved performance for the each, map, and filter functions in the Puppet language: Previously, the Puppet language built-in functions each, map, and filter showed poor performance and consumed unnecessary resources when run on JRuby software. The issue was resolved to enhance performance.
Puppet Server provides more reliable warnings when it cannot check for an update: By default, Puppet Server periodically checks whether a new version of Puppet Server is available. Previously, if Puppet Server could not connect to the update server, users were not provided with adequate information about the error. Starting with Puppet Server 7.10.1, a warning about the error is available in the log file.

Deprecations and removals

Deprecated PSON: In previous releases, Pure JavaScript Open Notation (PSON) was used in Puppet to serialize data for transmission.
PSON is deprecated in Puppet 7 and will be removed in Puppet 8.

Resolved issues

Tasks page is available following a software update

After upgrading PE from 2019.8 to 2021.7.1, the Tasks overview page in the PE console sometimes failed to load because of a timeout error. The issue is fixed in PE 2021.7.3 and 2023.1.

Scheduled task jobs run successfully without a defined timeout

In PE 2023.0, task jobs failed to start if they were scheduled without an explicitly defined timeout. In PE 2023.1, the issue is resolved to help ensure that task jobs start as scheduled even without an explicitly specified timeout option. If a timeout option is not explicitly defined, the default timeout for tasks is applied.

Timeout and concurrency values for scheduled tasks can be viewed and edited in the console

In PE 2023.0, the timeout and concurrency values for a scheduled task could not be viewed or edited in the PE console. This issue is fixed in PE 2023.1:

When you view a scheduled task in the console, any specified timeout and concurrency values are displayed in the new Timeout and Concurrency fields.
When you edit a scheduled task in the console, you can update the values in the new Timeout and Concurrency fields.
Any timeout or concurrency values that you specify for scheduled tasks will be applied.

When tasks are rerun in the console, timeout and concurrency attributes are preserved

In PE 2023.0, tasks that were rerun in the PE console did not properly preserve the concurrency and timeout attributes of the task job. This issue is fixed in PE 2023.1.

Access rights for remote users can be revoked and reinstated from the console

In PE 2023.0, a defect was introduced that prevented the revocation or restoration of some remote users by using the PE console. This issue is resolved in PE 2023.1.

Performance issue with Puppet agent runtimes is resolved

After an upgrade from PE 2019.8.12 to PE 2021.7.1, some users saw a significant increase in Puppet agent runtimes. The increase was caused by Facter 4, which was not using cached information to resolve facts. As a result, facts were resolved multiple times. The issue is now resolved to normalize the performance of the Puppet agent.

Enabling the lockless code deploy feature no longer causes performance issues in PuppetDB catalog compilation

When the versioned_deploys setting is enabled, Puppet previously reported the full directory path to the environment after resolving symbolic links as the source for resources in a catalog. Puppet now reports the path to the resource before resolving symbolic links in the environment path to help prevent instability of the PuppetDB instance.

Certificates and keys can be backed up and restored by specifying the certs scope

Previously, if you ran the

puppet-backup
                                create

command and specified a scope of certs, the command failed to back up the certificate authority root key and certificates. This issue occurred because Puppet 7 introduced a new default path for the certificate authority (CA) directory (/etc/puppetlabs/puppetserver/ca), but the puppet-backup create command failed to locate the new directory. Similarly, if you ran the puppet-backup restore command with a scope of certs, the restore operation failed. The CA directory issue is resolved so that backup and restore operations can run successfully.

Timeouts can be specified for SAML authentication

Previously, when users configured the PE console to specify session-timeout and session-maximum-lifetime values, the settings were applied to Lightweight Directory Access Protocol (LDAP) tokens and local login tokens. However, the specified settings were not applied to Security Assertion Markup Language (SAML) tokens, which are used for authentication with SAML identity providers. The issue is corrected to ensure that the specified settings also apply to SAML session lifetimes.

Updates implemented to help users enter valid URLs

In previous versions of PE, the role-based access control (RBAC) service permitted the entry of invalid URLs when users specified the Organizational URL setting. Login attempts would then fail with the following error message:

'Invalid settings: organization_not_enough_data'

In PE 2021.7.3 and 2023.1, the RBAC service is updated to enforce valid URLs when users create or update a connection to a SAML identity provider, and the PE console displays a warning if the user enters an invalid URL for the Organizational URL setting.

User-defined temporary directory is honored during PE restore operations

After you back up your PE infrastructure, you can use the puppet-backup restore command to restore the backup. Previously, if you set the —tmpdir flag or the TMPDIR environment variable to specify a temporary directory for restore operations, the directory was not honored, and the default /tmp directory was used in some cases. In addition, some files were not cleaned up after the restore operation. This issue is corrected to ensure that the user-specified directory is used, and all temporary files are removed after the restore operation.

Issue that caused an unexpected increase in CPU usage is resolved

In PE 2021.7.1, 2021.7.2, and 2023.0, an issue with Puppet Server caused an unexpected increase in central processing unit (CPU) usage in some environments. CPU usage continued to grow and some operations took longer than expected until the Puppet Server service was restarted. This issue is resolved in PE 2023.1 and 2021.7.3.

Security fixes

Addressed CVE-2023-1894 and CVE-2023-26048.

PE 2023.0

Released January 2023

Important: PE 2023 is our new leading-edge PE release stream (also referred to as STS). For important information about upgrading to 2023, see Upgrading Puppet Enterprise.

If you're on the LTS stream (2021.7), you'll find release notes and other information for that series in the 2021.7 documentation.

Customers on 2019.8.z, which is EOL, are encouraged to upgrade to either 2021.7 or 2023.

New features

Authenticate users in multiple LDAP domains

You can now connect multiple Lightweight Directory Access Protocol (LDAP) domains to PE. This new feature brings many changes to the role-based access control (RBAC) API and LDAP-related pages in the PE console.

In the PE console, view and manage all of your LDAP external directory service connections on the LDAP tab of the Access control page.

The Test connection button is removed. When you Connect to external directory services, the Connect button now automatically tests the connection before saving the configuration.

Use the Certificate chain field (or cert_chain API key) to define unique certificate chains across servers.

The following new endpoints replace deprecated or removed endpoints. For a list of deprecated and removed endpoints, refer to the Deprecations and removals section of these release notes.

Responses from these endpoints now include the identity_provider_id:

Default timeout limits for tasks and plans

Timeout limits forcibly stop tasks and plans that run too long. This feature is useful for stopping tasks and plans that are stuck without requiring you to manually monitor task or plan progress.

CAUTION: The feature for forcibly stopping tasks and plans can result in incomplete Puppet runs, partial configuration changes, and other issues. When setting timeout limits, consider the task or plan scope, typical runtime, and your infrastructure's capacity (such as concurrency limits).

The default timeout limits are 40 minutes for tasks (per node) and 60 minutes for plans (for the entire plan run). You can change the global default limits by modifying the default_task_node_timeout and default_plan_timeout settings in your Orchestrator and pe-orchestration-services parameters.

Alternatively, you can set timeout limits for an individual task or plan when Running tasks from the console, Running plans from the console, or running tasks and plans with the Orchestrator API.

You can use the timeout option with the following Orchestrator API endpoints:

Unique status for queued jobs

To better differentiate queued-but-unstarted jobs from jobs that are running, a new pending state was introduced for queued jobs.

The pending state is visible in the console and in responses from GET /plan_jobs and GET /plan_jobs/<job-id>.

View and edit scheduled tasks in the console

You can now view and edit scheduled task details in the console.

Enhancements

Java 17 upgrade: This version upgrades Java from version 11 to 17 and changes the default garbage collector from Parallel to G1.; Thoroughly test PE 2023.0 in a non-production environment before upgrading if you customized PE Java services or you use plug-ins that include Java code.
Stop in-progress plans in the console: When Running plans in PE, you can click Stop plan on the plan's run details page to stop the plan. In this way, you can prevent new tasks from starting and allow in-progress tasks to finish. To forcibly stop in-progress tasks from a stopped plan, follow the instructions in Stop a task in progress.
Forcibly stop in-progress tasks in the console: To Stop a task in progress, you can now both stop and forcibly stop in-progress tasks from the console. Previously, you had to use the Orchestrator API to forcibly stop tasks.; CAUTION: A forcible stop is the last resort when a task is stuck. This type of stop can result in incomplete Puppet runs, partial configuration changes, and other issues.
Provisioning replicas requires matching agent versions: When provisioning a replica, the target node's agent version must match the primary server's agent version. If the versions don't match, the puppet infra provision replica command fails before initializing the provisioning process. Previously, the agent version wasn't checked, and mismatched agent versions caused provisioning to fail partway through.
Increased task_concurrency limit: The default value of the task_concurrency orchestrator parameter was increased from 250 to 1000.
recover_configuration command recreates nodes files: Previously, the puppet infrastructure recover_configuration command merged new values into the nodes files (at /etc/puppetlabs/enterprise/conf.d/nodes) instead of overwriting the files. This process caused problems if you deleted a value relevant to one or more nodes, because the deleted value would remain in these files and continue to be applied.; Now, the recover_configuration command fully rewrites the nodes files on each invocation. This process matches how the command handles changes to the user_data.conf file.
Notification when session expires due to inactivity: PE redirects users to the login page when a session expires due to inactivity. When this happens, the login page now includes a message that indicates why the user was logged out.
Improved performance when regenerating agent certificates for multiple agents: The puppet infrastructure run regenerate_agent_certificate action is now faster when you Regenerate agent certificates for multiple agents. You can also now use the agent_pdb_query parameter to use a PDB query to generate a list of agents for which you want to regenerate certificates.; This action now uses the Puppet Server CA API endpoints directly, rather than relying on the puppetserver ca CLI, as it did previously. This process is faster, but, if you encounter problems, you can revert to the previous behavior by including use_puppetserver_cli=true in the command.
Specify Code Manager worker cache cleanup interval: The deploy_pool_cleanup_interval specified how often workers pause to clean their on-disk caches. Learn more about this setting in Code Manager parameters.

This release includes enhancements to cipher compatibility. For a complete list, go to Compatible ciphers.

CHACHA20 ciphers, compatible with non-FIPS PE installs: TLS_CHACHA20_POLY1305_SHA256 (TLSv1.3); TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (TLSv1.2); TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256 (TLSv1.2); TLS_DHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (TLSv1.2)
AES versions of two GCM ciphers, compatible with FIPS and non-FIPS installs: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 (TLSv1.2); TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 (TLSv1.2)
Removed restrictions: TLS_CHACHA20_POLY1305_SHA256 is no longer limited to Bolt server, ACE server, and NGINX.; ECDHE-ECDSA-CHACHA20-POLY1305 is no longer limited to NGINX.; ECDHE-RSA-CHACHA20-POLY1305 is no longer limited to NGINX.

Platform support

With this release, several previously deprecated platforms were removed. Before upgrading, review the important information provided in Platforms removed in 2023.0 and later.

Removed primary server platforms: CentOS 8
Removed agent platforms: CentOS 8; Debian 9; Fedora 32; Fedora 34; Ubuntu 16.04
Removed patch management platforms: Debian 9; Fedora 34

Deprecations and removals

Deprecated RBAC API endpoints: POST /v1/groups and POST /v2/groups are replaced by POST /command/groups/create.; PUT /v1/ds is replaced by POST /command/ldap/create, POST /command/ldap/update, and POST /command/ldap/delete.; GET /v2/ds is replaced by GET /ldap.; GET /ds/test and PUT /ds/test are replaced by POST /command/ldap/test.
Removed RBAC API endpoints: Removed the previously deprecated GET /v1/ds/, which is replaced by GET /ldap.
Removed platforms: For information about platforms removed in this release, see the Platform Support section.

Resolved issues

Code Manager respects full_deploy setting in Hiera

The full_deploy parameter is now correctly applied when you Customize Code Manager configuration in Hiera.

Previously, full_deploy was disregarded when included in your Code Manager configuration in Hiera. As a work-around, you could create a separate .conf file to manually manage this parameter.

Important: If you created a .conf file for the full_deploy parameter, you must remove this file and reconfigure the parameter in Hiera (as described in Configuring module deployment scope).

Certain plans correctly restore puppet service to pre-plan state

Due to a bug introduced in PE 2021.6, some plans that must stop the puppet service while the plans run were not restoring the puppet service to its pre-plan state after the plan finished running.

The four affected plans, and their associated puppet infra commands, are as follows:

The secondary_cert_regen plan, which is triggered by puppet infra run regenerate_compiler_certificate and puppet infra run regenerate_replica_certificate
The convert_legacy_compiler plan, which is triggered by puppet infra run convert_legacy_compiler
The reprovision_replica plan, which is triggered specifically by puppet infra upgrade replica --only-recreate-databases
The enable_ha_failover plan, which is triggered by puppet infra run enable_ha_failover

Important: If you were running PE 2021.6, 2021.7.0, or 2021.7.1 before upgrading to 2023.0, and you ran any of these four plans while running 2021.6, 2021.7.0, or 2021.7.1, check the state of the puppet service on your infrastructure nodes.

PuppetDB database user can purge reports

An issue was fixed to ensure that the PuppetDB database user can purge reports.

Corrected fact list handling in some PE console UI components

Some UI components in the PE console use fact lists. A recent change caused these component to use the entire list of fact names, which caused performance problems in environments with many facts. The handling of fact lists was corrected to fix this issue and improve performance.

Orchestrator code directories excluded from puppet-backup create --scope=config

When Customizing backup and restore scope, the orchestrator code directories (specifically /opt/puppetlabs/server/data/orchestration-services/data-dir and /opt/puppetlabs/server/data/orchestration-services/code) are excluded when you specify the config scope.

These directories are included in the code scope.

Plan action jobs have user data

Previously, jobs started as a result of plan action function didn't have an associated user stored in the database, which caused problems with some orchestrator commands. Now, user data is stored for these jobs.

Garbage collection log fixes

The introduction of Java 11 resulted in two issues relating to garbage collection logs. The issues are now fixed:

Dates and times are now included in garbage collection logs.

The maximum volume of retained garbage collection logs is 256 MB.

Security fixes

Addressed CVE-2022-41946 and CVE-2022-41404.

Was this page helpful?

We’re sorry to hear that!
Please tell us why so we can help. Enter your feedback and email. This form is sent to the Puppet docs team. We ask for your email as we might contact you regarding your feedback. If you need help with the product itself, visit Puppet Support or ask in Puppet Community on Slack. Feedback:

Email Address:

To learn about how Puppet uses your personal information, visit our privacy policy.

If you leave us your email, we may contact you regarding your feedback. For more information on how Puppet uses your personal information, see our privacy policy.

See an issue? Please file a JIRA ticket in our [DOCUMENTATION] project.