Impact analysis limitations

Impact analysis has some technical limitations, and code changes can impact your infrastructure beyond what the impact analysis report displays.

Important: We've designed impact analysis to give you helpful information about the potential results of code changes to your infrastructure. However, impact analysis is not intended to be an exhaustive report — do not use it as a substitute for more exhaustive testing.

There are certain circumstances in which we can't reliably calculate the full impact of a code change. As an impact analysis user, it's critical that you understand these limitations exist, and you are aware of the possibility that code changes can have consequences for your infrastructure beyond what impact analysis shows.

This is a non-exhaustive list of some of the limitations of impact analysis. While this list might be updated periodically, it is never complete.
  • Changes to an environment's environment.conf file. Impact analysis can't determine the downstream impact of changes to an environment.conf file.
  • Brand-new code. Impact analysis can't determine the impact of new Puppet classes, facts, or functions that have never been applied to any of your nodes.
  • Changes to functions. The impact of changes to a function cannot be analyzed.
  • Changes to facts. The impact of changes to a fact cannot be analyzed.
  • Changes to class names used in classification. Changes to class names that were previously classified in the classifier cannot be analyzed. A classification update in the classifier is required before Puppet can locate the newly renamed class.
  • Imported resources. While impact analysis can provide information about nodes that export resources, impact analysis can't determine the impact to nodes that receive those exported resources.
  • Sensitive data types. Any changes to data types marked as sensitive are not be analyzed.
  • Alias metaparameter. Any resources using the alias metaparameter cannot be analyzed.
  • Changes to Embedded Puppet templates. The impact of changes made to Embedded Puppet (.epp) or Embedded Ruby (.erb) template files alone cannot be analyzed.
  • Patch fact generation script files. The pe_patch_fact_generation.ps1 and pe_patch_fact_generation.sh files are excluded from impact analysis reports because these files always report a change of n/a to n/a.
Remember: Impact analysis reports are provided as-is.