Deploying changes to your environment
Configuration changes you make in the SC//Reliantâ„¢ Platform Manager are not applied to your nodes at the moment you save them. The Platform Manager stores the desired configuration, and Puppet is what deploys it. The properties set at the site, node group, and node level are compiled into a Puppet catalog, and a Puppet agent on each node applies that catalog on the next Puppet run. Until Puppet runs on a node, that node continues running its previous configuration.
A change reaches a node in one of two ways. Either you wait for the next scheduled Puppet agent run, or you trigger a Puppet run yourself. This page covers triggering a run using the SC//Reliant API and the SC//Reliant Virtual Controller (RVC) CLI.
Automated Puppet runs are driven by a cron schedule that you control through node group properties, so the schedule varies from one environment to another. Do not assume a particular interval when planning a deployment. To find out when the next scheduled run is due on a node running RVC 3.13.0 or later, use rvc puppet status and read the Next Run At value. On earlier RVC versions, use rvc sys crontab to see the schedule. For more information about the settings that govern the schedule, see Configuring automated Puppet runs.
IMPORTANT: The procedures on this page show you how to run Puppet. They do not define a deployment strategy. Deciding which nodes to deploy to, in what order, and in what change window is specific to your environment and the change you are making. Deploying a configuration change to every node in a large estate at once is rarely what you want. Determine your target set and sequencing before you begin.
Before you deploy
Confirm the following before triggering a run.
- Your change is saved in the SC//Reliant Platform Manager. For node group changes, this means you completed the impact review and confirmation. For more information, see Updating a node group.
- You know which nodes the change affects. When you update a node group, the impact screen reports the number of affected nodes.
- Puppet is enabled on the target nodes. A node with Puppet disabled does not apply the catalog. To check, use
rvc puppet disabled?, which returnstruewhen Puppet is disabled. - No Puppet run is already in progress on the target nodes. With RVC
3.13.0or later, uservc puppet statusand confirmStateisstopped. On earlier versions,rvc puppet runrefuses to start while another run is in progress and exits with code1.
Previewing a change before you deploy it
Puppet can simulate a run without applying anything. This is called noop mode, and it reports what the run would change while leaving the node untouched. Use it to confirm a change does what you expect before you deploy it for real, particularly on the first node of a larger rollout.
Using the RVC CLI:
rvc puppet run --noop
Using the SC//Reliant API:
ruby api.rb <customer_code> post api/jobs/puppet '{"action":"runonce","noop":true,"hostnames":["<node_hostname>"]}'
A noop run is a safe way to verify targeting and catalog contents. It is not a guarantee, because a real run can still fail on a node whose live state differs from what the simulation assumed.
Deploying using the SC//Reliant API
The API creates a Puppet job that runs against the targets you specify. This is the method to use when deploying to more than one node, because a single request can target a node group or a saved filter rather than a list of hostnames. Puppet runs are started through the Jobs v1 Puppet endpoint, api/jobs/puppet, with the runonce action. The Jobs v2 RVC endpoint does not run Puppet.
In order to perform this action, the following conditions must be met:
- Must have the
Remote Job Executionfeature turned on for your SC//Reliant Platform Manager tenant. - The user making the API request must have the
Create and Update a Puppet Run jobcapability.
Deploying to specific nodes
Target one or more nodes by hostname.
ruby api.rb <customer_code> post api/jobs/puppet '{"action":"runonce","hostnames":["<node_hostname>"]}'
Deploying to a node group
Target a node group to run Puppet on all of its member nodes. This is the natural counterpart to updating a node group, because the target set matches the set of nodes the change affects.
ruby api.rb <customer_code> post api/jobs/puppet '{"action":"runonce","node_group_name":"<node_group_name>"}'
Deploying to a saved filter
Target a saved filter to run Puppet on all nodes that match it. Use this when your target set does not correspond to a single node group.
ruby api.rb <customer_code> post api/jobs/puppet '{"action":"runonce","saved_filter_name":"<saved_filter_name>"}'
Retrieving the result
The API response confirms the job was created and scheduled, not that Puppet finished. Use the job ID from the response to retrieve the result.
ruby api.rb <customer_code> get api/jobs/<job_id>\?include_payload=true
The payload confirms that a Puppet run was started on each node. It does not report the outcome of the run. To check the outcome, see Verifying a deployment, or start a job with the last_run_summary action. For more information, see Retrieve a Puppet last run summary.
For the full request and response reference, including the environment parameter, see Run the Puppet agent once.
Deploying using the RVC CLI
Running Puppet from the RVC CLI applies the catalog on the node you are connected to. Use this method when you are deploying to a single node, when you are working through a rollout node by node, or when you are troubleshooting a change that did not apply as expected.
- Connect to the node.
- Confirm Puppet is enabled and no run is in progress. This command requires RVC
3.13.0or later. On earlier versions, uservc puppet disabled?instead.
rvc puppet status
- Run the Puppet agent.
rvc puppet run
- Confirm the run completed and review the result. On RVC versions earlier than
3.13.0, review the output ofrvc puppet runinstead.
rvc puppet status
The following options are available on rvc puppet run and are useful during a deployment.
| Option | Description |
|---|---|
--noop | Simulates the run without applying changes. See Previewing a change before you deploy it. |
--environment | Runs against a different Puppet environment than the one set in the SC//Reliant Platform Manager. Useful for testing a change from a feature branch on a single node before merging it. |
--random-wait | Delays the start of the run by a random amount of time, up to the given number of seconds. If you leave out this option, the run starts immediately. A value of 0 doesn't mean no delay: RVC applies its default random wait of up to 300 seconds instead. To start a run with effectively no delay, leave out the option or use 1. |
--force | Runs Puppet even when it is disabled. Puppet is enabled, the run is started, and Puppet is disabled again afterward. Use this deliberately, since Puppet is usually disabled for a reason. |
--cluster | Coordinates the run across a cluster. Accepted values are none, primary, and secondary. Requires RVC 3.13.0 or later. |
With RVC 3.13.0 or later, Puppet writes to /var/log/puppetlabs/current_execution.log while a run is in progress. When the run completes, the file is renamed to /var/log/puppetlabs/last_execution.log. Check these logs when a deployment does not produce the result you expected. Earlier RVC versions don't write these files, so review the output of rvc puppet run or the RVC server journal with journalctl -u rvc-server.
To stop a run that is in progress, use rvc puppet stop.
For the full command reference, including all arguments, see the rvc puppet pages in the RVC client reference.
Configuring automated Puppet runs
Between the on-demand runs you trigger yourself, the Puppet agent runs automatically on a cron schedule. The properties that control the schedule, and how to check the effective schedule on a node, are described in Configuring automated Puppet runs.
Deciding whether you need maintenance mode or standby
A Puppet run to deploy general configuration changes usually does not require maintenance mode, and it usually does not require placing a node on standby either. Both controls exist to stop Pacemaker from reacting to something you are doing deliberately, and a routine configuration deployment does not normally provoke a reaction worth suppressing. For the full explanation of both controls and how they interact, see Operating the cluster.
Consider whether a control is warranted when your change touches resources that Pacemaker manages. Node group properties such as Cluster KVM Resources, Cluster Docker Resources, Cluster Mount Points, Cluster Virtual IPs, and Cluster Interfaces define cluster-managed resources. When Puppet reconfigures one of these, Pacemaker may see the resource stop and restart, and it may respond by migrating workloads or, in the case of a monitored interface, by acting on what looks like a failure.
The two controls address different situations.
| Control | When it is worth considering during a deployment |
|---|---|
| Maintenance mode | Your change reconfigures cluster-managed resources and you do not want Pacemaker to fail workloads over while the change is applied. Maintenance mode freezes resource management cluster-wide, so workloads stay where they are. Remember to turn it off afterward, because failover stays disabled while it is on. |
| Standby | You want a node emptied before the change is applied to it, such as a rolling deployment where you move workloads off one node, deploy and verify, then return it to service. Standby requires that the cluster is not in maintenance mode, because it relies on Pacemaker to migrate the resources. |
Do not combine the two expecting both effects. Placing a node on standby while the cluster is in maintenance mode marks the node standby but leaves its resources stranded on it, which can be mistaken for a completed evacuation. See Operating the cluster.
If you are unsure whether a specific change warrants maintenance mode, contact SC//Reliant Support or your account manager before deploying.
Verifying a deployment
Verify the outcome on the nodes rather than assuming the run succeeded.
- Confirm the run finished. With RVC
3.13.0or later,Stateshould bestoppedandLast Run Atshould reflect your run. On earlier versions, the output ofrvc puppet runreports the result.
rvc puppet status
- Confirm the change itself is present on the node. What you check depends on the change, such as the state of a container, a virtual machine definition, or the contents of a managed file.
- If the change affected a cluster, confirm the cluster is healthy before considering the deployment complete. For more information, see Verifying cluster health.
If a node was offline or unreachable during the deployment, no action is needed to retry it. Puppet enforces the desired state on every run, so the node applies the change on its first successful run after it returns.
Related documentation
- Updating a node group: making the configuration change that this page deploys.
- Operating the cluster: maintenance mode, standby, and how they interact.
- Performing maintenance on a node: taking a single node out of service, including the standby procedure.
- Verifying cluster health: the service checks to review after a deployment.
- Understanding configuration data: the property hierarchy that determines the effective configuration and Puppet run schedule for each node.
- Configuring automated Puppet runs: the properties that schedule automated Puppet runs.
- Transitioning from Puppet Launcher to RVC Puppet: differences between RVC versions and the scheduled run methods.
- Puppet: what Puppet is and its role in the platform.