Rebooting a virtual machine

Rebooting a virtual machine

If you run KVM virtual machines (VMs) on the SC//Reliantâ„¢ platform, you can reboot an individual VM without restarting the physical node that hosts it. This page describes what a reboot does, the syntax used to request one, and two complete procedures for rebooting a VM.

Overview

Rebooting a VM is the equivalent of choosing Restart inside the guest operating system. The reboot is scoped to that VM alone, so the host operating system, the SC//Reliant Virtual Controller (RVC), and the other virtual nodes on the same host are unaffected.

Graceful reboot compared to a forced reboot

There are two ways to reboot a VM, and the difference matters because one can lose data.

Reboot typeWhat happensWhen to use it
Graceful (default)An Advanced Configuration and Power Interface (ACPI) restart signal is sent to the guest operating system. The guest shuts its applications down cleanly, flushes data to disk, and then restarts.Always try this first.
ForcedThe VM is reset immediately, without asking the guest operating system to shut down. This is the equivalent of holding the power button on a physical machine. Unsaved data in the guest can be lost, and the guest filesystem can be left needing a repair on the next start.Only when the guest operating system is unresponsive and a graceful reboot has already failed.

A graceful reboot depends on the guest operating system responding to the restart signal. If the guest has hung, the signal is ignored and the VM stays up. That is the situation a forced reboot is for.

Virtual machines inside a cluster

If the VM is managed by the cluster, the clustering software monitors it and can trigger a failover when it sees the VM stop. Both procedures on this page issue the reboot through RVC, which coordinates with the clustering software on your behalf, so a normal reboot of a cluster-managed VM does not require you to turn on maintenance mode first. The In Cluster column in the VM list tells you whether a given VM is cluster managed.

If you are making a change that stops or modifies the VM resource beyond a reboot, use maintenance mode. For more information, see Operating the cluster.

Choosing a method

Both methods perform the same underlying RVC action, kvm.reboot. The difference is where you run it from.

MethodRun fromBest forRequires
SSH and the RVC CLIA terminal on your own computer, connected to the nodeInteractive, one-off work where you want to see the result immediatelyShell access and an SSH key on your user account
The SC//Reliant APIAny machine that can reach the APIScripting, automation, and rebooting the same VM across many nodes at onceAPI access, the Remote Job Execution feature, and RVC job capabilities

Method 1: Rebooting a VM by using SSH and the RVC CLI

In this method you connect to the physical node over SSH, then run the RVC CLI on that node.

For the full command syntax, arguments, and the --json and --dry-run flags available on every RVC command, see kvm.reboot and the SC//Reliant Virtual Controller 3.x client reference.

Prerequisites for the RVC CLI

This method requires shell access to the node. The API prerequisites do not apply.

  • A terminal. On macOS or Linux, use Terminal. On Windows, use Windows Terminal, PowerShell, or PuTTY.
  • Shell access on your user account. By default, a new user does not have shell access. An administrator turns it on for your account.
  • An SSH key pair, with the public key added to your account. The platform requires an RSA 4096-bit key. For instructions on generating a key and adding it to your account, see Configuring shell access.
  • The hostname of the physical node that hosts the VM. Node hostnames follow the pattern sit0001red0, where 0001 is the site number. You can find node hostnames in the SC//Reliant Platform Manager at https://platform.reliant.io/.

Procedure

  1. Use SSH to connect to your node. Your key authenticates the connection, either from your local agent or from the identity file configured for the host.

    ssh <node_hostname>

    For example, ssh sit0001red0.

    Once connected, the command prompt changes to show the node's hostname. Every command you type from this point runs on the node, not on your own computer.

  2. Retrieve a list of your KVM virtual machines. You need the exact VM name for the reboot, and this list is where you get it.

    rvc kvm list

    For example:

     Id    Name                           State          In Cluster
    -----------------------------------------------------------------
     37    kvm1                           running        yes
     38    kvm2                           running        yes

    The columns are the following.

    ColumnDescription
    IdThe internal ID KVM assigned to the running VM. A VM that is shut off has no ID and shows a hyphen.
    NameThe VM name. This is the value to pass to --hostname.
    StateThe current state of the VM, for example running or shut off.
    In ClusterWhether the clustering software manages this VM.

    Confirm the VM you intend to reboot is listed and that its State is running. A VM that is shut off cannot be rebooted. Start it instead with rvc kvm start --hostname <kvm_hostname>.

  3. Reboot the desired KVM virtual machine by using the VM name from the previous step.

    rvc kvm reboot --hostname <kvm_hostname>

    For example, rvc kvm reboot --hostname kvm1.

    Response

    Rebooting kvm1...
    Reboot successfully triggered.

    Reboot successfully triggered means the restart signal was accepted by the guest operating system. It does not mean the VM has finished restarting. The guest shutdown and startup continue in the background.

  4. Verify the VM came back up. Wait for the guest operating system to finish starting, then check its state.

    rvc kvm status --hostname <kvm_hostname>

    Response

    running

    A running result means the VM is powered on. How long the reboot takes depends on the guest operating system and the applications inside it, so if the state is not yet running, wait and check again. Also confirm that the application inside the VM is reachable, because a VM can be running before its services have finished starting.

  5. If the VM did not reboot, and only after a graceful reboot has failed, force the reboot.

    rvc kvm reboot --hostname <kvm_hostname> --force

    A forced reboot resets the VM without a clean guest shutdown, so unsaved data in the guest can be lost.

  6. Close the SSH connection when you are finished.

    exit

For the full command reference, see kvm.reboot and kvm.list.

Method 2: Rebooting a VM by using the SC//Reliant API

In this method you never connect to the node. You send a request to the API, the platform creates an RVC job, and the job runs the same kvm.reboot action on the node for you. The call returns immediately with a job ID rather than the result, so each action below is two steps: create the job, then retrieve the job to see what the node returned.

The examples use api.rb, the Ruby helper script that the API reference uses throughout. The script handles request signing so that you do not have to build the authentication headers yourself.

For the full request body, parameters, and response fields, see Reboot a KVM virtual machine on a target.

Prerequisites for the API

This method requires API access and RVC job capabilities in the SC//Reliant Platform Manager. You do not need shell access or an SSH key to the node.

  • API access on your user account, with an authentication token and secret key. By default, a new user does not have API access. An administrator turns it on and provides you the token and secret key. For more information, see Updating a user and Authentication.
  • The api.rb script saved locally, with your token and secret key filled in. Copy the script from API examples using Ruby, save it as api.rb, and replace the YOUR_AUTH_TOKEN and YOUR_SECRET_KEY values.
  • Ruby installed on the machine where you run the script.
  • Your customer code, which is the short tenant identifier passed as the first argument to the script. Your administrator or Scale Computing Support can provide it.
  • The Remote Job Execution feature turned on at the site level.
  • The following capabilities on your user account. For more information, see Managing user permissions.
    • Create and Update a RVC job
    • Call KVM read-only methods, needed to list the VMs
    • Call KVM state altering methods, needed to reboot a VM

Every command below follows the same shape.

ruby api.rb <customer_code> <verb> <api_endpoint> '<json_body>'

Procedure

  1. Start a job to list the KVM virtual machines on the node. As with the CLI method, you need the exact VM name before you can reboot it.

    ruby api.rb <customer_code> POST api/v2/jobs/rvc '{"method":"kvm.list","hostnames":["sit0001red0"]}'

    Response

    {
      "id": 1597949,
      "type": "RVC3",
      "status": "pending",
      "message": null,
      "trigged_by": "test.user",
      "parameters": {
        "method": "kvm.list",
        "hostnames": [
          "sit0001red0"
        ]
      },
      "time_started": "2025-04-08T17:36:00Z"
    }

    The response confirms the job was created, not that it has finished. A status of pending means the job is scheduled to run. Record the id value, because that is how you retrieve the result.

  2. Retrieve the job result to see the list of VMs. Add include_payload=true so the response includes the output the node returned. The question mark is escaped with a backslash so that your shell does not interpret it.

    ruby api.rb <customer_code> GET api/jobs/1597949\?include_payload=true

    Response

    {
      "id": 1597949,
      "type": "RVC3",
      "status": "finished",
      "payload": {
        "sit0001red0": [
          {
            "result": [
              {
                "id": "1",
                "name": "kvm1",
                "status": "running",
                "cluster_managed": true
              },
              {
                "id": "2",
                "name": "kvm2",
                "status": "running",
                "cluster_managed": true
              }
            ],
            "timestamp": 1744133763
          }
        ]
      },
      "trigged_by": "test.user",
      "success_nodes_count": 1,
      "failed_nodes_count": 0,
      "total_nodes_count": 1,
      "time_started": "2025-04-08T17:36:00Z",
      "time_finished": "2025-04-08T17:36:02Z"
    }

    Read the results from the payload object, which is keyed by node hostname. Each VM entry gives its name, its status, and whether it is cluster_managed. Use the name value in the next step.

    If status is still pending or running, the job has not finished yet. Wait a few seconds and run the same command again.

  3. Start a job to reboot the VM. Set force to false for a graceful reboot.

    The request contains two different hostnames, and mixing them up is the most common mistake here. The hostname inside params is the name of the virtual machine to reboot, taken from the previous step. The value in hostnames is the name of the physical node that hosts it.

    ruby api.rb <customer_code> POST api/v2/jobs/rvc '{"method":"kvm.reboot","params":{"hostname":"kvm1","force":false},"hostnames":["sit0001red0"]}'

    Response

    {
      "id": 1598246,
      "type": "RVC3",
      "status": "pending",
      "message": null,
      "trigged_by": "test.user",
      "parameters": {
        "method": "kvm.reboot",
        "params": {
          "hostname": "kvm1",
          "force": false
        },
        "hostnames": [
          "sit0001red0"
        ]
      },
      "time_started": "2025-04-08T22:52:01Z"
    }

    Record the new job id.

  4. Retrieve the job result to confirm the reboot was triggered.

    ruby api.rb <customer_code> GET api/jobs/1598246\?include_payload=true

    Response

    {
      "id": 1598246,
      "type": "RVC3",
      "status": "finished",
      "message": null,
      "payload": {
        "sit0001red0": [
          {
            "result": "Rebooting kvm1...\nReboot successfully triggered.",
            "timestamp": 1744152723
          }
        ]
      },
      "trigged_by": "test.user",
      "success_nodes_count": 1,
      "failed_nodes_count": 0,
      "total_nodes_count": 1,
      "parameters": {
        "method": "kvm.reboot",
        "params": {
          "hostname": "kvm1",
          "force": false
        },
        "hostnames": [
          "sit0001red0"
        ]
      },
      "time_started": "2025-04-08T22:52:01Z",
      "time_finished": "2025-04-08T22:52:03Z"
    }

    Check two things. A status of finished with failed_nodes_count of 0 means the job ran successfully on the node. The result string inside payload is the same output the CLI prints, and Reboot successfully triggered. confirms the guest accepted the restart signal.

  5. Verify the VM came back up by starting a kvm.status job and retrieving its payload.

    ruby api.rb <customer_code> POST api/v2/jobs/rvc '{"method":"kvm.status","params":{"hostname":"kvm1"},"hostnames":["sit0001red0"]}'

    Then retrieve the payload for the returned job ID, as in the previous step. A result of running means the VM is powered on.

  6. If the VM did not reboot, and only after a graceful reboot has failed, send the same request with force set to true.

    ruby api.rb <customer_code> POST api/v2/jobs/rvc '{"method":"kvm.reboot","params":{"hostname":"kvm1","force":true},"hostnames":["sit0001red0"]}'

    A forced reboot resets the VM without a clean guest shutdown, so unsaved data in the guest can be lost.

Rebooting a VM across many nodes

Because a VM naming convention is often repeated across sites, you can reboot the same VM name on every node in a node group with a single request. Replace the hostnames array with node_group_name.

ruby api.rb <customer_code> POST api/v2/jobs/rvc '{"method":"kvm.reboot","params":{"hostname":"kvm1","force":false},"node_group_name":"Test Node Group"}'

The job runs on all nodes that are members of the node group, and the payload contains one entry per node. You can target a saved filter the same way with saved_filter_name or saved_filter_id. For more information, see Using saved filters.

Confirm the scope before you run a group-wide reboot, because the job restarts the named VM on every matching node.

Troubleshooting

SymptomLikely causeWhat to do
ssh: Could not resolve hostnameThe node hostname is mistyped, or your machine cannot reach the node's network.Verify the hostname in the SC//Reliant Platform Manager and confirm you are on a network that can reach the node.
Permission denied (publickey)Shell access is not turned on for your account, your public key is not on the account, or your private key is not loaded in your SSH agent.See Configuring shell access.
rvc: command not foundYou are on a machine that does not have RVC installed, or you are still on your own computer rather than the node.Confirm the prompt shows the node's hostname, then run rvc --version.
The VM is not in rvc kvm listThe VM name is different than expected, or the VM is on a different node.Compare the Name column against what you expected, and check the other nodes at the site.
The reboot reports success but the VM never returns to runningThe guest operating system is hung and ignored the restart signal, or the guest is slow to start.Wait longer, check again, then reboot with --force or "force":true.
The API returns an authorization errorAPI access, the Remote Job Execution feature, or the required capabilities are missing.Review the Prerequisites for the API.
The API job finishes with failed_nodes_count greater than 0RVC could not run the action on that node, for example because the node was offline or the VM name was wrong.Read the result value in the payload for that node's entry. For a list of API errors, see Common errors.