Cloudera Enterprise 6.3.x | Other versions

Adding and Deleting Clusters

Minimum Required Role: Full Administrator

Cloudera Manager can manage multiple clusters. Furthermore, the clusters do not need to run the same major version of CDH. Starting with Cloudera Enterprise 6.2, you can create dedicated compute clusters with access to data in a base cluster. For more information, see Virtual Private Clusters and Cloudera SDX.

Continue reading:

Adding a Cluster Using New Hosts

On the Cloudera Manager Home page, click the Add drop-down button at the top right, or the Clusters drop-down button at the top left, and then click Add Cluster. This launches the Add Cluster - Installation wizard, which allows you to create either a regular cluster or a compute cluster .

You can also launch the wizard by selecting Add Compute Cluster from the drop-down menu next to the cluster name. Launching the wizard from there skips the Welcome page and restricts the wizard to creating only a compute cluster.

The following sections guide you through each page of the wizard:

Welcome (Add Cluster - Installation)

The Welcome page of the Add Cluster - Installation wizard provides a brief overview of the installation and configuration procedure, as well as some links to relevant documentation.

If you launched the wizard using the Add Compute Cluster option, this page is not displayed.

Click Continue to proceed with the installation.

Cluster Basics

The Cluster Basics page allows you to specify the Cluster Name and select the Cluster Type:
  • Regular Cluster: A Regular Cluster contains storage nodes, compute nodes, and other services such as metadata and security collocated in a single cluster.
  • Compute Cluster: A Compute Cluster consists of only compute nodes. To connect to existing storage, metadata or security services, you must first choose or create a Data Context on a Base Cluster.

If you are performing a new installation, Regular Cluster is the only option. You cannot add a compute cluster if you do not have an existing base cluster.

For more information on regular and compute clusters, and data contexts, see Virtual Private Clusters and Cloudera SDX.

If you are adding a compute cluster to an existing base cluster, click Choose Data Context... to create or select a Data Context.

After selecting a cluster type and data context (if applicable), enter a cluster name and then click Continue.

Setup Auto-TLS

  Important: Auto-TLS is only available with an Enterprise license.

The Setup Auto-TLS page provides instructions for initializing the certificate manager for auto-TLS if you have not done so already. If you already initialized the certificate manager in Step 3: Install Cloudera Manager Server, the wizard displays a message indicating that auto-TLS has been initialized. Click Continue to proceed with the installation.

If you have not already initialized the certificate manager, and you want to enable auto-TLS, follow the instructions provided on the page before continuing. When you reload the page as instructed, you are redirected to https://<server_host>:7183, and a security warning is displayed. You might need to indicate that you trust the certificate, or click to proceed to the Cloudera Manager Server host. You might also be required to log in again and re-complete the previous steps in the wizard.

For more information, see Configuring TLS Encryption for Cloudera Manager and CDH Using Auto-TLS .

If you do not want to enable auto-TLS at this time, click Continue to proceed.

Specify Hosts

  Note: This section covers the procedure for new hosts only. For existing managed hosts, see Adding a Cluster Using Currently Managed Hosts.

Choose which hosts will run CDH and other managed services.

  1. To enable Cloudera Manager to automatically discover hosts on which to install CDH and managed services, enter the cluster hostnames or IP addresses in the Hostnames field. You can specify hostname and IP address ranges as follows:
    Expansion Range Matching Hosts
    10.1.1.[1-4] 10.1.1.1, 10.1.1.2, 10.1.1.3, 10.1.1.4
    host[1-3].example.com host1.example.com, host2.example.com, host3.example.com
    host[07-10].example.com host07.example.com, host08.example.com, host09.example.com, host10.example.com
      Important: Unqualified hostnames (short names) must be unique in a Cloudera Manager instance. For example, you cannot have both host01.example.com and host01.standby.example.com managed by the same Cloudera Manager Server.

    You can specify multiple addresses and address ranges by separating them with commas, semicolons, tabs, or blank spaces, or by placing them on separate lines. Use this technique to make more specific searches instead of searching overly wide ranges. Only scans that reach hosts running SSH will be selected for inclusion in your cluster by default. You can enter an address range that spans over unused addresses and then clear the nonexistent hosts later in the procedure, but wider ranges require more time to scan.

  2. Click Search. If there are a large number of hosts on your cluster, wait a few moments to allow them to be discovered and shown in the wizard. If the search is taking too long, you can stop the scan by clicking Abort Scan. You can modify the search pattern and repeat the search as many times as you need until you see all of the expected hosts.
      Note: Cloudera Manager scans hosts by checking for network connectivity. If there are some hosts where you want to install services that are not shown in the list, make sure you have network connectivity between the Cloudera Manager Server host and those hosts, and that firewalls and SE Linux are not blocking access.
  3. Verify that the number of hosts shown matches the number of hosts where you want to install services. Clear host entries that do not exist or where you do not want to install services.
  4. Click Continue.

    The Select Repository screen displays.

Select Repository

  Important: You cannot install software using both parcels and packages in the same cluster.

The Select Repository page allows you to specify repositories for Cloudera Manager Agent and CDH and other software.

In the Cloudera Manager Agent section:

  1. Select either Public Cloudera Repository or Custom Repository for the Cloudera Manager Agent software.
  2. If you select Custom Repository, do not include the operating system-specific paths in the URL. For instructions on setting up a custom repository, see Configuring a Local Package Repository.

In the CDH and other software section:

  1. Select the repository type to use for the installation. In the Install Method section select one of the following:
    • Use Parcels (Recommended)

      A parcel is a binary distribution format containing the program files, along with additional metadata used by Cloudera Manager. Parcels are required for rolling upgrades. For more information, see Parcels.

    • Use Packages

      A package is a standard binary distribution format that contains compiled code and meta-information such as a package description, version, and dependencies. Packages are installed using your operating system package manager.

  2. Select the version of CDH to install. For compute clusters using parcels, the supported CDH versions display (Supported) next to the parcel name. For compute clusters using packages, you must make sure that you have installed a supported CDH version on all compute cluster hosts.
    1. If you selected Use Parcels and you do not see the version you want to install, click the More Options button to add the repository URL for your version. Repository URLs for CDH 6 version are documented in CDH 6 Download Information. After adding the repository, click Save Changes and wait a few seconds for the version to appear. If your Cloudera Manager host uses an HTTP proxy, click the Proxy Settings button to configure your proxy.
        Note: Cloudera Manager only displays CDH versions it can support. If an available CDH version is too new for your Cloudera Manager version, it is not displayed.
    2. If you selected Use Packages, and the version you want to install is not listed, you can select Custom Repository to specify a repository that contains the desired version. Repository URLs for CDH 6 version are documented in CDH 6 Download Information.
  3. If you selected Use Parcels, specify any Additional Parcels you want to install. If you are installing CDH 6, do not select the KAFKA, KUDU, or SPARK parcels, because they are included in CDH 6.
  4. Click Continue.

The Accept JDK License page displays.

Accept JDK License

  Note: Cloudera, Inc. acquired Oracle JDK software under the Oracle Binary Code License Agreement. Pursuant to Item D(v)(a) of the SUPPLEMENTAL LICENSE TERMS of the Oracle Binary Code License Agreement, use of JDK software is governed by the terms of the Oracle Binary Code License Agreement. By installing the JDK software, you agree to be bound by these terms. If you do not wish to be bound by these terms, then do not install the Oracle JDK.

To allow Cloudera Manager to automatically install the Oracle JDK on cluster hosts, read the JDK license and check the box labeled Install Oracle Java SE Development Kit (JDK8) if you accept the terms. If you installed your own Oracle JDK version in Step 2: Install Java Development Kit, leave the box unchecked.

If you allow Cloudera Manager to install the JDK, a second checkbox appears, labeled Install Java Unlimited Strength Encryption Policy Files. These policy files are required to enable AES-256 encryption in JDK versions lower than 1.8u161. JDK 1.8u161 and higher enable unlimited strength encryption by default, and do not require policy files.

After reading the license terms and checking the applicable boxes, click Continue.

Enter Login Credentials

  1. Select root for the root account, or select Another user and enter the username for an account that has password-less sudo privileges.
  2. Select an authentication method:
    • If you choose password authentication, enter and confirm the password.
    • If you choose public-key authentication, provide a passphrase and path to the required key files.

    You can modify the default SSH port if necessary.

  3. Specify the maximum number of host installations to run at once. The default and recommended value is 10. You can adjust this based on your network capacity.
  4. Click Continue.

The Install Agents page displays.

Install Agents

The Install Agents page displays the progress of the installation. You can click on the Details link for any host to view the installation log. If the installation is stalled, you can click the Abort Installation button to cancel the installation and then view the installation logs to troubleshoot the problem.

If the installation fails on any hosts, you can click the Retry Failed Hosts to retry all failed hosts, or you can click the Retry link on a specific host.

If you selected the option to manually install agents, see Manually Install Cloudera Manager Agent Packages for the procedure and then continue with the next steps on this page.

After installing the Cloudera Manager Agent on all hosts, click Continue.

If you are using parcels, the Install Parcels page displays. If you chose to install using packages, the Detecting CDH Versions page displays.

Install Parcels

If you selected parcels for the installation method, the Install Parcels page reports the installation progress of the parcels you selected earlier. After the parcels are downloaded, progress bars appear representing each cluster host. You can click on an individual progress bar for details about that host.

After the installation is complete, click Continue.

The Inspect Cluster page displays.

Detecting CDH Versions (Package Install Only)

If you are installing CDH using packages instead of parcels, theDetecting CDH Versions page validates that the correct versions of the CDH components are installed. The packages must be installed before proceeding. If any issues are reported, correct them, and then click Check Again.

After the validation succeeds, click Continue.

Inspect Cluster

The Inspect Cluster page provides a tool for inspecting network performance as well as the Host Inspector to search for common configuration problems. Cloudera recommends that you run the inspectors sequentially:

  1. Run the Inspect Network Performance tool. You can click Advanced Options to customize some ping parameters.
  2. After the network inspector completes, click Show Inspector Results to view the results in a new tab.
  3. Address any reported issues, and click Run Again (if applicable).
  4. Click Inspect Hosts to run the Host Inspector utility.
  5. After the host inspector completes, click Show Inspector Results to view the results in a new tab.
  6. Address any reported issues, and click Run Again (if applicable).

If the reported issues cannot be resolved in a timely manner, and you want to abandon the cluster creation wizard to address them, select the radio button labeled Quit the wizard and Cloudera Manager will delete the temporarily created cluster and then click Continue.

Otherwise, after addressing any identified problems, select the radio button labeled I understand the risks, let me continue with cluster creation, and then click Continue.

This completes the Add Cluster - Installation wizard and launches the Add Cluster - Configuration wizard. For further instructions, see Step 7: Set Up a Cluster Using the Wizard in the installation guide.

Adding a Cluster Using Currently Managed Hosts

  Important: This section covers the procedure for creating a cluster from existing managed hosts only. For instructions using new (currently unmanaged) hosts, see Adding a Cluster Using New Hosts.

Before continuing, make sure that the managed hosts have the desired CDH version packages pre-installed.

On the Cloudera Manager Home page, click the Add drop-down button at the top right, or the Clusters drop-down button at the top left, and then click Add Cluster. This launches the Add Cluster - Installation wizard, which allows you to create either a regular cluster or a compute cluster .

You can also launch the wizard by selecting Add Compute Cluster from the drop-down menu next to the cluster name. Launching the wizard from there skips the Welcome page and restricts the wizard to creating only a compute cluster.

The following sections guide you through each page of the wizard:

Welcome (Add Cluster - Installation)

The Welcome page of the Add Cluster - Installation wizard provides a brief overview of the installation and configuration procedure, as well as some links to relevant documentation.

If you launched the wizard using the Add Compute Cluster option, this page is not displayed.

Click Continue to proceed with the installation.

Cluster Basics

The Cluster Basics page allows you to specify the Cluster Name and select the Cluster Type:
  • Regular Cluster: A Regular Cluster contains storage nodes, compute nodes, and other services such as metadata and security collocated in a single cluster.
  • Compute Cluster: A Compute Cluster consists of only compute nodes. To connect to existing storage, metadata or security services, you must first choose or create a Data Context on a Base Cluster.

For more information on regular and compute clusters, and data contexts, see Virtual Private Clusters and Cloudera SDX.

If you are adding a compute cluster to an existing base cluster, click Choose Data Context... to create or select a Data Context.

After selecting a cluster type and data context (if applicable), enter a cluster name and then click Continue.

Setup Auto-TLS

  Important: Auto-TLS is only available with an Enterprise license.

The Setup Auto-TLS page provides instructions for initializing the certificate manager for auto-TLS if you have not done so already. If you already initialized the certificate manager in Step 3: Install Cloudera Manager Server, the wizard displays a message indicating that auto-TLS has been initialized. Click Continue to proceed with the installation.

If you have not already initialized the certificate manager, and you want to enable auto-TLS, follow the instructions provided on the page before continuing. When you reload the page as instructed, you are redirected to https://<server_host>:7183, and a security warning is displayed. You might need to indicate that you trust the certificate, or click to proceed to the Cloudera Manager Server host. You might also be required to log in again and re-complete the previous steps in the wizard.

For more information, see Configuring TLS Encryption for Cloudera Manager and CDH Using Auto-TLS .

If you do not want to enable auto-TLS at this time, click Continue to proceed.

Specify Hosts

  Note: This section covers the procedure for creating a cluster from existing managed hosts only. For instructions using new (currently unmanaged) hosts, see Adding a Cluster Using New Hosts.

Select the hosts for your cluster by clicking the Currently Managed Hosts tab. This tab does not appear if you have no unassigned managed hosts. You cannot select a mixture of new hosts and currently managed hosts.

If you are installing CDH and other services using packages instead of parcels, make sure that you have manually installed the CDH packages on each host before continuing.

Select the hosts you want to add to the cluster, and then click Continue.

Select Repository

  Important: You cannot install software using both parcels and packages in the same cluster.

The Select Repository page allows you to specify repositories for Cloudera Manager Agent and CDH and other software. If you are installing on currently managed hosts, the Cloudera Manager Agent section is not displayed.

In the Cloudera Manager Agent section:

  1. Select either Public Cloudera Repository or Custom Repository for the Cloudera Manager Agent software.
  2. If you select Custom Repository, do not include the operating system-specific paths in the URL. For instructions on setting up a custom repository, see Configuring a Local Package Repository.

In the CDH and other software section:

  1. Select the repository type to use for the installation. In the Install Method section select one of the following:
    • Use Parcels (Recommended)

      A parcel is a binary distribution format containing the program files, along with additional metadata used by Cloudera Manager. Parcels are required for rolling upgrades. For more information, see Parcels.

    • Use Packages

      A package is a standard binary distribution format that contains compiled code and meta-information such as a package description, version, and dependencies. Packages are installed using your operating system package manager.

    If you select Use Packages, make sure that you have manually installed the CDH packages on each host before continuing.

  2. Select the version of CDH to install. For compute clusters using parcels, the supported CDH versions display (Supported) next to the parcel name. For compute clusters using packages, you must make sure that you have installed a supported CDH version on all compute cluster hosts.

    If you selected Use Parcels and you do not see the version you want to install, click the More Options button to add the repository URL for your version. Repository URLs for CDH 6 version are documented in CDH 6 Download Information. After adding the repository, click Save Changes and wait a few seconds for the version to appear. If your Cloudera Manager host uses an HTTP proxy, click the Proxy Settings button to configure your proxy.

      Note: Cloudera Manager only displays CDH versions it can support. If an available CDH version is too new for your Cloudera Manager version, it is not displayed.
  3. If you selected Use Parcels, specify any Additional Parcels you want to install. If you are installing CDH 6, do not select the KAFKA, KUDU, or SPARK parcels, because they are included in CDH 6.
  4. Click Continue.

If you are using parcels, the Install Parcels page displays. If you chose to install using packages, the Detecting CDH Versions page displays.

Install Parcels

If you selected parcels for the installation method, the Install Parcels page reports the installation progress of the parcels you selected earlier. After the parcels are downloaded, progress bars appear representing each cluster host. You can click on an individual progress bar for details about that host.

After the installation is complete, click Continue.

The Inspect Cluster page displays.

Detecting CDH Versions (Package Install Only)

If you are installing CDH using packages instead of parcels, theDetecting CDH Versions page validates that the correct versions of the CDH components are installed. The packages must be installed before proceeding. If any issues are reported, correct them, and then click Check Again.

After the validation succeeds, click Continue.

Inspect Cluster

The Inspect Cluster page provides a tool for inspecting network performance as well as the Host Inspector to search for common configuration problems. Cloudera recommends that you run the inspectors sequentially:

  1. Run the Inspect Network Performance tool. For compute clusters, you can test the network performance between the compute cluster and its base cluster, as well as within the compute cluster itself.

    You can also click Advanced Options to customize some ping parameters.

  2. After the network inspector completes, click Show Inspector Results to view the results in a new tab.
  3. Address any reported issues, and click Run Again (if applicable).
  4. Click Inspect Hosts to run the Host Inspector utility.
  5. After the host inspector completes, click Show Inspector Results to view the results in a new tab.
  6. Address any reported issues, and click Run Again (if applicable).

If the reported issues cannot be resolved in a timely manner, and you want to abandon the cluster creation wizard to address them, select the radio button labeled Quit the wizard and Cloudera Manager will delete the temporarily created cluster and then click Continue.

Otherwise, after addressing any identified problems, select the radio button labeled I understand the risks, let me continue with cluster creation, and then click Continue.

This completes the Add Cluster - Installation wizard and launches the Add Cluster - Configuration wizard. For further instructions, see Step 7: Set Up a Cluster Using the Wizard in the installation guide.

Deleting a Cluster

To delete a cluster:

  1. Stop the cluster.
  2. On the Home > Status tab, click the drop-down arrow to the right of the cluster name and select Delete.
Page generated August 29, 2019.