Cloudera Enterprise 6.3.x | Other versions

Configuring HBase Garbage Collection

  Warning: Configuring the JVM garbage collection for HBase is an advanced operation. Incorrect configuration can have major performance implications for your cluster. Test any configuration changes carefully.
Garbage collection (memory cleanup) by the JVM can cause HBase clients to experience excessive latency. See Tuning Java Garbage Collection for HBase for a discussion of various garbage collection settings and their impacts on performance.

To tune the garbage collection settings, you pass the relevant parameters to the JVM.

Example configuration values are not recommendations and should not be considered as such. This is not the complete list of configuration options related to garbage collection. See the documentation for your JVM for details on these settings.

-XX:+UseG1GC
Use the 'G1' garbage collection algorithm. You can tune G1 garbage collection to provide a consistent pause time, which benefits long-term running Java processes such as HBase, NameNode, Solr, and ZooKeeper. For more information about tuning G1, see the Oracle documentation on tuning garbage collection.
-XX:MaxGCPauseMillis=value
The garbage collection pause time. Set this to the maximum amount of latency your cluster can tolerate while allowing as much garbage collection as possible.
-XX:+ParallelRefProcEnabled
Enable or disable parallel reference processing by using a + or - symbol before the parameter name.
-XX:-ResizePLAB
Enable or disable resizing of Promotion Local Allocation Buffers (PLABs) by using a + or - symbol before the parameter name.
-XX:ParallelGCThreads=value
The number of parallel garbage collection threads to run concurrently.
-XX:G1NewSizePercent=value
The percent of the heap to be used for garbage collection. If the value is too low, garbage collection is ineffective. If the value is too high, not enough heap is available for other uses by HBase.

Configure HBase Garbage Collection Using Cloudera Manager

Minimum Required Role: Full Administrator

  1. Go to the HBase service.
  2. Click the Configuration tab.
  3. Select Scope > RegionServer.
  4. Select Category > Advanced.
  5. Locate the Java Configuration Options for HBase RegionServer property or search for it by typing its name in the Search box.
  6. Add or modify JVM configuration options.
  7. Enter a Reason for change, and then click Save Changes to commit the changes.
  8. Restart the role.

Using the HBase Garbage Collector with JDK11

G1GC is the default garbage collector when JDK11 is used. It operates well with large heap sizes, at least 16G, and can cause low throughput and OOM errors in a heap constrained application environment.

You can improve the performance of G1GC by increasing the number of concurrent making threads. You can set this number using the -XX:ConcGCThreads=n option. By default this option is set to 10% of the available total host CPUs. However in an environment with more CPUs, you can increase this number to add more concurrent marking threads.

Althought G1GC is the default garbage collector, you can use other garbage collector with JDK11. For example, you can enable CMS using the -XX:+UseConcMarkSweepGC java option at runtime.

JDK11 uses the JVM unified logging framework. You can print garbage collection information with the following command: -Xlog:gc*. The result of this command is a detailed overview during and outside the pauses about garbage collection activity, including the type of collection and time spent in specific phases of the pause. If you want to see both the garbage collection and heap information, use the -Xlog:gc+heap option.

Disabling the BoundedByteBufferPool

HBase uses a BoundedByteBufferPool to avoid fragmenting the heap. The G1 garbage collector reduces the need to avoid fragmenting the heap in some cases. If you use the G1 garbage collector, you can disable the BoundedByteBufferPool in HBase. This can reduce the number of "old generation" items that need to be collected. This configuration is experimental.

To disable the BoundedByteBufferPool, set the hbase.ipc.server.reservoir.enabled property to false.

Disable the BoundedByteBufferPool Using Cloudera Manager

  1. Go to the HBase service.
  2. Click the Configuration tab.
  3. Select Scope > RegionServer.
  4. Select Category > Advanced.
  5. Locate the HBase Service Advanced Configuration Snippet (Safety Valve) for hbase-site.xml property, or search for it by typing its name in the Search box.
  6. Add the following XML: 
    <property>
  <name>hbase.ipc.server.reservoir.enabled</name>
  <value>false</value>
</property>
  7. Enter a Reason for change, and then click Save Changes to commit the changes.
  8. Restart the service.
Page generated August 29, 2019.