Cloudera Enterprise 6.3.x | Other versions

Limiting the Speed of Compactions

You can limit the speed at which HBase compactions run, by configuring hbase.regionserver.throughput.controller and its related settings. The default controller is org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController, which uses the following algorithm:

If compaction pressure is greater than 1.0, there is no speed limitation.
In off-peak hours, use a fixed throughput limitation, configured using hbase.hstore.compaction.throughput.offpeak, hbase.offpeak.start.hour, and hbase.offpeak.end.hour.
In normal hours, the max throughput is tuned between hbase.hstore.compaction.throughput.higher.bound and hbase.hstore.compaction.throughput.lower.bound (which default to 20 MB/sec and 10 MB/sec respectively), using the following formula, where compactionPressure is between 0.0 and 1.0. The compactionPressure refers to the number of store files that require compaction.
```
lower + (higher - lower) * compactionPressure
```

To disable compaction speed limits, set hbase.regionserver.throughput.controller to org.apache.hadoop.hbase.regionserver.compactions.NoLimitCompactionThroughputController.

Configure the Compaction Speed Using Cloudera Manager

Minimum Required Role: Configurator (also provided by Cluster Administrator, Full Administrator)

Go to the HBase service.
Click the Configuration tab.
Select HBase or HBase Service-Wide.

Search for HBase Service Advanced Configuration Snippet (Safety Valve) for hbase-site.xml. Paste the relevant properties from the following example into the field and modify the values as needed:

<property>
  <name>hbase.regionserver.throughput.controller</name>
  <value>org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController</value>
</property>
<property>
  <name>hbase.hstore.compaction.throughput.higher.bound</name>
  <value>20971520</value>
  <description>The default is 20 MB/sec</description>
</property>
<property>
  <name>hbase.hstore.compaction.throughput.lower.bound</name>
  <value>10485760</value>
  <description>The default is 10 MB/sec</description>
</property>
<property>
  <name>hbase.hstore.compaction.throughput.offpeak</name>
  <value>9223372036854775807</value>
  <description>The default is Long.MAX_VALUE, which effectively means no limitation</description>
</property>
<property>
  <name>hbase.offpeak.start.hour</name>
  <value>20</value>
  <description>When to begin using off-peak compaction settings, expressed as an integer between 0 and 23.</description>
</property>
<property>
  <name>hbase.offpeak.end.hour</name>
  <value>6</value>
  <description>When to stop using off-peak compaction settings, expressed as an integer between 0 and 23.</description>
</property>
<property>
  <name>hbase.hstore.compaction.throughput.tune.period</name>
  <value>60000</value>
</property>

Enter a Reason for change, and then click Save Changes to commit the changes.
Restart the service.

Page generated August 29, 2019.