Package org.apache.kafka.common.metrics.stats

Class TokenBucket

java.lang.Object
org.apache.kafka.common.metrics.stats.TokenBucket
All Implemented Interfaces:
Measurable, MeasurableStat, MetricValueProvider<Double>, Stat
public class TokenBucket
extends Object
implements MeasurableStat
The TokenBucket is a MeasurableStat implementing a token bucket algorithm that is usable within a Sensor. The Quota.bound() defined the refill rate of the bucket while the maximum burst or the maximum number of credits of the bucket is defined by * MetricConfig#timeWindowMs() * Quota#bound(). The quota is considered as exhausted when the amount of remaining credits in the bucket is below zero. The enforcement is done by the Sensor. Token Bucket vs Rate based Quota: The current sampled rate based quota does not cope well with bursty workloads. The issue is that a unique and large sample can hold the average above the quota until it is discarded. Practically, when this happens, one must wait until the sample is expired to bring the rate below the quota even though less time would be theoretically required. As an example, let's imagine that we have: - Quota (Q) = 5 - Samples (S) = 100 - Window (W) = 1s A burst of 560 brings the average rate (R) to 5.6 (560 / 100). The expected throttle time is computed as follow: ((R - Q / Q * S * W)) = ((5.6 - 5) / 5 * 100 * 1) = 12 secs. In practice, the average rate won't go below the quota before the burst is dropped from the samples so one must wait 100s (S * W). The token bucket relies on continuously updated amount of credits. Therefore, it does not suffers from the above issue. The same example would work as follow: - Quota (Q) = 5 - Burst (B) = 5 * 1 * 100 = 500 (Q * S * W) A burst of 560 brings the amount of credits to -60. One must wait 12s (-(-60)/5) to refill the bucket to zero.

  • Constructor Details

    • TokenBucket

      public TokenBucket()

    • TokenBucket

      public TokenBucket​(TimeUnit unit)

  • Method Details

    • measure

      public double measure​(MetricConfig config, long timeMs)
      Description copied from interface: Measurable
      Measure this quantity and return the result as a double
      Specified by:
      measure in interface Measurable
      Parameters:
      config - The configuration for this metric
      timeMs - The POSIX time in milliseconds the measurement is being taken
      Returns:
      The measured value

    • record

      public void record​(MetricConfig config, double value, long timeMs)
      Description copied from interface: Stat
      Record the given value
      Specified by:
      record in interface Stat
      Parameters:
      config - The configuration to use for this metric
      value - The value to record
      timeMs - The POSIX time in milliseconds this value occurred

    • toString

      public String toString()
      Overrides:
      toString in class Object