Reliable Insights

If you've determined a metric should be tested, how do you go about that?

While you shouldn't automatically test metrics, Prometheus client libraries offer facilities to make it as easy as possible to do so.

 

Taking Python as an example, let's say we had a counter that we wanted to check was incremented:

from prometheus_client import Counter
my_counter = Counter('my_counter_total', 'A useful help string.')

def my_function():
    my_counter.inc()

To test this we compare the before and after values:

import unittest
from prometheus_client import REGISTRY

class TestMyFunction(unittest.TestCase):
    def test_metric_incremented(self):
       before =  REGISTRY.get_sample_value('my_counter_total')
       my_function()
       after = REGISTRY.get_sample_value('my_counter_total')
       self.assertEqual(1, after - before)

For gauges that are incremented/decremented and histograms you can use the same technique. For gauges that are set, you can simply compare the values.

The same approach works for Java using the getSampleValue method of CollectorRegistry.defaultRegistry.

 

Prometheus metrics are usually file-level global variables, registered with a global registry. While you could plumb around the registry to use, this would cause friction and decrease the number of useful metrics that'd end up being added. The global nature of metrics does not make them harder to unit test. In fact it reduces the chances that you'll miss a bit of plumbing, resulting in the metric and tests passing perfectly but not being exposed.

 

Need help testing your metrics? Contact us.