Netdata Community

Hall effect sensor support, water flow calculations

I have a water flow sensor (Hall effect) feeding a Raspberry Pi that I am struggling with to pick up the values from into graphs. The values I see value(!) in is the flow and pulse / speed with it’s variations of the impeller and the total volume passing. The variation of speed is in my mind a valuable metrics for detecting wear and failure of the impeller as me as many others are using those cheap Amazon / Chinese flow sensors in combination with some cooling water flow of sorts. Personally I’m using it to measure water flow from a deep well that I’m cooling my server with via a fan coil unit.

I’ve picked up some useful code with sound maths that can give me the values I’m looking for but I cant for the life of me get it picked up by the Netdata python orchestrator. So far I have managed to run the code snippets inside the basic ‘sxs731.chart.py’ from /usr/libexec/netdata/plugins.d with ‘./python.d.plugin sxs731 -ppython3’. I get a printout so the while function runs and spits out proper values but I’m lost on how to use those values to populate the CHARTS.
Below are first the standalone python script that grabs the pulses from the hall effect sensor with a specified sample rate to calculate the values between the two samples:

#!/usr/bin/env python
DEBUG = False
import time, sys
import RPi.GPIO as GPIO
from datetime import datetime

# Configurations
INPUT_PIN = 23
GPIO.setmode(GPIO.BCM)
GPIO.setup(INPUT_PIN, GPIO.IN, pull_up_down=GPIO.PUD_UP)

# Variables
total_liters = 0
seconds = 0

# Sampling
sample_rate = 5  # Sampling each 5 seconds
time_start = 0
time_end = 0
period = 0;
hz = []       # Frequency !important!
m = 0.0015    # See data for specific Hall sensor application!

# Data 
db_good_sample = 0
db_hz = 0
db_liter_by_min = 0

while True:
    # start / end 
    time_start = time.time();
    init_time_start = time_start # undetect last edge 
    time_end = time_start + sample_rate
    hz = []
    sample_total_time = 0

    # Edge
    current = GPIO.input(INPUT_PIN)
    edge = current # Rising edge / Falling edge

    try:
        while time.time() <= time_end:
            t = time.time();
            v = GPIO.input(INPUT_PIN)
            if current != v and current == edge:
                period = t - time_start # Impulsion period
                new_hz = 1/period
                hz.append(new_hz)               # Period = 1/period
                sample_total_time += t - time_start
                time_start = t;
               
                if DEBUG:
                    print(round(new_hz, 4))     # Print hz
                    sys.stdout.flush()
            current = v;

        # Sums
        print('')
        seconds += sample_rate
        nb_samples = len(hz);
        if nb_samples >0:
            average = sum(hz) / float(len(hz));
            # Calcul % of good sample in time range
            good_sample = sample_total_time/sample_rate
            print(round(sample_total_time,4),'(sec) good sample')
            db_good_sample = round(good_sample*100,4)
            print(db_good_sample,'(%) good sample')
            average = average * good_sample
        else:
            average = 0
        average_liters = average*m*sample_rate;
        total_liters += average_liters
        db_hz = round(average,4);
        db_liter_by_min= round(average_liters*(60/sample_rate),4)
        print(db_hz,'(hz) average')
        print(db_liter_by_min,'(L/min)')
        print(round(total_liters,4),'(L) total')
        print(round(seconds/60,4), '(min) total')
        print('')

    except KeyboardInterrupt:
        print('\n CTRL+C - Exiting')
        GPIO.cleanup()
        sys.exit()
GPIO.cleanup()
print('Done')

Above script gives me this printout:


4.9495 (sec) good sample
98.9895 (%) good sample
25.4353 (hz) average
2.2892 (L/min)
1.2925 (L) total
0.75 (min) total

Very useful! :+1:

Below is my pathetic attempt to implement that snippet into a ‘generic’ *.chart.py and I get a printout, but I can’t wrap my head around on how to get them into the pre assigned CHARTS. Maybe populate variables like ‘flow’, ‘(hz) average’, ‘Liter/min’, ‘Liter Total’ that can be picked up for the CHARTS population? Also when run this code I kill my other sensor feed (BME680) so something is hanging the python orchestrator or such:

# _*_ coding: utf-8 _*_
# Description: sxs731 netdata module
# Author: Lennong
# SPDX-License-Identifier: GPL-3.0-or-Later
DEBUG = False
import time, sys
import RPi.GPIO as GPIO
from datetime import datetime

# configurations
INPUT_PIN = 23
GPIO.setmode(GPIO.BCM)
GPIO.setup(INPUT_PIN, GPIO.IN, pull_up_down=GPIO.PUD_UP)

# variables
total_liters = 0
seconds = 0

# Sampling
sample_rate = 5  # Sampling each 5 seconds
time_start = 0
time_end = 0
period = 0;
hz = []       # Frequency !important!
m = 0.0015    # See data for specific Hall sensor application!

# data 
db_good_sample = 0
db_hz = 0
db_liter_by_min = 0

from bases.FrameworkServices.SimpleService import SimpleService

ORDER = [
    'rotation',
    'flow',
    'total',
]

CHARTS = {
    'rotation': {
        'options': [None, 'Rotation', 'RPM', 'rotation', 'sxs731.rotation', 'line'],
        'lines': [
            ['rotation']
        ]
    },
    'flow': {
        'options': [None, 'Flow', 'L/min', 'flow', 'sxs731.flow', 'line'],
        'lines': [
            ['flow']
        ]
    },
    'total': {
        'options': [None, 'Total', 'Liter', 'total', 'sxs731.total', 'line'],
        'lines': [
            ['total']
        ]
    }
}


class Service(SimpleService):
    def __init__(self, configuration=None, name=None):
        SimpleService.__init__(self, configuration=configuration, name=name)
        self.order = ORDER
        self.definitions = CHARTS
        self.am = None

while True:
    # start / end 
    time_start = time.time();
    init_time_start = time_start # undetect last edge 
    time_end = time_start + sample_rate
    hz = []
    sample_total_time = 0

    # Edge
    current = GPIO.input(INPUT_PIN)
    edge = current # Rising edge / Falling edge

    try:
        while time.time() <= time_end:
            t = time.time();
            v = GPIO.input(INPUT_PIN)
            if current != v and current == edge:
                period = t - time_start # Impulsion period
                new_hz = 1/period
                hz.append(new_hz)               # Period = 1/period
                sample_total_time += t - time_start
                time_start = t;
               
                if DEBUG:
                    print(round(new_hz, 4))     # Print hz
                    sys.stdout.flush()
            current = v;

        # Sums
        seconds += sample_rate
        nb_samples = len(hz);
        if nb_samples >0:
            average = sum(hz) / float(len(hz));
            # Calcul % of good sample in time range
            good_sample = sample_total_time/sample_rate
            print('')
            print(round(sample_total_time,4),'(sec) good sample')
            db_good_sample = round(good_sample*100,4)
            print(db_good_sample,'(%) good sample')
            average = average * good_sample
        else:
            average = 0
        average_liters = average*m*sample_rate;
        total_liters += average_liters
        db_hz = round(average,4);
        db_liter_by_min= round(average_liters*(60/sample_rate),4)
        print(db_hz,'(hz) average') # **<--- rotation**
        print(db_liter_by_min,'(L/min)') # **<--- flow**
        print(round(total_liters,4),'(L) total') # **<--- total**
        print(round(seconds/60,4), '(min) total')
        print('')

    except KeyboardInterrupt:
        print('\n CTRL+C - Exiting')
        GPIO.cleanup()
        sys.exit()
GPIO.cleanup()
print('Done')

def get_data(self):
    try:
        return {
            'rotation': self.am.rotation,
            'flow': self.am.flow,
            'total': self.am.total,
        }

    except (OSError, RuntimeError) as error:
        self.error(error)
        return None

Specs for Hall flow sensor:
https://www.amazon.se/gp/product/B07BSXS731

Zerodis Hall effect flow sensor:
Material: Copper
Type: G1/2"(External Thread Diameter), DN20mm
Water Quality Requirement: ≤60℃
Start Flow Range: 1.5L/min
Flow Range: 1-30L/min
1 Liter: 596 pulses
Maximum Water Pressure: 1.75MPa
Working Voltage Range: DC4.5-18V
Max Current: 10mA
Insulation resistance:>100MΩ
Electrical Strength: AC500V, 50Hz
Copper Switch Length: Approx. 60mm / 2.4inch
Wire Length: Approx. 35cm / 13.8inch
Weight: Approx. 95g

Any ideas?

Also, you need to add netdata user to gpio group if running this on rasbian / raspberry pi:
adduser netdata gpio

Hi, @Lennong.

You need to implement Service.get_data() method. It should return a dictionary: keys are dimensions, values are actual values. Python.d.plugin calls get_data() every update_every (so there shouldn’t be any infinite loops).

An example

class Service(SimpleService):
    def __init__(self, configuration=None, name=None):
        SimpleService.__init__(self, configuration=configuration, name=name)
        self.order = ORDER
        self.definitions = CHARTS
        self.am = None

    def get_data(self):
               # calculations
                return {
                    'rotation': rotation_value,
                    'flow': flow_value,
                    'total': total_value,
               }
           

Ahaa! Yes, I suspected the loop was out of place… I will get right on it!

…I did botch-up another script based on what you supported to another poor soul that needed to read out sensors data (from a log file). https://github.com/netdata/netdata/issues/2813#issuecomment-340784223
It has worked fine for my case as well until now but if implement as above it surely will save cpu cycles compared to spit out a log for only this purpose.

Thanks!!! :+1:

Ok, I gave it another try but I still fail to pick up the value within the ‘def get_data(self):’ class. Even though I declare the ‘flow’ variable as global it just wont be picked up. This Python stuff is really sooo different from regular shell scripts… This is the most scaled down counter I could put together and I even reset the gpio at every run of it. Still no success:

# _*_ coding: utf-8 _*_
# Description: sxs731 netdata module
# SPDX-License-Identifier: GPL-3.0-or-Later

import RPi.GPIO as GPIO
import time, sys

FLOW_SENSOR_PIN = 23

from bases.FrameworkServices.SimpleService import SimpleService

ORDER = [
    'flow',
]

CHARTS = {
    'flow': {
        'options': [None, 'Flow', 'L/min', 'flow', 'sxs731.flow', 'line'],
        'lines': [
            ['flow']
        ]
    }
}


class Service(SimpleService):
    def __init__(self, configuration=None, name=None):
        SimpleService.__init__(self, configuration=configuration, name=name)
        self.order = ORDER
        self.definitions = CHARTS
        self.am = None

    def get_data(self):
        GPIO.setmode(GPIO.BCM)
        GPIO.setup(FLOW_SENSOR_PIN, GPIO.IN, pull_up_down = GPIO.PUD_UP)
        global count
        count = 0
        def countPulse(channel):
            global count
            if start_counter == 1:
                count = count+1

        GPIO.add_event_detect(FLOW_SENSOR_PIN, GPIO.FALLING, callback=countPulse)

        start_counter = 1
        time.sleep(1)
        start_counter = 0
        flow = (count * 60 * 0.75 / 1000)
        #format_flow = "{:.2f}".format(flow)
        print(flow)
        GPIO.cleanup()
        
        return {
            'flow': flow,
        }

I get a value printed out from the ‘print(flow)’ call but nothing gets picked up in the 'return { call…

2021-11-04 04:14:32: python.d INFO: plugin[main] : sxs731[sxs731] : check success
CHART netdata.runtime_sxs731 '' 'Execution time for sxs731' 'ms' 'python.d' netdata.pythond_runtime line 145000 1 '' 'python.d.plugin' 'sxs731'
DIMENSION run_time 'run time' absolute 1 1

0.99
CHART sxs731.flow '' 'Flow' 'L/min' 'flow' 'sxs731.flow' line 60000 1 '' 'python.d.plugin' 'sxs731'
DIMENSION 'flow' 'flow' absolute 1 1 ' '

BEGIN sxs731.flow 0
SET 'flow' = 0
END

BEGIN netdata.runtime_sxs731 0
SET run_time = 1003
END

0.945
BEGIN sxs731.flow 2000855
SET 'flow' = 0
END

BEGIN netdata.runtime_sxs731 2000855
SET run_time = 1003
END

0.99
BEGIN sxs731.flow 1999997
SET 'flow' = 0
END

BEGIN netdata.runtime_sxs731 1999997
SET run_time = 1003
END

^C2021-11-04 04:14:38: python.d INFO: plugin[main] : exiting from main...

I mostly sit with shell scripts and the logic with this is soo weird to me to get my head around… :exploding_head:

The return dict values should be ints.

External plugins API

  • value
    is the collected value, only integer values are collected. If you want to push fractional values, multiply this value by 100 or 1000 and set the DIMENSION divider to 1000.

So make it

...
        'lines': [
            ['flow', None, 1, 1000]
        ]
...
        return {
            'flow': flow * 1000,
        }
  • And don’t forget to remove all print (the func writes to stdout, Netdata parser will complain and shut down the plugin), use self.info() instead.
  • unlikely you need time.sleep(1), the plugin executes X.get_data() every second by default.

Great stuff! Up and running now with a low 6% CPU load compared to the whole before ‘solution’ that spit out a log and then read it (25% CPU load).

I had to keep the time.sleep(1) to be able to count the pulses within the same iteration it flows over the code (callback=countPulse) and get a pulse/sec reading.

I had some visitors and a bender weekend witht them but as soon as back on track I will get back to the original python code and share it. …will also share/pull request the BME680 sensor and possibly something useful for the apcusbd script as well.

Thanks again!! :+1: :+1: