Issue debugging and mods to decoder_test.py

[NateEaton]

Having forked and cloned the latest dev repo, I have built decoder and gateway from source and the gateway (running on a rpi Zero W) is successfully sending data for Inkbird IBT 4X (iBBQ) and Maestro Blue Tempo Disc to my Home Assistance instance (on an rpi 4B).

I am now trying my hand at creating a new device decoder for a SensorBug (mentioned in another thread). I'm including the json header file below.

Although I have through trial and error made a few mods to IBT and BM (e.g., the IBT now sends a separate message for each temperature probe indicating if a probe is plugged into the base unit), I find I am butting my head against SBUG. At this point, I have even tried simplifying the device definition such that it just has the mfrdata condition and then emits a static value (i.e., no conditions set for defining a property) but even that didn't work. It seems pretty simple: if mfrdata starts with '8500' in index 0, then it's a SensorBug but nothing I've tried has returned any data.

In addition, while testing, I kept getting an error out of decoder_test.py whenever the decoder returned None:

$ python  decoder_test.py
None 

Traceback (most recent call last):
  File "/home/pi/decoder/tests/python_ble/decoder_test_orig.py", line 10, in <module>
    p = json.loads(z)
  File "/usr/lib/python3.9/json/__init__.py", line 339, in loads
    raise TypeError(f'the JSON object must be str, bytes or bytearray, '
TypeError: the JSON object must be str, bytes or bytearray, not NoneType

To address that, I made a mod to test whether z is None and only proceed if not (see code below).

Note that in my test, I have three conditions:

• working mfrdata (Blue Maestro)
• the mfrdata from a scan with ScanAndDecode.py that showed what it was picking up from the SensorBug and sending to decoder
• the null test from original test code

I've included the result of test run, too.

Anything obvious about what is going on? Have a just misunderstood the input being sent to decoder? Something else, e.g., a noob snafu?

Thanks,
Nate

SBUG_json.h (which I did ensure is referenced in devices.h)

const char* _SBUG_json = "{\"brand\":\"BlueRadios\",\"model\":\"SensorBug\",\"model_id\":\"BR-BUTTON-S3A\",\"condition\":[\"manufacturerdata\",\"index\",0,\"8500\"|\"manufacturerdata\",\"index\",0,\"0085\"],\"properties\":{\"batt\":{\"condition\":[\"manufacturerdata\",8,\"=\",\"3C\"],\"decoder\":[\"value_from_hex_data\",\"manufacturerdata\",10,4,true,false]},\"motion\":{\"condition\":[\"manufacturerdata\",14,\"=\",\"41\"],\"decoder\":[\"value_from_hex_data\",\"manufacturerdata\",16,4,true,false]},\"light\":{\"condition\":[\"manufacturerdata\",20,\"=\",\"42\"],\"decoder\":[\"value_from_hex_data\",\"manufacturerdata\",22,4,true,false]},\"tempc\":{\"condition\":[\"manufacturerdata\",28,\"=\",\"43\"],\"decoder\":[\"value_from_hex_data\",\"manufacturerdata\",30,4,true,false],\"post_proc\":[\"*\",0.0625]}}}";
/*R""""(
{
   "brand":"BlueRadios",
   "model":"SensorBug",
   "model_id":"BR-BUTTON-S3A",
   "condition":["manufacturerdata", "index", 0, "8500" | "manufacturerdata", "index", 0, "0085" ],
   "properties":{
      "batt":{
         "condition":["manufacturerdata", 8, "=", "3C"],
         "decoder":["value_from_hex_data", "manufacturerdata", 10, 4, true, false]
      },
      "motion":{
         "condition":["manufacturerdata", 14, "=", "41"],
         "decoder":["value_from_hex_data", "manufacturerdata", 16, 4, true, false]
      },
      "light":{
         "condition":["manufacturerdata", 20, "=", "42"],
         "decoder":["value_from_hex_data", "manufacturerdata", 22, 4, true, false]
      },
      "tempc":{
         "condition":["manufacturerdata", 28, "=", "43"],
         "decoder":["value_from_hex_data", "manufacturerdata", 30, 4, true, false],
         "post_proc":["*", 0.0625]
      }
   }
})"""";*/

const char* _SBUG_json_props = "{\"properties\":{\"batt\":{\"unit\":\"%\",\"name\":\"battery\"},\"motion\":{\"unit\":\"status\",\"name\":\"motion\"},\"light\":{\"unit\":\"int\",\"name\":\"lightlevel\"},\"tempc\":{\"unit\":\"°C\",\"name\":\"temperature\"}}}";
/*R""""(
{
   "properties":{
      "batt":{
         "unit":"%",
         "name":"battery"
      },
      "motion":{
         "unit":"status",
         "name":"motion"
      },
      "light":{
         "unit":"int",
         "name":"light level"
      },
      "tempc":{
         "unit":"°C",
         "name":"temperature"
      }
   }
})"""";*/

Revised decoder_test.py

from TheengsDecoder import decodeBLE as dble
from TheengsDecoder import getProperties
from TheengsDecoder import getAttribute
import json

# x = {"servicedata":"712098004a63b6658d7cc40d071003f32600"} 
w = {"manufacturerdata": "850002003c610041014442160100433c01"}
print("Decoder sent:", w, "\n");
z = dble(json.dumps(w))

print("Decoder result (something expected?):\n")
print(z, "\n")

if z is not None:
    p = json.loads(z)
    print(getProperties(p['model_id']), "\n")
    brand = getAttribute(p['model_id'], 'brand')
    model = getAttribute(p['model_id'], 'model')
    print("brand:", brand, ", model:", model)

x = {"manufacturerdata": "330117630e10015c00a301ac00240100", "name": "C2075592"}
print("Decoder sent:", x, "\n");
z = dble(json.dumps(x))

print("Decoder result (something expected):\n")
print(z, "\n")

if z is not None:
    p = json.loads(z)
    print(getProperties(p['model_id']), "\n")
    brand = getAttribute(p['model_id'], 'brand')
    model = getAttribute(p['model_id'], 'model')
    print("brand:", brand, ", model:", model)

y = {}
print("Decoder sent:", y, "\n");
z = dble(json.dumps(y))
print("decoder result (None expected): ", z)
print("Done")

Output from decoder_test.py

$ python  decoder_test.py
Decoder sent: {'manufacturerdata': '850002003c610041014442160100433c01'} 

Decoder result (something expected?):

None 

Decoder sent: {'manufacturerdata': '330117630e10015c00a301ac00240100', 'name': 'C2075592'} 

Decoder result (something expected):

{"name":"C2075592","brand":"BlueMaestro","model":"TempoDisc","model_id":"BM_V23","tempc":16.3,"tempf":61.34,"hum":42.8,"tempc2_dp":3.6,"tempf2_dp":38.48,"volt":2.56} 

{"properties":{"volt":{"unit":"V","name":"voltage"},"tempc":{"unit":"°C","name":"temperature"},"tempc2_dp":{"unit":"°C","name":"dew point"},"hum":{"unit":"%","name":"humidity"}}} 

brand: BlueMaestro , model: TempoDisc
Decoder sent: {} 

decoder result (None expected):  None
Done

[DigiH]

Hi @NateEaton

"condition":["manufacturerdata", "index", 0, "8500" | "manufacturerdata", "index", 0, "0085" ],

have a look at other decoders and their model conditions with how "&" and "|" are implemented

"condition":["manufacturerdata", 20, "=", "42"],
"decoder":["value_from_hex_data", "manufacturerdata", 22, 4, true, false]

same for property conditions for evaluation data at a certain index.

Hope this helps

[NateEaton]

@DigiH , thanks. Guess I should work on this stuff during the day instead of the middle of the night... corrected code works:

"condition":["manufacturerdata", "index", 0, "8500"],

Although I get that the typo was having a bare OR operator instead of having it quoted, I had only added that when I was not getting results with "8500" and yet it works fine like this. At least it does once I made two other edits, adding reference to the new device in decoder.h and besides the "include" in devices.h, I was missing the reference to the json structure names.

Once I have finished the finer details on decoding the SensorBug light level and motion attributes, I'll get everything committed to my repo and plan to also add some suggested documentation updates on the specific steps to add a new device (based on my experience) with an eye towards submitting that as a PR.

"condition":["manufacturerdata", 20, "42"],
"decoder":["value_from_hex_data", "manufacturerdata", 22, 4, true, false]

One other thing I ran into that was biting me: afraid I was leaning a bit too hard on the LightBlue Android app for scanning devices and using the results as reference for my development. In it, hex values are in all upper-case. It turns out that the hex strings in python have lower-case letters so the condition I used to confirm the battery value exists in the mfrdata , comparing for "3C", was failing even when everything else in the new SBUG decoder was working. Changing it to "3c" was the needed fix. I figured I would include that here for benefit of anyone else who is struggling to get a new decoder working.

Appreciate you pointing me in the right direction.