January 30th, 2017
Updated: June 22nd, 2017
Previous: NodeMCU, MQTT and Domoticz - part 1

Getting data to the Wemos D1 mini is slightly more involved than having the D1 mini push data out.

Subscribe and Display Messages from an MQTT Broker

To test communication with an MQTT broker there must obviouly be one on the local area network. As before, I will assume it is running on a computer with IP address 192.168.1.49. In my case this is

It is necessary to specify an on "message" callback function that will do something with the received messages on a subscribed topic. For this first example, the D1 mini will print them.

> m = mqtt.Client("myNode", 120, "", "") -- blank user and password > m:on("connect", function() print("connected") end ) > m:on("message", function(client, topic, data) print("received: " .. topic .. ": " .. data) end ) > m:connect("192.168.0.49") -- my local broker's IP > connected > m:subscribe("home/#", 0) -- no QoS

To see it in action, publish a message to the MQTT broker on the "home/xxx" topic. For example:

pi@rpi2b:~ $ mosquitto_pub -h localhost -t "home/another" -m "hello back!"

The Raspberry Pi subscriber terminal will now show

pi@rpi2b:~ $ mosquitto_sub -h localhost -v -t home/# home/test hi from wemos home/another hello back!  

At the same time the terminal connected to the D1 mini will show

> m = mqtt.Client("myNode", 120, "", "") -- blank user and password > m:on("connect", function() print("connected") end ) > m:on("message", function(client, topic, data) print(topic .. ": " .. data) end ) > m:connect("192.168.0.49") -- my local broker's IP > connected > m:subscribe("home/#", 0) -- no QoS > received: home/another: hello back!

And if the subscriber terminal on the Raspberry Pi or desktop is still opened, it will show the message also.

Remotely turning on the D1 mini LED

The previous example shows that the board can certainly receive MQTT messages. Time to try something useful: turning the LED on and off remotely.

Here is a a Lua script, mqtt_in.lua , t hat will do that when receiving an 'ON' or 'OFF' message from the MQTT broker for the topic 'ha/D1mini/LED'.

-- Handling subscribed MQTT messages -- Lighting the builtin LED with published MQTT messages -- 'ON' or 'OFF' to topic 'ha/D1mini/LED' mqttIP = '192.168.0.49' -- IP address of MQTT broker mqttTopic = 'ha/D1mini/LED' -- subscribed topic builtin_led = 4 -- The D1 mini's builtin LED gpio.mode(builtin_led, gpio.OUTPUT) -- blinking parameters period = 30*1000 count = 10 -- the blink function used to signal -- the start of the program and an error function doBlink() for i=1,count,1 do gpio.write(builtin_led, gpio.LOW) tmr.delay(period) gpio.write(builtin_led, gpio.HIGH) tmr.delay(period) end end -- subscribe to MQTT broker once connected -- display information on connection with MQTT broker function clientConnected() m:subscribe(mqttTopic, 0) doBlink() end -- handle MQTT incoming message function receivedMsg(client, topic, data) if data == 'ON' then gpio.write(builtin_led, gpio.LOW) elseif data == 'OFF' then gpio.write(builtin_led, gpio.HIGH) else doBlink() -- signal error, will turn LED off if data ~= nil then print('** Error: invalid message parameter: '..data..' **') else print('** Error: missing message parameter **') end end end -- create and connect MQTT client, 120 second keep alive timer and NO security m = mqtt.Client("WemosTest", 120, "", "") m:on("connect", clientConnected) m:on("message", receivedMsg) m:connect(mqttIP)

The script does pretty much what was done manually in the previous section. There is a notable exception: subscribing to a topic with the MQTT broker is done in the on "Connect" message from the MQTT client. The reason for that arrangement is easily grasped when remembering that NodeMCU is event driven, not procedural. The sequence

    m:connect("192.168.0.49")
    m:subscribe("ha/D1mini/LED", 0)
will not work in a script because the connection with the broker will not yet be established when trying to subscribe with it. It worked when the code was entered manually in the interpreter because I am a slow typist.

To test the script, messages must be published to the broker. Try the following sequence:

michel@hp:~$ mosquitto_pub -h 192.168.0.49 -t 'ha/D1mini/LED' -m 'ON' michel@hp:~$ mosquitto_pub -h 192.168.0.49 -t 'ha/D1mini/LED' -m 'OFF' michel@hp:~$ mosquitto_pub -h 192.168.0.49 -t 'ha/D1mini/LED' -m 'xx'

In the last case, the LED will blink quickly indicating an error and then remain off. An error message is printed to the serial port. To see it, load the script, then start the terminal:

michel@hp:~/Documents/Site/michel/src/program/misc$ nodemcu-uploader upload mqtt_in.lua opening port /dev/ttyUSB0 with 115200 baud Preparing esp for transfer. Transferring mqtt_in.lua as mqtt_in.lua All done! michel@hp:~/Documents/Site/michel/src/program/misc$ miniterm.py --- Available ports: --- 1: /dev/ttyUSB0 USB2.0-Serial --- Enter port index or full name: 1 --- Miniterm on /dev/ttyUSB0 9600,8,N,1 --- --- Quit: Ctrl+] | Menu: Ctrl+T | Help: Ctrl+T followed by Ctrl+H --- ������ > dofile('mqtt_in.lua') Send the 'xx' message on the topic 'ha/D1mini/LED' to the MQTT broker > ** Error: invalid message parameter: xx **

It will probably be necessary to press the RESET button after uploading the file and before starting the terminal. At first garbage is displayed in the terminal, press the Enter key a couple of times. If that does not lead to the > prompt, press the RESET button on the D1 mini and then press the Enter key.

Turning on the D1 Mini LED with Domoticz

In the previous part we saw how to update as virtual sensor in Domoticz with MQTT messages. Now we will look at how a virtual switch in Domoticz can turn the D1 Mini LED on and off using MQTT messages.

There are four steps involved. A virtual switch has to be created in Domoticz and the home automation program must be setup to publish MQTT messages. Finally a Lua script to handle MQTT messages has to be uploaded to the D1 mini.

The first step is to create a virtual temperature sensor. The procedure is as follows.

  1. Click on the Setup tab.
  2. Click on the Hardware menu item.
  3. If you do not have Dummy (Does nothing, use for virtual switches only) hardware, you will have to create it.
    • Set the hardware Name: to something like Virtual or any other name you wish.
    • Set the hardware Type: to Dummy (Does nothing, use for virtual switches only) by selecting that type in the drop down list.
    • Click on the Add button at the bottom of the page. The Dummy hardware will be added to the table at the top.
  4. Click on the Create Virtual Sensors button in the Dummy hardware row which on my system I called Virtual.
  5. Name the device (I called it D1miniLED in the Create Virtual Sensor window and change the sensor type to Switch.
  6. Click on the OK button. A message saying that the device was created and that it can be found in the devices list should appear.
  7. Click on the Setup tab.
  8. Click on the Devices menu item.
  9. Find the newly created device by name in the Name column and note its index number in the Idx column. Alternatively, start typing the name in the Search box, the corresponding device row will quickly be displayed.

The second step, setting up Domoticz to send out MQTT messages, has already been disussed here. Quickly, here is what is involved.

  1. Click on the Setup tab.
  2. Click on the Hardware menu item.
  3. If MQTT Client Gateway with LAN interface is not in the table of installed hardware, then add as outlined:
    • Set the hardware Name: to something like MQTT or any other name you wish.
    • Set the hardware Type: to MQTT Client Gateway with LAN interface by selecting that type in the drop down list.
    • Click on the Add button at the bottom of the page. The Dummy hardware will be added to the table at the top.
  4. Select MQTT Client Gateway with LAN interface row in the hardware table.
  5. Set Publish Topic to out which corresponds to Flat in the description below.
  6. Everything else should be ok, so click on the Update button just below the hardware table.

    The Lua script used above must be modified to handle the MQTT messages from Domoticz. The important aspect of those messages is that the "payload" is a set of JSON formatted key, value pairs. While there are quite a few such pairs, only two are of importance for us are "idx" : 43 and "nvalue" : 1.

    -- Handling Domoticz MQTT messages -- Lighting the builtin LED with published MQTT messages -- 'ON' or 'OFF' to topic 'domoticz/out' with correct idx -- value in JSON payload mqttIP = '192.168.0.22' -- IP address of MQTT broker mqttTopic = 'domoticz/out' -- subscribe to all domoticz mqtt messages idx = 43 -- index of virtual switch D1MiniLED builtin_led = 4 -- The D1 mini's builtin LED -- setup gpio pins gpio.mode(builtin_led, gpio.OUTPUT) -- blinking parameters period = 30*1000 -- the blink function used to signal the start of the program (4 blinks) -- receipt of a pertinent MQTT message (2 blinks), -- and an error in the message (10 blinks) -- always terminates with LED off function doBlink(count) for i=1,count,1 do gpio.write(builtin_led, gpio.LOW) tmr.delay(period) gpio.write(builtin_led, gpio.HIGH) tmr.delay(period) end end -- subscribe to MQTT broker once connected -- display information on connection with MQTT broker function clientConnected() m:subscribe(mqttTopic, 0) doBlink(4) end -- handle MQTT incoming message function receivedMsg(client, topic, data) t = cjson.decode(data) if t["idx"] == idx then doBlink(2) nval = t["nvalue"] if nval == nil then nval = 0 end print(mqttTopic..'{ "idx" : '..idx..', "nvalue" : '..nval..' }') if (nval==1) then print("Turning LED ON") gpio.write(builtin_led, gpio.LOW) elseif (nval==0) then print("Turning LED OFF") gpio.write(builtin_led, gpio.HIGH) else print("Invalid nvalue: "..nval) doBlink(10) end end end -- create and connect MQTT client, 120 second keep alive timer and NO security m = mqtt.Client("WemosTest", 120, "", "") m:on("connect", clientConnected) m:on("message", receivedMsg) m:connect(mqttIP)

    June 22nd, 2017
    Update
    The CJSON module of NodeMCU has been replaced with the SJSON module. Since the latter can be used as a "drop-in replacement" of the former, the only change that needs to be made to the above script is
    ... -- handle MQTT incoming message function receivedMsg(client, topic, data) t = cjson.decode(data) ...
    to
    ... -- handle MQTT incoming message function receivedMsg(client, topic, data) local t = sjson.decode(data) ...
    The addition of the local directive is just better coding; it should have been in the original script. The significant change is sjson instead of cjson. If you get a JSON parse error: INVALID_NUMBER, then you have an older version of the SJSON module. Line feeds were not being treated as white space to be ignored. The bug has been identified and fixed. As of June 22nd, builds against the master branch obtained from NodeMCU custom builds did not contain the correction. Perhaps those agains the dev branch do. Until you get the corrected version of SJSON, you can try the following
    ... -- handle MQTT incoming message function receivedMsg(client, topic, data) local t = sjson.decode(string.gsub(data, "\n", "")) ...
    It worked for me.

    The only meaningful difference with the previous script is found in function receivedMsg(client, topic, data). In this case data, the message, is a JSON string such as:

    {
       "Battery" : 255,
       "RSSI" : 12,
       "dtype" : "Lighting 1",
       "id" : "68",
       "idx" : 1,
       "name" : "Commode",
       "nvalue" : 0,
       "stype" : "X10",
       "switchType" : "On/Off",
       "unit" : 1
    }
    which must be parsed to make sure that the information concerns our device (using the "idx" key) and to find out the new state of the switch (using the "nvalue" key). Luckily, the CJSON module of NodeMCU does the heavy lifting. The JSON string is broken out in to a Lua table with the line t = cjson.decode(data) (or, more recently, t = cjson.decode(data), see the above sidebar) so that the value associated with a key are accessed as t["key"].

    The Lua script is available, right click on this link, domoticz_in.lua and select download.

    So what remains to be done is to load the Lua script into the device and to run it. As before this is done from a terminal.

    michel@hp:~/Documents/misc$ nodemcu-uploader upload domoticz_in.lua opening port /dev/ttyUSB0 with 115200 baud Preparing esp for transfer. Preparation already done. Not adding functions again. Transferring domoticz_in.lua as domoticz_in.lua All done! michel@hp:~/Documents/misc$ miniterm.py /dev/ttyUSB0 115200 --- Miniterm on /dev/ttyUSB0 115200,8,N,1 --- --- Quit: Ctrl+] | Menu: Ctrl+T | Help: Ctrl+T followed by Ctrl+H --- > print(wifi.sta.getip()) 192.168.0.123 255.255.255.0 192.168.0.1 > dofile('domoticz_in.lua') >
    Note that I checked that the ESP8266 is connected to the network and then executed the script.

    To test, it might be a good idea to open another terminal and to subscribe to all messages with the MQTT broker.

    michel@hp:~/Documents/misc$ mosquitto_sub -h 192.168.0.22 -v -t "#"

    The test if to click on the bulb icon of the virtual switch to turn it on and off and verifying that the LED of the D1 mini follows along.

    More info

    I found a very detailed tutorial Introduction to the MQTT Protocol on NodeMCU by Patrick Lloyd on ALL ABOUT CIRCUITS. The site looks very interesting.