May 27rd, 2017

A few months ago, I showcased an imperfect approach to adding X10 lamp modules which are light dimmers. Back then, I knew that I would not be keeping the X10 devices so I did not go further. But now that I am passing the hardware on to a sibling, I feel I should do better and I think I have come up with an acceptable solution given the limitations of the hardware. This was tested with an older (pre 2007) LM465 lamp module. It may not work properly with newer LM465

Table of Contents

  1. Adding a Virtual Dimmer in Domoticz
  2. Understanding the Domoticz Virtual Dimmer
  3. Behaviour of the LM465 Lamp Module
  4. A Lua Script to Control a Lamp Module
  5. A Bash Script to Update Domoticz
  6. Presets
  7. Downloads

  1. Adding A Virtual Dimmer Switch in Domoticz
  2. The first step in using a lamp module is to create a virtual dimmer switch that will represent the physical X10 lamp module with address B3. The process is initially the same as in the previous post.

    1. Click on the Setup tab
    2. Select Hardware in the drop down menu.
    3. Add a virtual dimmer switch by clicking on the Create Virtual Sensors in the Heyu hardware.
    4. Fill in the Name field in the Create Virtual Sensor window.
    5. Select Switch for Sensor Type
    6. Press the OK button

    To change the virtual On/Off switch to a virtual dimmer,

    1. Click on the Switches tab.
    2. Click on the Edit button of the newly created switch:
    3. The switch type will show as On/Off, select Dimmer.
    4. Click on the Save button
    5. Click on the Back button.

    You will see that the bulb icon will be different and that the virtual dimmer now has a movable slider to set the bulb intensity.

    Some add a virtual switch by clicking on Manual Light/Switch in the Switches tab. If they define it to be an X10 switch of type dimmer then that gives a virtual a dimmer with a slider that cannot be moved under any circumstances. They may then go on to define the dimmer to be of type AC or LightwaveRF which will result in a virtual dimmer with moveable slider. I think that may be a mistake. For one thing, the number of discrete light intensity levels supported by these devices does not correspond to the number of levels of X10 lamp modules. For another, these virtual devices correspond to supported physical devices, so I assume that when a user modifies their state, Domoticz will try to perform some action until it realizes that there is no hardware. And because these dimmers will have to be scripted to control the lamp module, those scripts will still have to be executed, and of course, written before hand.

    As recently as January 16th of this year, gizmocuz confirmed in the forum that the software does "not support dimming for X10" and he goes on to suggest creating a dimmer in the fashion described here.

  3. Understanding the Domoticz Virtual Dimmer
  4. There is no way that I can change how the virtual dimmer works, unless I were to fork Domoticz and modify the program. Since that's not about to happen; it is best to understand how the virtual dimmer works and try make the X10 lamp module behave in the same way using scripts.

    Here, in bullet form, is the gist of the behaviour.

    1. The bulb icon of a virtual dimmer works just like the bulb icon of an on/off switch.
    2. Clicking on the bulb icon to turn the light on or off never changes the position of the slider.
    3. The lamp can be turned off completely by moving the slider to the left edge of its track. However, after performing an 'Off' action, the slider will spring back to its initial position before the slider motion was started. In other words, turning the lamp off of by sliding the intensity control to the left edge is exactly the same as clicking on the bulb icon.
    4. Moving the slider always turns the bulb icon on if it was off unless the slider is released at the left edge of the track.

    It seems to me that the developers decided that a virtual dimmer should mimic a physical dimmer comprised of a rheostat in series with an on/off switch. It is therefore possible to set the level with the slider/rheostat and then switch the lamp off. When the lamp is lit again, its intensity will be at the set level.

    The following lua script making use of the logging capabilities of Domoticz allowed me to see what information will be available for scripting the dimmer:

    -- Script to investigate virtual dimmer switch -- Adjust the value assigned to NAME to reflect the dimmer switch name in Domoticz -- Adjust VERBOSE to control messages to Domoticz' log commandArray = {} NAME = 'Test Dimmer' VERSION = '1' -- to check correct version loaded VERBOSE = 0 -- 0 = logs main values only, 1 logs everything function LogVariables(x,depth,name) for k,v in pairs(x) do if (depth>0) or ((string.find(k,'device')~=nil) or (string.find(k,'variable')~=nil) or (string.sub(k,1,4)=='time') or (string.sub(k,1,8)=='security')) then if type(v)=="string" then print(name.."['"..k.."'] = '"..v.."'") end if type(v)=="number" then print(name.."['"..k.."'] = "..v) end if type(v)=="boolean" then print(name.."['"..k.."'] = "..tostring(v)) end if type(v)=="table" then LogVariables(v,depth+1,k); end end end end if devicechanged[NAME] then print('') print('') if (VERBOSE > 0) then print('Version: '..VERSION) end print('devicechanged['..NAME..']: '..devicechanged[NAME]) print('otherdevices_svalues['..NAME..']: ' .. otherdevices_svalues[NAME]) if (VERBOSE > 0) then print('') LogVariables(_G,0,''); end end

    Here is a sample of the log where the intensity of the lamp was set to 60% with the slider, then the lamp was turned off and lastly, turned on by clicking on the bulb icon.

    2017-05-28 13:36:59.577 User: Admin initiated a switch command (2/Test Dimmer/Set Level)
    2017-05-28 13:36:59.614 LUA: devicechanged[Test Dimmer]: Set Level: 60 %
    2017-05-28 13:36:59.614 LUA: otherdevices_svalues[Test Dimmer]: 60
    2017-05-28 13:36:59.579 (Heyu) Light/Switch (Test Dimmer)
    
    2017-05-28 13:37:26.636 User: Admin initiated a switch command (2/Test Dimmer/Off)
    2017-05-28 13:37:26.671 LUA: devicechanged[Test Dimmer]: Off
    2017-05-28 13:37:26.671 LUA: otherdevices_svalues[Test Dimmer]: 0
    2017-05-28 13:37:26.638 (Heyu) Light/Switch (Test Dimmer)
    
    2017-05-28 13:37:38.529 User: Admin initiated a switch command (2/Test Dimmer/On)
    2017-05-28 13:37:38.563 LUA: devicechanged[Test Dimmer]: On
    2017-05-28 13:37:38.563 LUA: otherdevices_svalues[Test Dimmer]: 0
    2017-05-28 13:37:38.531 (Heyu) Light/Switch (Test Dimmer)

    This log confirms that when Domoticz turns the lamp on, and expects it to resume the level of intensity it had when it was last turned off, it will not tell the script what that intensity was.

  5. Behaviour of the LM465 Lamp Module
  6. Using direct Heyu commands it becomes possible to investigate the behaviour of the X10 module. The following describes the behaviour of an older, pre 2007, LM465 lamp module.

    pi@domopole:~ $ heyu off b3 -- completely off at once to 0% intensity (0/22) pi@domopole:~ $ heyu on b3 -- completely on at once to 100% intensity (22/22) pi@domopole:~ $ heyu dim b3 12 -- dims by 12 steps (12/22 = 55%) to 45% intensity (22-12)/22 pi@domopole:~ $ heyu bright b3 8 -- brightens by 8 steps (8/22 = 36%) to 82% (22-12+8)/22 pi@domopole:~ $ heyu dim b3 14 -- dims by 14 steps (14/22 = 64%) to 18% (22-12+8-14)/22 pi@domopole:~ $ heyu on b3 -- does nothing! pi@domopole:~ $ heyu bright b3 22 -- brightens by 22 steps (22/22 = 100%) to 100% (can't go higher) pi@domopole:~ $ heyu dim b3 14 -- dims by 14 steps (14/22 = 64%) to 36% (22-14)/22 pi@domopole:~ $ heyu off b3 -- completely off at once to 0% intensity (0/22) pi@domopole:~ $ heyu bright b3 6 -- completely on at once to 100% intensity! pi@domopole:~ $ heyu dim b3 22 -- complete off in 22 steps to 0% intensity (22-22)/22 pi@domopole:~ $ heyu bright b3 6 -- brightens by 6 steps to 27% (6/22) pi@domopole:~ $ heyu dim b3 22 -- complete off in 22 steps to 0% intensity (6-22)/22 (can't go lower) pi@domopole:~ $ heyu on b3 -- again, does nothing! pi@domopole:~ $ heyu dim b3 1 -- does nothing, lamp already off pi@domopole:~ $ heyu off b3 -- does nothing, lamp already off pi@domopole:~ $ heyu dim b3 1 -- completely on at once to 100% intensity.

    The 'on' command only works if the previous command was an 'off' command. As can be seen, if the lamp was totally turned of with 'dim' commands, the 'on' command will not work. When the previous command was 'off', a 'bright' command (and, strangely, a 'dim' command) is the same as an 'on' command. In other words a 'bright' command cannot be used to set the light intensity to an intermediate level if an 'off' was issued before. On the other hand, a 'bright' command when the lamp was completely off with dims works as expected.

    Just to complicate things a little bit, the post 2007 redesigned WS467 wall switch cannot be turned on from the off state by a dim or bright alone, according to the Heyu man page. There are other commands that change the light intensity. According to the man page installed along with the program

    Direct Command listing (H = Housecode, HU = Housecode|Units):
    CommandAction
    dimb HU <level>Dim to <level> (1-22) after full bright
    obdim HU <level>Dim to <level> after on and full bright
    brightb HU <level>Brighten by <level> (1-22) after full bright

    These commands are actually macros: sequences of 'dim', 'bright', and 'on' commands. Here is what heyu monitor (using a second SSH session into the Raspberry Pi) reveals:

    command: heyu dimb b3 16 05/24 23:10:54 sndc addr unit 5 : hu B3 (_no_alias_) 05/24 23:10:55 sndc func Bright : hc B bright %100 [210+] 05/24 23:10:59 sndc func Dim : hc B dim %80 [~167] command: heyu obdim b3 16 05/24 23:11:37 sndc addr unit 5 : hu B3 (_no_alias_) 05/24 23:11:37 sndc func On : hc B 05/24 23:11:38 sndc func Bright : hc B bright %100 [210+] 05/24 23:11:43 sndc func Dim : hc B dim %80 [~167] command: heyu brightb b3 6 05/24 23:12:42 sndc addr unit 3 : hu B3 (_no_alias_) 05/24 23:12:43 sndc func Bright : hc B bright %100 [210+] 05/24 23:12:48 sndc func Bright : hc B bright %27 [~57]

    Looking at these macros, it appears that the man page definitions are a bit confusing as both dimb and obdim actually dim by <level> after turning the lamp on to full intensity. The obdim macro is for use with the redesigned WS467 wall switch which does nothing in response to the 'bright' command if the previous command had been 'off'. Finally, to quote the man page, the "... 'brightb' command (brighten after brightening to 100%) is essentially useless" because it will turn the lamp on at full brightness no matter what value is given to <level> or, in the case of the redesigned wall switch WS467, it will do nothing if the previous command had been 'off'.

    It is now clear that there is no Heyu command to set the absolute level of intensity of an X10 lamp module. Furthermore, not only must our script keep track of the light intensity level, it appears that it must keep track of the state of the module. This will require the use of two user defined variables.

  7. A Lua Script to Control a Lamp Module
  8. These are the steps required to create a user variables. Click on the Setup tab, chose More Options and then User variables.

    Add the user variable named Bright_Level_B3, make sure it is of type Integer, set its initial value to 0 and the click on the Add button.

    Repeat these steps to create another user variable of type integer named Last_Cmnd_B3. It will be used to store the previously sent command to the lamp module. These will be encoded as follows: 0 means 'off', 1 means 'on' and 2 means 'bright' or 'dim'.

    This is the lua script that will be executed by Domoticzto update the lamp module with commands sent to Heyu.

    -- Script to update X10 dimmer switch using Heyu -- Adjust the value assigned to NAME to reflect the dimmer switch name in Domoticz -- Adjust the value assigned to ADDR to reflect the X10 house and device code of the dimmer switch -- Adjust VERBOSE to control messages to Domoticz' log commandArray = {} NAME = 'Test Dimmer' ADDR = 'B3' IDX = '2' VERBOSE = 0 -- 0 = quiet, 1 = logs os.execute commands, 2 = debug if devicechanged[NAME] then cmd = string.lower(string.sub(devicechanged[NAME], 1, 9)) name_level_var = 'Bright_Level_'..ADDR last_cmd_var = 'Last_Cmd_'..ADDR wanted_percent = otherdevices_svalues[NAME] current_percent = uservariables[name_level_var] last_cmd = uservariables[last_cmd_var] -- 0 = off, 1 = on, 2 = dim/bright if (VERBOSE > 1) then print('devicechanged['..NAME..']: '..devicechanged[NAME]) print('cmd: ' .. cmd) print('otherdevices_svalue['..NAME..']: ' .. wanted_percent) print('user variable ('..name_level_var..'): ' .. current_percent) print('user variable ('..last_cmd_var..'): ' .. last_cmd) print('') if (VERBOSE > 2) then LogVariables(_G,0,''); end end if (cmd == 'off') then cmd = 'sudo -u pi /usr/local/bin/heyu off '.. ADDR os.execute(cmd) commandArray['Variable:'..last_cmd_var] = "0" if (VERBOSE > 0) then print(cmd) end if (VERBOSE > 1) then print("commandArray['Variable: '"..last_cmd_var.."] = 0") end elseif (cmd == 'on') then cmd = 'sudo -u pi /usr/local/bin/heyu on ' .. ADDR os.execute(cmd) commandArray['Variable:'..last_cmd_var] = "1" -- turns lamp to full on, now prepare to dim to previous value cmd = 'set level' wanted_percent = current_percent current_percent = 100 if (VERBOSE > 0) then print(cmd) end if (VERBOSE > 1) then print("commandArray['Variable: '"..last_cmd_var.."] = 1") print('current_percent now: '..current_percent) print('wanted_percent now: '..wanted_percent) end --fall through to 'set level' to dim lamp end if (cmd == 'set level') then wanted_level = math.ceil(wanted_percent * 0.22) current_level = math.ceil(current_percent * 0.22) commandArray['Variable:'..name_level_var] = tostring(wanted_percent) if (last_cmd == 0) then -- last command off, so bright will turn light full on, do it -- preemptively and the dim down cmd = 'sudo -u pi /usr/local/bin/heyu on ' .. ADDR os.execute(cmd) if (VERBOSE > 0) then print(cmd) end current_level = 22 end delta = wanted_level - current_level if (VERBOSE > 1) then print('wanted heyu dim level (0..22): ' .. wanted_level) print('current heyu dim level (0..22): ' .. current_level) print('delta (-22..22): ' .. delta) end if delta > 0 then action = 'bright ' elseif delta < 0 then action = 'dim ' delta = -delta end if delta ~= 0 then cmd = 'sudo -u pi /usr/local/bin/heyu '..action.." "..ADDR.." ".. delta os.execute(cmd) commandArray['Variable:'..last_cmd_var] = "2" if (VERBOSE > 0) then print(cmd) end if (VERBOSE > 1) then print("commandArray['Variable:'"..last_cmd_var.."] = 2") end end end end return commandArray

    The script begins by assigning only the first 9 letters of the changed device state, converting them to lower case, to a variable named cmd. Three other local variables are assigned values: wanted_percent set to the changed device svalues, current_percent set to the value of the user defined variable Bright_Level_B3, and last_cmd set to the value of the user defined variable Last_Cmd_B3.

    The action performed by the script depends on the content of cmd. The 'off' command is the simplest to handle. The built-in procedure os.execute will be invoked to pass on the command 'sudo -u pi /usr/local/bin/heyu on B3' to the operating system. This is obviously a Linux command: heyu is being executed with root privileges as user pi. Then the user variable Last_Cmd_B3 is set to 0.

    If the command is 'on', then something slightly more complex needs to be done. Recall that Domoticz will send this command only if the bulb was off. The first step is to invoke Heyu to turn the module on and the user variable Last_Cmd_B3 is set to 1. The on command will brighten the bulb to full intensity, not to the intensity saved in current_percent. So wanted_percent is set to current_percent, and then current_percent is updated to 100 reflecting the fact that the bulb is fully on. Finally cmd is changed to 'set level'. Execution of the script goes on and the test (cmd == 'set level') will be true so that that part of the script will then adjust the lamp level.

    If the command is 'set level' then the intensity of the bulb must be adjusted. Remember that the X10 lamp module has 22 discrete on levels, 23 if completely off is counted. So the wanted and current intensity measured in percentage are converted to levels (0-22) and stored in the local variables wanted_level and current_level. The difference is saved in the local variable delta. If delta is not 0 then either a 'bright' or a 'dim' command is passed on to Heyu as appropriate. Before delta is calculated, there is a slight bit of legerdemain. Recall that if the module is in an 'off' state, a 'bright' or 'dim' command will turn the light bulb full on and the level will be ignored. So we pass an 'on' command to Heyu immediately, and set current_level to 22 indicating that the lamp is fully on. That way the calculation of delta in the following line will be correct.

    I will conclude this discussion of this script with three observations. At first it may be amusing, but it will probably be tiresome after a while, to see the light bulb go full on before dimming to the desired level. Nevertheless, it most often gets there. Which brings me to my second observation: sometimes it does not get there. Last_Cmd_B3 stores the last command sent to the module and this may not be the current state of the module. To see what I mean, dim the bulb to a low level using Domoticz. Unplug the module and then plug it back in and slightly increase the brightness of lamp slightly with Domoticz again. The bulb will go to full intensity and will not dim down to the desired level. The last observation is that this problem cannot be solved even by querying Heyu about the state of the module. It gets confused by this unplugging and plugging scenario.

    pi@domopole:~ $ heyu off b2 pi@domopole:~ $ heyu on b2 pi@domopole:~ $ heyu dim b2 18 pi@domopole:~ $ heyu offstate b2 0 -- lamp is not off pi@domopole:~ $ heyu onstate b2 1 -- lamp is on pi@domopole:~ $ heyu dimstate b2 1 -- lamp is dimmed Unplug the module and then plug it back in. The lamp will be off and not in a dimmed state, but i@domopole:~ $ heyu offstate b2 0 -- heyu still thinks the lamp is not off pi@domopole:~ $ heyu onstate b2 1 -- heyu thinks the lamp is on pi@domopole:~ $ heyu dimstate b2 1 -- heyu thinks lamp is dimmed
    This is not a farfetched scenario, it describes what can happen during a power failure.

  9. A Bash Script to Update Domoticz
  10. The final step is to create bash scripts that will update Domoticz when the lamp module is dimmed or brightened with an X10 controller.

    pi@domopole:~ $ nano .heyu/dim_b3.sh
    #!/bin/bash # Script to update X10 dimmer module status in Domoticz when the level has been changed manually # # Adjust the value assigned to IDX to reflect the module idx number in Domoticz # Adjust the value assigned to ADDR to reflect the module house and device code # IDX=2 ADDR=B3 # get the module's current dim level LEVEL=`sudo -u pi heyu dimlevel $ADDR` # debugging messages #echo 'LEVEL = ' $LEVEL >> debug_dim.log & #curl "http://192.168.0.22:8080/json.htm?type=command&param=addlogmessage&message=Updating%20B1%20to%20level%20$LEVEL" # Update the Domoticz user variable holding the dim level. This MUST be done before updating the device to avoid action curl "http://192.168.0.22:8080/json.htm?type=command&param=updateuservariable&vname=Bright_Level_$ADDR&vtype=0&vvalue=$LEVEL" # This will update the slider (svalue) of the Domoticz device, but when the lua script is called, there will be # no action, the calculated change in dim level, delta, will be 0. curl "http://192.168.0.22:8080/json.htm?type=command&param=switchlight&idx=$IDX&switchcmd=Set%20Level&level=$LEVEL"

    This is a three line script:

    It is important to do these steps in order. That is the way to avoid an unwanted loop that will cause the slider (and lamp) will be more or less quickly go to full on or full off whenever it is changed. It is not necessary to update the Last_Cmd_B3 variable, Domoticz, or more to the point, the above lua script will do that.

    Make sure that the script is executable:

    pi@domopole:~ $ sudo chmod +x .heyu/dim_b3.sh

    The last step is to modify the Heyu configuration file to call on the bash script when pertinent the power line command from the X10 controller is detected:

    pi@domopole:~ $ nano .heyu/x10config
    ... # B3 is a dimmer switch: do not set the device's svalue # (which is the dim level) when updating on/off status # SCRIPT B3 on rcvi :: curl "http://192.168.0.22:8080/json.htm?type=command&param=udevice&idx=2&nvalue=1" SCRIPT B3 off rcvi :: curl "http://192.168.0.22:8080/json.htm?type=command&param=udevice&idx=2&nvalue=0" SCRIPT B3 dim rcvi :: /home/pi/.heyu/dim_b3.sh SCRIPT B3 bright rcvi :: /home/pi/.heyu/dim_b3.sh

    The bash script will not work if the Domoticz web site is password protected and if local networks have not been defined. In that case, you can pass on the user name and password in the URL request. Look at Authorization_with_parameters on the Domoticz API/JSON URL's Wiki page for details.

  11. New LM465 Lamp Modules and Presets
  12. What about newer (post 2007) LM465 Lamp Modules? And couldn't preset dim values be used?

    Good questions. Since only post 2007 LM465 lamp modules respond to extended codes, these really boil down to one question. Unfortunately, the one newer LM465 module I had broke down very quickly after I got it, so that I am not able to test any ideas I might have about the newer lamp modules. <diatribe>Typical X10 quality control from what I can see, the RF portion of a second generation CM15A USB computer interface packed it in also. This is one the reasons I decided to rebuild my home automation system around a different technology.</diatribe>

    The following paragraph from the document entitled New LM465.txt certainly hints at the the possibility of avoiding the 'full on before dimming' behaviour.

    The new design has a "resume" feature similar to the LM14A, i.e., the ability of returning from Off to the previously set brightness level. However this feature is not called into play when the module receives an On signal but when it receives a Dim signal. The dimming continues from that point, so to repeatedly turn Off and then resume the exact same level, it is necessary to follow the Dim signal with a Bright signal of the same magnitude. If the new design receives an On signal when Off, it ramps up gradually over about two seconds, but to full brightness regardless of the previous level.

    I take that to mean the following:

    pi@domopole:~/domoticz/scripts/lua $ heyu off b3 -- completely off at once to 0% intensity (0/22) pi@domopole:~/domoticz/scripts/lua $ heyu on b3 -- completely on at once to 100% intensity (22/22) pi@domopole:~/domoticz/scripts/lua $ heyu dim b3 12 -- dims by 12 steps (12/22 = 55%) to 45% intensity (22-12)/22 pi@domopole:~/domoticz/scripts/lua $ heyu off b3 -- completely off at once to 0% intensity (0/22) pi@domopole:~/domoticz/scripts/lua $ heyu dim b3 1 -- lamp resumes pre off intensity then dims by 1 to 41% (22-12-1)/22 pi@domopole:~/domoticz/scripts/lua $ heyu bright b3 1 -- lamp brightens by one level to attain pre off intensity of 45%

    I leave it as an exercise to the reader with a newer LM465 to rewrite the scripts to take advantage of this functionality. If someone wants to "donate" a working newer LM465 module, then I would not refuse it. Send me an e-mail by clicking on my name at the bottom of this page. Hurry, because I will no longer have a working CM11A interface by early next August and it will not be feasible to borrow it from my sister as she lives a thousand kilometres away.

  13. Downloads
  14. The Bash and Lua scripts as well as the Heyu configuration file are available for downloads on the Downloads page or below: