Saturday 22 December 2018

Autogate Adventures: Voice Control via Google Home. Part 3 of 3: Debugging Adafruit MQTT disconnects

One reason why I love doing software is the instant gratification: the speed at which you can try out something. And when a prototype works, the impulse is to immediately move to something else.

But a prototype is seldom suitable for customer use, and there are long weeks of grinding work where you monitor for failures and fix them as they appear.  A robust system is usually achieved sometimes years of constant fixing. If you are lucky enough to have a customer for your prototype the key is responsiveness- you need to fix it as fast as possible.

In Part 2, the MQTT-based ESP8266 IoT autogate was usable in less than a day. But it would not respond after a few days, usually after I disconnected the ADSL modem during thunderstorms (we get really spectacular lightning strikes and the ADSL modem is usually the first to go).

I can get the IoT Autogate to work again by power-cycling it, but it really looked bad next to the Sonoff switches, which seemed to recover after no more than 30 minutes as long as the broadband connection is restored. Since the Sonoff is also based on the ESP8266 and they both used the same ADSL modem and routers the problem must be the software.

Debug setup: ESP-01S on its CH340 USB Adapter


To debug the software, it would be nice to get to the Arduino IDE Serial Monitor, but since the  MQTT-based ESP8266 IoT autogate is out of reach inside the autogate enclosure. One way is to program another ESP-01S and run it on its CH340 USB adapter. When Google Assistant triggers the autogate (via IFTTT and Adafruit MQTT) both devices will now respond.

You will get messages like these every 5 seconds (Wife being the other House Member):

Wife Connecting to MQTT... Wife MQTT Connected!
Connecting to MQTT... MQTT Connected!
Wife Connecting to MQTT... Wife MQTT Connected!
Connecting to MQTT... MQTT Connected!

It would be nice to monitor it while at work, and since Adafruit's MQTT broker allows an IoT device to send messages to a feed, and since I have to make an IoT report it state back to Google Home at some point, I might as well use it to report debug messages.

In Adafruit-speak, when we want Google Assistant to control a device the ESP8266 IoT program (ie its Arduino sketch) needs to 'subscribe' to a feed. When it is sending data back to Google Assistant, it needs to 'publish'. So publish we shall. I added another feed, 'log' to my Adafruit account. 

Adafruit_MQTT_Publish mqtt_log = Adafruit_MQTT_Publish(&mqtt, AIO_USERNAME"/feeds/log");

No setup() code is necessary and you simple insert the following code snippet in loop():


{
    char logmsg[50]; 

          sprintf(logmsg, "YOUR_LOG_MESSAGE %d", Your_Log_Number); 
          if (! mqtt_log.publish(logmsg)) {
            Serial.println(F("publish Failed"));
          } else {
            Serial.println(F("publish OK!"));
          } 
}

Now log into your Adafruit account, navigate to

https://io.adafruit.com/youraccountname/feeds/log

And you have a remote Serial Monitor. It is that simple.

Logging debug messages in the Adafruit feed


But what of the ESP8266 MQTT IoT device going unresponsive problem? It turned out that just having the ESP8266 publish regularly (preferably not less than once every five minutes) made the problem go away. 

For now I can only guess when a device goes silently unresponsive, the MQTT broker may sometimes be unaware that the connection is broken, ie the mqtt.connect() function returned 0 (meaning connected) even when it is not. On the other hand we know when a periodic MQTT report does not arrive on schedule.

Notice I have disabled the "watchdog reset" to make the IoT keep trying to connect without restarting the WiFi code.    

void loop() {
  MQTT_connect();

    .
    .
    .
 publish++; 

  if (publish >= 20) //  ping every 4min
  {
    // Now we publish stuff! ping limit is 300s so 60s seems plenty
    Serial.print(F("\nPinging adafruit mqtt ... State "));
    Serial.print(State);
    Serial.print("...");
    sprintf(logmsg, "State is %d %d", State, Wife_Autogate_State);
    if (! mqtt_log.publish(logmsg)) {
      Serial.println(F("publish Failed"));
    } else {
      Serial.println(F("publish OK!"));
    }
    publish = 0;
  }
}

void MQTT_connect() {
  int8_t ret;

  // Stop if already connected.
  if (mqtt.connected()) {
    return;
  }
  
  Serial.print("Connecting to MQTT... ");
  
  uint8_t retries = 3;
  
  while ((ret = mqtt.connect()) != 0) { // connect will return 0 for connected
    Serial.println(mqtt.connectErrorString(ret));
    Serial.println("Retrying MQTT connection in 5 seconds...");
    mqtt.disconnect();
    delay(5000);  // wait 5 seconds
    //  retries--;
    if (retries == 0) {
      // basically die and wait for WDT to reset me
      while (1);
    }
  }
  Serial.println("MQTT Connected!");
}

So, for now the ESP8266 MQTT IoT Autogate goes back for more testing. We still do not know exactly what caused the problem- it might still be there for all we know. At least it is reconnecting when the Internet comes back, just like the Sonoff devices. Dave has some good work on ESP8266 reconnection problems. There is some talk here that it may be due to problems in the Expressif SDK.

By the way if you suspect the ESP8266 Adafruit MQTT library itself, you can turn on its debug messages:
 #define MQTT_DEBUG

Happily my customer (in this case Home Member Wife) does not seem to mind. We quickly got used to the Google Home/Assistant voice control of the autogate. It is especially handy when you have your hands full driving or carrying the groceries. Now if only I can get Google Assistant to recognize the dogs' voices ...

Happy Christmas, and Happy Trails.   

Sunday 9 December 2018

Driving Inductive loads with the Sonoff Basic

The Sonoff Basic Smart Home power switch is very handy and has a claimed output rating of 10A at 230Vac. The basis for this is the output relay rating, a SONGLE SRD-05VDC-SL-C.

SONGLE SRD-05VDC-SL-C Miniature Relay


Now in tropical Malaysia, one of the most handy home appliances is a fan. There is usually quietly ruthless jostling for fans so there is usually at least one in use.

Poppy the dog usually wins pole position


One of my favourites is a Pensonic PFF-20B, rated at a modest 200W or  0.87A.

Pensonic PFF-20B 230V 200W

Trouble is the Sonoff Basic does not last long switching the PFF-20B, even at a modest 1A load. The contacts often get stuck, ie the relay might turn on but fail to turn off. A sharp knock on the Sonoff Basic will often jar the contacts back into position, but it will invariably fail again. Inductive loads cause arcing of the contacts resulting in damaging micro-welds.

On the other end of the spectrum is a magnetic contactor, like the LSIS MC-9b. Modestly rated at 9A for inductive loads (AC3), it is a veritable King Kong next to the SONGLE SRD-05VDC-SL-C.

LSIS MC-9b 230V 9A (AC3)

The obvious answer is to use the Sonoff Basic to drive the MC-9b. The MC-9b has a 230Vac input relay coil. I picked one up at RM36 from my local store, but you can find them online for as low as RM28. The wiring is straightforward, so I'll dispense with the schematics.


The contactor jumps about quite a bit, so it is necessary to mount it securely and to ensure the jolting does not transfer too much to the Sonoff Basic.

Here's a youtube video of a voice controlled Pensonic PFF-20B. The MC-9b did not faze Poppy at all. In Malaysia, it takes a lot to give up one's spot in front of the fan.

[2020-07-04 update] 



Happy Trails.

Friday 30 November 2018

Autogate Adventures: Voice Control via Google Home. Part 2 of 3: ESP8266, MQTT and IFTTT

In Part 1, the autogate became a smarthome device, controlled by voice via my Google Home speaker or Google Assistant on my phone. However every time we went out, it needed four voice commands to open and close the gate.

I can also use an ESP8266 relay board, MQTT and IFTTT like I did for my ancient lantern, but there were a couple of problems. First the MQTT code would stop working after a few days. Secondly I had not figured out how to share the device on Google Home with my wife.

Top: 230Vac to 5V@1A power supply. Bottom: ESP-01S with 1-channel relay adapter

It did not take long to find out why the program stopped running. The routine MQTT_connect() is run in a loop and normally returns non-zero for a valid connection to MQTT server. However if the connection should drop (maybe WiFi interference, power outtage affecting the router, it tries to reconnect 3 times and then goes into an infinite loop waiting for a watchdog timer.

The watchdog timer is enabled by default, and is serviced by various routines in the library, so there is really no good way to use it except as a delayed reset. There is a good article on it here.

  while ((ret = mqtt.connect()) != 0) { // connect will return 0 for connected
    Serial.println(mqtt.connectErrorString(ret));
    Serial.println("Retrying MQTT connection in 5 seconds...");
    mqtt.disconnect();
    delay(5000);  // wait 5 seconds
    retries--;
    if (retries == 0) {
      // basically die and wait for WDT to reset me
      while (1);
    }
  }

The reason for the failure seems to be mqtt.connect() or perhaps the WiFi router sometimes cannot detect there is no longer a connection. For now the best workaround seems to be to have an active MQTT input, that is to regularly 'publish' to the MQTT broker Adafruit. After a few minutes (normally less than 30) mqtt.connect() will eventually work and the device will reconnect.

Getting Google Home to share the device took a little more time. On the face of it, IFTTT allows me to share: I can publish my Applet for unlimited sharing. But that is too much sharing: letting a random user open my gate from the Internet is a not good idea.

I get my MQTT service from Adafruit, (which IFTTT needs to link the ESP8266 to Google Home) and I can share just my Adafruit feed for the autogate with my wife. Indeed she can activate the autogate from the Adafruit App, but Google Home will not link to a shared feed. Furthermore the Adafruit share turned out to be a global share as well, again an unacceptable security risk.

It turned out that if I got the wife to register with IFTTT and Adafruit, and if I duplicated the ESP8266 code using my wife's Adafruit account and secret key (AIO) the device will support both Adafruit feeds.

Adafruit_MQTT_Client mqtt(&client, AIO_SERVER, AIO_SERVERPORT, AIO_USERNAME, AIO_KEY);
Adafruit_MQTT_Client wife_mqtt(&wife_client, AIO_SERVER, AIO_SERVERPORT, AIO_WIFENAME, AIO_WIFEKEY);

// Setup feeds Adafruit_MQTT_Subscribe lantern = Adafruit_MQTT_Subscribe(&mqtt, AIO_USERNAME"/feeds/frontgate");
Adafruit_MQTT_Subscribe wife_autogate = Adafruit_MQTT_Subscribe(&wife_mqtt, AIO_WIFENAME"/feeds/frontgate");

Converting the Open/Close Google Home commands to autogate pulses is now easy:

while ((subscription = mqtt.readSubscription(2000))) {
    if (subscription == &gate) {
      Serial.print(F("Got: "));
      Serial.println((char *)gate.lastread);
      int gate_State = atoi((char *)gate.lastread);
      Serial.print("Autogate openclose ");
      Serial.println(gate_State);
      digitalWrite(0, 1); // Issue a command to toggle gate
      delay(500);
      digitalWrite(0, 0); // Autogate CPU input is monostable. Issue command to reset  
    }
  }

Tight fit: clockwise from top left: autogate control board, RF remote control module, 12V battery and ESP8266 relay module

Here's a little youtube video of the result.

I am still a little unhappy with it: I would it like to feedback (ie 'publish' in MQTT-speak) actual gate open/close state, and I would have to manually dismantle and reprogram it should my parents come to visit. Also it would be nice to keep a log of the autogate usage, but perhaps that is a future project, an MQTT server. But in the spirit of DevOps, let's just put it up and look for those all-important gotchas that are so critical to a product's eventual success.

There you have it- a voice-controlled autogate. I have not looked very hard, but with a little luck, this should be one of the first homebrew Google Home autogate smart remotes.

Happy Trails.

Thursday 29 November 2018

Autogate Adventures: Voice Control via Google Home. Part 1 of 3: Sonoff Basic

About a year ago I decided to maintain the autogate myself, and naturally could not resist adding a Bluetooth remote function, in addition to its keychain RF remote control. Later I added a Raspberry Pi Zero W IoT gateway, which let me control it from my smartphone. The next step would be to make it a smarthome device - voice control via my Google Home speaker or Google Assistant on my phone.

The obvious way would be to use an ESP8266 relay board, MQTT and IFTTT like I did for my ancient lantern, but there were a couple of problems. First the MQTT code would stop working after a few days. Secondly I had not figured out how to share the device on Google Home with my wife. So I though I would use a commercially-available IoT device, a Sonoff Basic.

Sonoff Basic was RM23 from Lazada
There was no trouble sharing the Sonoff Basic with home members and it was a lot more reliable. I had been using it to control some of the lights in the house. The Basic takes 230Vac input and puts out 230Vac when it turns on. But the autogate needed a dry (ie voltage-free) contact, so I simply added a 230Vac relay , an Omron MY2N.

Omron MY2N 230Vac coil relay, RM10 
I found difficult to mount in the available space in the autogate enclosure, so eventually replaced it with a solid-state relay, the Omron G3TA-IAZR02S.

The OMRON G3TA-IAZR025 was a whopping RM45
Sonoff Basic converted to output dry contact
It worked ... here's a video ... sort of. Trouble was the autogate controller needed a pulse to toggle the gate open and shut. The Sonoff Basic puts out a steady on or off command. Opening the gate was easy:

"OK Google Open autogate"

Now I need to deactivate the signal to the autogate controller, so

"OK Google Close autogate"

But the gate is still open, so now it is time to close it:

"OK Google Open autogate"

And then

"OK Google Close autogate"

It took all of 15 minutes before I got tired of it. I guess it is time for Part 2.


Saturday 27 October 2018

Voice-controlled ESP8266 IoT AC Mains Power Extension Part 3 of 3

230Vac Fluorescent battery-backed lantern  
I have had this lantern for over 20 years. It is obsolete- dual 230Vac fluorescent lamps running from a 6V 7Ah lead-acid battery. Yet it has never failed - I replaced the battery many times but the lamps never blew. It is always plugged, in float-charging the battery and when the mains power goes the lamps automatically comes on.

It would be nice to have a bit of extra light sometimes during normal conditions, so why not mount it on the ESP8266 IoT Extension? And it would even be nicer if I could control it by voice command.

I do not have Google Home or Amazon Echo yet, but my Android smartphone does have Google Assistant which will capture my voice commands. I just need to forward the command words to the ESP8266 IoT device.

Robosapien has a great writeup on using the ESP8266 with Google Assistant, IFTTT and Adafruit IO. I really wish I could improve on it, but I believe Robosapien has had the last word on the subject. I simply followed all the steps and replaced his hardware with my ESP8266 IoT AC Mains Power Extension.

The process just took half a day, and I now have a voice controlled 20-year old lantern!

Update (2018-11-27): after running the IoT lantern a few days, I noticed a new WiFi access point with an extremely strong signal:

iwlist wlan0 scan
          Cell 04 - Address: xx:xx:xx:xx:xx:xx
                    Channel:10
                    Frequency:2.457 GHz (Channel 10)
                    Quality=70/70  Signal level=-34 dBm
                    Encryption key:off
                    ESSID:"ESP_1CC60D"
                    Bit Rates:5.5 Mb/s; 11 Mb/s; 1 Mb/s; 2 Mb/s; 6 Mb/s
                              12 Mb/s; 24 Mb/s; 48 Mb/s
                    Bit Rates:54 Mb/s; 9 Mb/s; 18 Mb/s; 36 Mb/s
                    Mode:Master
                    Extra:tsf=0000000008b54204
                    Extra: Last beacon: 4442ms ago
                    IE: Unknown: 000A4553505F314343363044
                    IE: Unknown: 01088B9682840C183060
                    IE: Unknown: 03010A
                    IE: Unknown: 0706534720010D14
                    IE: Unknown: 32046C122448
                    IE: Unknown: DD0918FE34030100000000

A signal level of -34dBm was stronger than my own access point! In fact I can detect and log into it two houses down, some 150m away through 2 concrete walls.

It turned out to come from the lantern code, and while it does not interfere with its operation, it is also a security risk as it is an open Access Point. As usual the Internet provided an answer; there is a bug in the ESP8266 Arduino library. There is a workaround: you just need to insert WiFi.mode(WIFI_STA) after calling WiFi.begin().

My setup code now looks like:

void setup() {
  Serial.begin(115200);

  pinMode(0, OUTPUT); // 2018-10-26 set gpio to o/p

  // Connect to WiFi access point.
  Serial.println(); Serial.println();
  Serial.print("Connecting to ");
  Serial.println(WLAN_SSID);

  WiFi.begin(WLAN_SSID, WLAN_PASS);
  WiFi.mode(WIFI_STA); // 2018-11-27 remove rogue AP
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  Serial.println();

  Serial.println("WiFi connected");
  Serial.println("IP address: ");
  Serial.println(WiFi.localIP());


  // Setup MQTT subscription for onoff feed.
  mqtt.subscribe(&lantern);
}



Lantern with IoT Power Extension
Now the logic is reversed for the lantern as it comes on when there is no mains power. This I did by entering a data '0' when the IFTTT trigger is on.

Note reversed (data zero) logic
Google Assistant, IFTTT and Adafruit IO all worked first time. Here is a youtube video of it:

Youtube video
There you have it, a voice-controlled IoT lantern. It is not ideal and much work needs to be done on it, but it still works as before without the Internet, and in the spirit of DevOps, just put it out there as fast as we can ...  Happy Trails.

Sunday 7 October 2018

ESP8266 IoT AC Mains Power Extension Part 2 of 3




ESP8288 ESP-01S IoT AC Mains Power Extension, shown with light bulb in E27 adapter
Rather than building separate Internet of Things (IoT) lamps, TVs, speakers, etc I found it much more flexible to make an IoT power extension. This lets me test the various IoT devices at leisure.

In Part 1, half the space is taken up by the IoT power supply. Here, in Part 2, is a proper 1-gang IoT AC mains power extension using the ESP8266 NodeMCU ESP-01S and a very handy 700mA 5V AC mains power supply.


5VDC 700mA adapter from 220VAC 
I bought mine from autobotic at lelong.com for about RM7 each. They were so handy I actually bought up all the available stock in Malaysia (sorry). The 5V 1A version is over twice in size and not as useful.

Assembly is straightforward. The power socket fits nicely over the power module and the ESP-01S relay modle without coming into contact. Note the cable ties to hold the modules in place
Assemble is simple. Note the cable ties used to hold the modules in place. This is advisable as there is Live voltages on the power board and you do not want them in contact with the other bits. The modules should not be exposed from the bottom either, as some of the holes there are quite large.

You may need to drill a hole in the side for the power cord, and maybe enlarge the various holes in the bottom enclosure to fit your cable ties.

Assembled IoT mains power extension
There you have it, a proper IoT AC mains power extension. In Part 3 of this series I hope to detail how to remotely control it from your smartphone or your Google Home smart speaker.

Happy Trails.

Thursday 27 September 2018

ESP8266 IoT AC Mains Power Extension Part 1 of 3

Right socket:: regular UK 230V power socket with builtin USB charger. Left socket: ESP-01S controllerd
Using the same parts as the IoT Porchlight, it is possible to make a remotely-switched AC power extension. This lets you switch almost any low-power electrical device without having to worry about fitting the ESP8266 bits. All you need is a stack of IoT Power Extensions ...

Warning- this project involves hazardous AC mains voltages. Do not attempt this if you are not a qualified electrician/engineer. 


Luckily here in Malaysia we use the UK 230V standard. This means it is easy to buy AC wall socket parts.

Two-gang UK Mains AC wall socket
I salvaged a two-gang wall socket enclosure, and swapped one of the wall sockets with a new type (SIRIM approved) with a built-in USB charger. This solved the problem of the 5V power supply for ESP-01S Relay Module. The charger is rated for 1500mA at 5V. The ESP-01S Relay Module required 60mA on standby and 130mA when the relay is on. That leaves plenty of wiggle room for I noticed that these built-in chargers tend to get hot at full load, probably due to the lack of ventilation.

AC mains wall socket with built-in USB charger. I paid RM25 for mine.

It is not difficult to fit the ESP-01S Relay Module into the socket enclosure. By fitting it face-down over the opening in the middle, this makes sure the AC mains on the PCB cannot be accessed from outside the enclosure. It also makes the ESP-01S's blinkenlights visible and gives its antenna an unobstructed view of the WiFi modem router.
  
Fitting in the ESP-01S Relay Module
A cheap (RM2) phone charger cable from Alibaba is used to connect the Relay Module to the charger output.

Wired thus, the right socket is a normal, manually switched AC mains socket at full 13A rating. Only the left socket can be remotely switched and although the load of 10A is claimed on the little 5V relay, it is prudent to switch no more than a 2A load maximum. Anything more the little relay tends to get stuck, especially with inductive loads, and most switched-mode power supplies (even little USB chargers) are inductive loads.

There you have it: an IoT (ie remotely-switched) AC mains power socket. Happy Trails.

Monday 6 August 2018

RS-485 Modbus IoT Gateway using ESP8266 NodeMCU ESP-12E: TCP/IP Slave Part 2 of 3

Clockwise from top: USB3 externally-powered hub, RS-485 dongle, RS-485 to TTL serial PCB and ESP-12E ESP8266 NodeMCU module

Modbus TCP/IP to RS-485 passthrough gateway for RM25.


Part 1 describes an ESP-12E Modbus RTU Master using RS-485 interface. It can read and write RS-485 Modbus devices but it uses the ESP-12E debug serial port to do so. This works if the ESP-12E itself is the host controller, but Modbus masters usually have a lot more horsepower.



To be really useful, we can also make the ESP-12E a TCP/IP Modbus Slave. It still can work as a host, but this will make it a Modbus TCP/IP to RS485 "passthrough" gateway. A real Modbus host, say a desktop or Industrial PC can then orchestrate a whole bunch of Modbus devices to control a whole buildings' services.

A real TCP/IP to RS-485 Modbus gateway is some RM680

Of course the ESP-12E is not as powerful as a regular Modbus TCP/IP to RS485 passthrough but at RM25 it is an ideal way of retrofitting IoT functionality to Modbus devices.

If we replace the desktop with a cloud-based server we can scale  
As usual someone has already provided the required code. yaacov's ModbusSlaveTCP made an excellent template. I downloaded it as a zip file and copied yaacov's files from ArduinoModbusSlaveTCP-master/src into my Arduino IDE directory <your Linux account>/Arduino/libraries/ModbusSlaveTCP/

Then I expanded the sketch in Part 1:

#include <ESP8266WiFi.h>
#include <ModbusSlaveTCP.h>

const char* ssid = "YourAccessPoint";
const char* pass = "StrongPassword";
IPAddress staticIP(10,0,0,100);
IPAddress gateway(10,0,0,1);
IPAddress subnet(255,255,255,0);


// slave id = 1, rs485 control-pin = 8, baud = 9600
#define SLAVE_ID 1
// Modbus object declaration
ModbusTCP slave(SLAVE_ID);

#include <ModbusMaster232.h>
#include <SoftwareSerial.h>  // Modbus RTU pins   D7(13),D8(15)   RX,TX
// MAX485 half duplex control lines
#define not_RE 14 // D5. Enable receiver, active low
#define DE 12 // D6  Enable Transmitter, active high

// Instantiate ModbusMaster object as slave ID 1
  ModbusMaster232 node(1);

void setup() {
  pinMode(not_RE, OUTPUT);
  pinMode(DE, OUTPUT);
  // default to transmit mode to reduce noise
  digitalWrite(not_RE, HIGH); // disable receiver
  digitalWrite(DE, HIGH); // enable transmitter
 
  Serial.begin(9600);
  delay(100);
  node.begin(9600);  // Modbus RTU
  delay(100);
 
   
    /* Connect WiFi to the network
     */
    Serial.print("Connecting to ");
    Serial.println(ssid);
    WiFi.disconnect();
    WiFi.hostname("ModbusTCPslaveRS485master");
    WiFi.mode(WIFI_STA);
    WiFi.begin(ssid, pass);
    WiFi.config(staticIP, gateway, subnet); // Static IP. Not required for dhcp

    int wifi_loop = 0;
    while (WiFi.status() != WL_CONNECTED) {
        delay(1000);
        Serial.print(".");
        /*
        if (wifi_loop++ == 10)
        {
            Serial.println("Reconnecting ...");
            WiFi.disconnect();
            WiFi.mode(WIFI_STA);
            WiFi.begin(ssid, pass);
            delay(1000);
            wifi_loop = 0;
        }
        */
    }
    Serial.println(WiFi.localIP());
    /* register handler functions
     * into the modbus slave callback vector.
     */
    slave.cbVector[CB_WRITE_COIL] = writeDigitlOut;
    slave.cbVector[CB_READ_DISCRETE_INPUT] = readDigitalIn; //    
    slave.cbVector[CB_READ_COILS] = readDigitalIn;
    slave.cbVector[CB_READ_REGISTERS] = readAnalogIn;
    slave.cbVector[CB_WRITE_MULTIPLE_REGISTERS] = writeAnalogOut; // cmheong
     
    /* start slave and listen to TCP port 502
     */
    slave.begin();
   
    // log to serial port
    Serial.println("");
    Serial.print("Modbus ready, listen on ");
    Serial.print(WiFi.localIP());
    Serial.println(" : 502");
}

int loop_i = 0;
uint16_t readDiscreteInputs[10];
int Mdelay = 10; // from 5


void loop() {

  /*
  node.readDiscreteInputs(loop_i, 1);
  readDiscreteInputs[loop_i] = node.getResponseBuffer(0);
  node.clearResponseBuffer();
  Serial.print("[");
  Serial.print(loop_i);
  Serial.print("] ");
  Serial.print(readDiscreteInputs[loop_i]);
  if (++loop_i >= 10)
  {
    Serial.println("");
    loop_i = 0;
  }
  */
  // delay(Mdelay); // no need for delay(5) since we print 5 char at 9600

    /* listen for modbus commands con serial port
     *
     * on a request, handle the request.
     * if the request has a user handler function registered in cbVector
     * call the user handler function.
     */
  slave.poll();
}

/**
 * Handel Force Single Coil (FC=05)
 * set digital output pins (coils) on and off
 */
void writeDigitlOut(uint8_t fc, uint16_t address, uint16_t status) {
    digitalWrite(address, status);
}

/**
 * Handel Read Input Status (FC=02/01)
 * write back the values from digital in pins (input status).
 *
 * handler functions must return void and take:
 *      uint8_t  fc - function code
 *      uint16_t address - first register/coil address
 *      uint16_t length/status - length of data / coil status
 */
void readDigitalIn(uint8_t fc, uint16_t address, uint16_t length)
{
    int data = 0;
 
    // read digital input
    Serial.printf("digital input bytes fc %02x at address %04x length %d data ", fc, address, length);
    node.readDiscreteInputs(address, length);

    for (int i = 0; i <= (length-1)/8; i++) // cmheong 2018-08-06
    {
      data = node.getResponseBuffer(i);
      slave.writeCoilsToBuffer(i, (uint8_t) data); // digitalRead(address + i));
      Serial.printf(" %x", node.getResponseBuffer(i));
    }

    delay(Mdelay);
   
    Serial.println("");
    node.clearResponseBuffer();
}

/**
 * Handel Read Input Registers (FC=04/03)
 * write back the values from analog in pins (input registers).
 */
void readAnalogIn(uint8_t fc, uint16_t address, uint16_t length) {
    // read analog input
    for (int i = 0; i < length; i++) {
        slave.writeRegisterToBuffer(i, analogRead(address + i));
    }
}

// cmheong 2018-07-31 write_registers()
void writeAnalogOut(uint8_t fc, uint16_t address, uint16_t length)
{
    Serial.printf("analog output bytes at address %04x length %d data ", address, length);
    for (int i = 0; i < length; i++)
    {
        Serial.printf("%x ", slave.readRegisterFromBuffer(i));
        Serial.println("");
        node.send(slave.readRegisterFromBuffer(i));
        // node.writeSingleRegister(address, slave.readRegisterFromBuffer(i));
        delay(Mdelay);  
    }
    node.writeMultipleRegisters(address, length);
}
  
I only tested 'Read Discrete Registers' (function code 2) and 'Write Multiple Registers' (function code 16) on a real Modbus RTU device, but you get the idea. yaakov's code did not process function code 2 properly, so I modifiled ModbusSlaveTCP.cpp of his library:

        case FC_READ_DISCRETE_INPUT: // read input state (digital in)
            address = word(bufIn[MLEN + 2], bufIn[MLEN + 3]); // coil to set.
            length = word(bufIn[MLEN + 4], bufIn[MLEN + 5]);

            // sanity check.
            if (length > MAX_BUFFER) return 0;

            // check command length.
            if (lengthIn != (MLEN + 6)) return 0;

            // build valid empty answer.
            lengthOut = MLEN + 3 + (length - 1) / 8 + 1; // cmheong 2018-08-06
            bufOut[MLEN + 2] = length;  // cmheong 2018-08-06

            // clear data out.
            memset(MLEN + bufOut + 2, 0, bufOut[2]);  // cmheong 2018-08-06

            if (cbVector[CB_READ_DISCRETE_INPUT]) // cmheong 2018-08-02
            {
                cbVector[CB_READ_DISCRETE_INPUT](fc, address, length);
            }
            break;

And added a new function:

void ModbusTCP::writeCoilsToBuffer(int offset, uint8_t state)
{
    int address = MLEN + 3 + offset;

    bufOut[address] = state;
}

 The ESP-12E will connect to your WiFi and use a fixed IP address (change it to suit your own address assignments) 10.0.0.100. To test it, I used my laptop to connect to the same WiFi access point. I then modified pymodbus's excellent synchronous-client.py thus:

from pymodbus.client.sync import ModbusTcpClient as ModbusClient
client = ModbusClient('10.0.0.100', method='rtu', port=502) # 2018-07-29

The test code is:
rr = client.read_discrete_inputs(1,1,unit=0x01)
if rr != None :
    print "\nread discrete inputs from", coils, rr.bits, '\n'

rq = client.write_registers(0x1001, [0x001f]*1, unit=0x01)
if rq != None :
    print "\write holding_registers from", 10, rq, '\n'

A sample working output is:

root@aspireF15:/home/heong/EMS/pymodbus/pymodbus-master/examples/current$python ./esp8266-tcpipclient.py 0x01
DEBUG:pymodbus.transaction:Current transaction state - IDLE
DEBUG:pymodbus.transaction:Running transaction 1
DEBUG:pymodbus.transaction:SEND: 0x0 0x1 0x0 0x0 0x0 0x6 0x1 0x2 0x0 0x1 0x0 0x1
DEBUG:pymodbus.client.sync:New Transaction state 'SENDING'
DEBUG:pymodbus.transaction:Changing transaction state from 'SENDING' to 'WAITING FOR REPLY'
DEBUG:pymodbus.transaction:Changing transaction state from 'WAITING FOR REPLY' to 'PROCESSING REPLY'
DEBUG:pymodbus.transaction:RECV: 0x0 0x1 0x0 0x0 0x0 0x4 0x1 0x2 0x1 0x7
DEBUG:pymodbus.framer.socket_framer:Processing: 0x0 0x1 0x0 0x0 0x0 0x4 0x1 0x2 0x1 0x7
DEBUG:pymodbus.factory:Factory Response[ReadDiscreteInputsResponse: 2]
DEBUG:pymodbus.transaction:Adding transaction 1
DEBUG:pymodbus.transaction:Getting transaction 1
DEBUG:pymodbus.transaction:Changing transaction state from 'PROCESSING REPLY' to 'TRANSACTION_COMPLETE'

read discrete inputs from 1 [True, True, True, False, False, False, False, False] 

DEBUG:pymodbus.transaction:Current transaction state - TRANSCATION_COMPLETE
DEBUG:pymodbus.transaction:Running transaction 2
DEBUG:pymodbus.transaction:SEND: 0x0 0x2 0x0 0x0 0x0 0x9 0x1 0x10 0x10 0x1 0x0 0x1 0x2 0x0 0x1f
DEBUG:pymodbus.client.sync:New Transaction state 'SENDING'
DEBUG:pymodbus.transaction:Changing transaction state from 'SENDING' to 'WAITING FOR REPLY'
DEBUG:pymodbus.transaction:Changing transaction state from 'WAITING FOR REPLY' to 'PROCESSING REPLY'
DEBUG:pymodbus.transaction:RECV: 0x0 0x2 0x0 0x0 0x0 0x6 0x1 0x10 0x10 0x1 0x0 0x1
DEBUG:pymodbus.framer.socket_framer:Processing: 0x0 0x2 0x0 0x0 0x0 0x6 0x1 0x10 0x10 0x1 0x0 0x1
DEBUG:pymodbus.factory:Factory Response[WriteMultipleRegistersResponse: 16]
DEBUG:pymodbus.transaction:Adding transaction 2
DEBUG:pymodbus.transaction:Getting transaction 2
DEBUG:pymodbus.transaction:Changing transaction state from 'PROCESSING REPLY' to 'TRANSACTION_COMPLETE'
Modbus device replied!

There you have it, a Modbus TCP/IP to RS485 passthrough gateway for less than RM25. Slap on an AWS or Google Cloud server and you are ready for a free docker microservice demon Modbus host.

Happy Trails!

Friday 27 July 2018

RS-485 Modbus IoT Gateway using ESP8266 NodeMCU ESP-12E Part 1 of 3

Clockwise from top: USB 3.0 powered hub, RS-485 USB dongle, Arduino RS-485 to TTL module and NodeMCU ESP-12E ESP8266
Many remote devices, especially for industrial use come with the RS-485 serial interface and use the Modbus protocol. These industrial remote devices are often easily converted to IoT devices using for example, an IoT Gateway that also has RS-485 interfaces.

The Schneider PM1200 3-phase Power Meter has an RS-485 interface and would make a great IoT device.

Schneider PM1200 Power Meter


Fuji FRENIC 15KW Inverter

The ESP8266 is incredibly tempting as such  an IoT gateway. It is low-cost, low-power and the WiFi module allows for a much safer non-galvanic, isolated data connection. Similar devices like the monster 10-30KW Inverters used in escalators, cranes, etc can be monitored safely from the cloud this way.

As usual I used a low-cost (RM3.38) Arduino RS-485 to serial TTL module. Now this is a 5V device and since the ESP8266 is strictly a 3.3V device, we should really use a logic level converter, but at a pinch it should work at 3.3V, especially if I powered  it from the ESP-12E.

I also used a modified version of the Trialcommand ESP8255 Master Modbus RTU (RS232) sketch. This in turn use the SoftwareSerial library, which essentially determined the wiring diagram.

RS-485 to 5V TTL module 

MOSFET-based Logic Level Converter

  The wiring diagram (sans level converter) is:

       ESP12-E    RS-485 PCB
            3V3
            GND
            D5 ------- *RE
            D6 ------- DE
            D7 ------- RO
            D8 ------- DI

NodeMCU ESP-12E Pinout

The wiring really makes sense when you look at the MAX485E IC datasheet:

Notice the MAX485E is a half-duplex device
I made a little ribbon cable with Moles 0.1" pitch Molex connectors:

Cable assembly. Right, top: data & control, right bottom: rs-485 power. Left: ESP-12E
When assembled the RS-485 module looked like this:


Download the libraries and copy them into your Arduino IDE libraries folder.

$ls -lt /root/Arduino/libraries
total 16
drwxr-xr-x 4 root  root  4096 Apr 26 20:44 Time-master
-rw-r--r-- 1 root  root    87 Feb 27 07:20 readme.txt
drwxr-xr-x 2 heong users 4096 Mar 15  2017 ModBusMaster232
drwxr-xr-x 3 heong users 4096 Mar 15  2017 SoftwareSerial

Since the ESP8266-Modbus-RTU-Master library uses full duplex TTL RS-232 I put in a tiny patch to convert it to half duplex required by the MAX485E. The file is ./libraries/ModBusMaster232/ModbusMaster232^Cpp and the change is in boldface at line 735:

#define not_RE 14 // D5. Enable receiver, active low
#define DE 12 // D6  Enable Transmitter, active high

  u16CRC = 0xFFFF;
  for (i = 0; i < u8ModbusADUSize; i++)
  {
  //Function  crc16  for ESP8266   - PDAControl
    u16CRC = _crc16_update2(u16CRC, u8ModbusADU[i]);
  }
  u8ModbusADU[u8ModbusADUSize++] = lowByte(u16CRC);
  u8ModbusADU[u8ModbusADUSize++] = highByte(u16CRC);
  u8ModbusADU[u8ModbusADUSize] = 0;

  // transmit request
  digitalWrite(not_RE, HIGH); // cmheong Set max485 to transmit
  digitalWrite(DE, HIGH);
  for (i = 0; i < u8ModbusADUSize; i++)
  {
                swSer.print(char(u8ModbusADU[i]));
                delay(2); // cmheong 2018-07-26
  }
  u16CRC = 0xFFFF;
  for (i = 0; i < u8ModbusADUSize; i++)
  {
  //Function  crc16  for ESP8266   - PDAControl
    u16CRC = _crc16_update2(u16CRC, u8ModbusADU[i]);
  }
  u8ModbusADU[u8ModbusADUSize++] = lowByte(u16CRC);
  u8ModbusADU[u8ModbusADUSize++] = highByte(u16CRC);
  u8ModbusADU[u8ModbusADUSize] = 0;

  // transmit request
  digitalWrite(not_RE, HIGH); // cmheong Set max485 to transmit
  digitalWrite(DE, HIGH);
  for (i = 0; i < u8ModbusADUSize; i++)
  {
                swSer.print(char(u8ModbusADU[i]));
                delay(2); // cmheong 2018-07-26
  }


I modified the Trialcommand sketch (patches in boldface):

#include <ModbusMaster232.h> 
#include <SoftwareSerial.h>  // Modbus RTU pins   D7(13),D8(15)   RX,TX
// MAX485 half duplex control lines
#define not_RE 14 // D5. Enable receiver, active low
#define DE 12 // D6  Enable Transmitter, active high

// Instantiate ModbusMaster object as slave ID 1
  ModbusMaster232 node(1);

void setup()
{
  pinMode(not_RE, OUTPUT);
  pinMode(DE, OUTPUT);
  // default to transmit mode to reduce noise
  digitalWrite(not_RE, HIGH); // disable receiver
  digitalWrite(DE, HIGH); // enable transmitter
  
  Serial.begin(9600);
  delay(100);  
  node.begin(9600);  // Modbus RTU
  delay(100);
  Serial.println("Connected "); 
  Serial.println("Modbus RTU Master Online");
  
}


void loop()
{
//Website http://trialcommand.com
///////// Holding Register [0]  A [9]   = 10 Holding Registers Escritura
///////// Holding Register [0] A [9] = 10 Holding Registers Writing

int Mdelay = 500; // from 5 

node.readDiscreteInputs(0, 1); 
Serial.print("[0] ");
Serial.print(node.getResponseBuffer(0));
node.clearResponseBuffer();
delay(Mdelay);
  
node.writeSingleRegister(0x1001, 0xff); 
delay(Mdelay);
  
}

There you have it: the ESP8266 NodeMCU ESP-12E will now be able to request digital input from the modbus device as well as output to it. Having obtained the data it will now need to relay it to the Internet via its webpage, which will be shown in Part 2.

Happy Trails.

Thursday 5 July 2018

A Long Goodbye for Slackware


I first heard of Linux in the winter of 1995. I was writing a device driver for Microsoft's Windows NT intending to knock it into shape for industrial use. And complaining about the poor quality of the Windows code. Like all the time. I think just to shut me up, my friend (and boss) Wiljan Derks told me about this Finnish student who was writing a UNIX operating system all on his own. He called it Linux, just because his name was Linus Torvalds.

Linus Torvalds 1995 Amsterdam

I escaped on home leave back to balmy Malaysia in February 1996, and there in the sleepy backwater of Ipoh was this book on Linux, 'Slackware Unleashed'. At the back was glued a CDROM with Slackware release 1.1. It was an omen: I bought the only copy immediately.

And it has been Slackware for 22 years From the first install using 12 floppy disks (cdrom drives were hard to find then) on a suitcase-sized Zeos 80486DX2 souped up to a dizzying 66MHz 12MB DRAM to a tiny unassuming Raspberry Pi Zero W blazing along at 1GHz and 512MB.

Patrick Volkerding, Slackware's Benevolent Dictator for Life
Incredibly, through all these years, just like Linux, Slackware is run by one person: Patrick Volkerding.

Initially I ran a number of operating systems: Windows NT, Slackware and SuSE. But by 2004 I dropped Windows, which I had used since 1984. When Novell bought SuSE I dropped that too, and it had only been Slackware since then.

Slackware has always been slow to release new versions. The emphasis has always been stability, which is perfect for me: I had always been using a mainline Linux as an embedded device, and Slackware was a lot less work.

True it required a much bigger footprint, but hardware got better all the time and since elevator projects had a development time of two years anyway, all it needed was a leap of faith: start development immediately using a desktop and bet on the embedded hardware being able to run Slackware by the time I needed it.

Happily, in 14 years Slackware and Moore's Law has not let me down. Slackware progressed from an Advantech Industrial PC to the Via EPIA to the Intel-based fanless boxes with aplomb.

Advantech IPC
Via EPIA
Quanmax Qbox: Intel-based fanless CPU

But then came the ARM-based CPUs like the Beagleboard and the Raspberry Pi. Now a mainline Linux can fit in the palm of your hand.
Beagleboard
Raspberry Pi 3

All this required a fair amount of updates. Despite its manual nature we could cope with Slackware updates, until now: browser vulnerabilities sometimes needed three or more upgrades a year. Browsers like Firefox and Chrome were huge applications and a real pain to upgrade manually, so Debian's siren call beckoned.

When the last two installs of Raspbian went without a hitch, it is time to test my main development laptops on Debian. I started with a spare laptop: an ancient Acer Aspire 5050. Debian 9 'Stretch' installed and ran with great ease. 

Debian 9 'Stretch' on an ancient Acer Aspire 5050

True the speakers did not work and the graphics were slower than my tiny Raspberry Pi Zero W's, but Firefox is up to to the minute, so I think this is the start of Slackware's long goodbye.


On second thought, it is more like au revoir, Slackware.