Exploring new dimensions for HMI technology
They are now employed almost universally in consumer electronics products and new opportunities are emerging all the time for their inclusion inside a broad spectrum of other system designs – including retail, medical, home automation, industrial control and white goods.
Though generally the HMIs for portable consumer products can be considered as simply standalone in nature, there are a great many situations elsewhere that are in need of multiple units being interfaced together to create a more comprehensive system. This article will look at some examples of applications in which this sort of HMI structure is warranted.
There are a plethora of potential everyday scenarios that will see major benefits from the coupling together of HMIs, in such a way that there is a consistency between what is displayed on each. Through this the data being input at one HMI will be easy to comprehend by whoever is operating/viewing another HMI to which it has been connected.
Arrangements are likely to vary – in certain cases these HMIs will be have a peer-to-peer topology, while in others there will be multiple units all connected to one another through a central server. For instance, in the retail sector, shoppers would be able to select the merchandise they want to buy from a store utilising a point-of sale unit.
This information could then be fed back to the warehouse so that these items could be made ready for collection. Likewise, in a restaurant touch panels could be embedded into each of the tables. Through these the customers could choose the food and beverages they would like to order and this data would subsequently be transferred to the corresponding display located in the kitchen. In home automation, it could also be of value, allowing the occupant to the set temperature in one room with the information being made available on each of the display units within the network.
So, given that there are already a multitude of possible opportunities for assemblies of this kind, why hasn’t this really been done before? The answer, at least partially, lies in the fact that the HMI systems currently in place are often too complex and costly to make it economically and operationally viable. However, things are changing – recent developments in the supporting semiconductor technology could have major implications on how engineers think about HMI construction in the future.
The changing face of HMI implementation
HMI systems will normally have a structure similar to the one described in Figure 1. Here a high performance (16-bit or 32-bit) microcontroller unit is responsible for creating and manipulating of the graphics that are rendered on the display. Alongside it there is a sizeable non-volatile memory resource in which all of these graphics are stored, plus a capacious frame buffer which is needed for graphics processing purposes and a series of wide data buses for carrying out communication between these devices.
Fig. 1: Conventional System for a Touch-Enabled HMI.
On top of all this, a touch controller and an audio DAC converter must be specified to take care of the touch inputs being registered and the sound output respectively. The number of different chips involved results in a substantial bill of materials being associated with this sort of system design. Furthermore, as each of these chips has quite a high pin count, the board space that such a system takes up can also be an issue. The system’s complexity means that a lot of engineering effort is required to implement it, and the relatively large power consumption is also a factor.
Figure 2 details an alternative solution which is far more streamlined than the conventional approach to HMI implementation. It greatly reduces the number of components needed and the amount of data that must be transferred between them.
Fig. 2: Rationalised HMI System Using EVE Technology.
The board real estate that must be allocated is much lower, as is the engineering overhead relating to its development. This solution is based on FTDI Chip’s EVE (Embedded Video Engine) technology. As EVE devices incorporate display, audio and touch functionality into a single chip (thanks to the built-in touch controller and single channel audio controller) there are immediate benefits in terms of board space and system complexity – but in addition to the EVE concept tackles HMI deployment in a whole different way to the conventional approach already discussed.
Here all of the graphic and audio elements that make up the HMI design are treated as objects. These objects – bitmap images, fonts, chirps, beeps, templates, overlays, logos, and suchlike, are effectively denoted in a ‘shorthand’ form. This means that they don’t have to be drawn and redrawn anew, they are simply accessed from a display list. As a result rather than having to render the display pixel by pixel, they can be rendered line by line at 1/16th of a pixel resolution, with the upshot that there are substantial savings in the required bandwidth.
As this object-oriented approach to HMI design leads to a marked reduction in the data transfer involved, the frame buffer and the large non-volatile memory can be eliminated from the design. The EVE chip’s internal object memory is more than sufficient for storing/executing several thousand different objects and rendering the display through its own built-in buffer.
Since the EVE chip can take care of the majority of the graphics processing, the microcontroller does not need to allocate much of its functionality to this task. As a result an 8-bit microcontroller will be more than adequate for the HMI system. Low bandwidth SPI or I2C serial interfaces can be used for data transfer, as opposed to the wide parallel buses shown in Figure 1.
For objects with a higher degree of complexity (such as keys, sliders, toggles, buttons, gauges, clocks, etc.) special EVE widgets have been designed. This means that rather than having to create them for scratch, all the engineer has to do is adjust the size colour and location that suits their particular HMI design. Touch tagging allows any coordinates within a certain area (e.g. coordinates delineating a button), rather than a specific point, to activate an assigned touch operation. This thereby reduces the risk of an incorrect touch response occurring, while also keeping the data transferred between the display and the supporting electronics to a minimum.
Other advantages of migrating to an object-oriented methodology include its more effective way of dealing with animated content. Whereas a conventional HMI system would have to keep redrawing what was going to be shown on the display again and again so as to accommodate the changes that occur during the animation, for this more innovative approach all that is needed is to update the coordinates that an object is to be placed at. A single object can also be repeated in multiple locations. This avoids having to waste bandwidth creating new objects many times. Finally, the anti-aliasing function enhances the quality of the images rendered, removing the jagged edges on lines and complex shapes.
Application example
The methodology just detailed proves much more effective for systems in which a number of HMIs are being interfaced together. Figure 3 describes a set up where data transfer is taking place between a pair of HMI sub-systems which have been connected to one another via an RS232 interface (though it could of course be applied equally well to other interface technologies such as Ethernet).
Fig. 3: A pair of HMI sub-systems connected to one another via an RS232 interface.
The respective HMI sub-systems are both reliant on a VM800P ‘Plus Board’ EVE evaluation module, with an attached VI800A-232U module. Each VM800P consists of an Atmel ATMEGA328P microcontroller, an FT800 EVE device from FTDI Chip, a 4.3-inch touch-enabled display, an SD card connector (for accessing stored data on an SD card) and a real-time clock (RTC) with battery backup.
Application firmware
The firmware included in the example code allows the transferring data from one VM800P to the other through the VI800A-232U daughter cards attached to each. In this particular case the data consists of a series of images, which are designed to demonstrate the capacity for data to be transferred both physically and visually. The firmware has two constituent parts:
Image_Transfer_Master.ino – This is firmware that has been loaded onto the first module, which is referred to as the ‘Master’.
Image_Transfer_Slave.ino – This is firmware that has been loaded onto second module, referred to as the ‘Slave’.
The images are stored on an SD card attached to the Master module, and are loaded one at a time into processor RAM before being transferred to the FT800 EVE device on this module for inflation and eventual storage inside the IC’s internal graphics RAM. The image filenames are hard coded and stored in the ImageName[][] array.
This means that the user can easily change the images simply by supplying new files and filenames. The loadJpegImage() function firstly checks that the file relating to the image to be loaded actually exists, then issues a CMD_LOADIMAGE command to the FT800 to load a JPEG image and inflate it. Once this is done it initiates the loadJpegBytes() function, which transfers the file data from SD card to FT800, supplying the data required by the CMD_LOADIMAGE command, in order that the FT800 can inflate the JPEG data and store it in the graphics RAM. The loadJpegBytes() function loads and transfers the JPEG file to the FT800. This is done 512byte blocks at a time, until the entire file has been processed.
Communication between Master and Slave
When the two VM800P modules are connected together in the way described in Figure 3 the Master establishes communications with the Slave, to which the Slave sends a response. A command signal COMMAND_IMAGE_START is sent in order to initiate the image transfer process (this command includes information on the size of the image).
To this the Slave returns a signal to confirm it is ready for image transfer. For every line of the image a COMMAND_IMAGE_SLICE signal is sent from the Master to the Slave (which contains the line number of the image being sent, together with the image data which has been read from the FT800’s graphics RAM).
A COMMAND_IMAGE_SLICE_CONFIRMED signal is returned, indicating that the Slave has received the image data, subsequently transferred it to the FT800 and is now ready to accept the next command. A COMMAND_IMAGE_DONE signal is sent to indicate that the image transfer has been completed.
In conclusion, the simplified HMI design strategy that is made possible by treating both image and audio content as objects has many notable upshots. In addition to lowering procurement costs, saving board space and shortening development times, it also makes it far easier to connect multiple HMIs together so that they can pass useful data between one another. This means that HMI units no longer need to be thought of in isolation, there is the option for them to become part of a much bigger system.
About the author:
Gordon Lunn is global customer engineer support manager at FTDI Chip – www.ftdichip.com
If you enjoyed this article, you will like the following ones: don't miss them by subscribing to :
