Take advantage of new HMI tools for embedded graphics design
The quality and quantity of graphics content in display-enabled embedded systems is increasingly rapidly. So is the need for an efficient and easy to use HMI/GUI (Human Machine Interface/Graphical User Interface) tool.
These kinds of tools enhance the system developer’s ability to input a design, develop and verify the application, then rapidly deploy to the target system. They can save time and effort, from rapid prototyping of user interfaces all the way to serial development of the embedded system.
Also important is the ability to complete these steps easily and cost-efficiently with a process flow that is as seamless as possible.
Described here are the most desirable traits of an HMI tool followed by discussion about a specific tool offering from Fujitsu, the CGI Studio, and how it delivers the listed benefits.
Desirable features of an HMI Tool
The HMI tool should be able to allow early evaluation of HMI development, and structured so that it supports design reviews even in very early stage of the project. It should be able to conveniently support changes to the project implementation, such as graphics content, in the later stages of the project. All of these factors contribute towards making it cost effective.
For example, the HMI tool should give realistic preview of the graphics and scenes as soon as they are imported. This is implemented within the IDE of the tool and enables the designer to have WYSIWYG (What You See Is What You Get) view of the graphics.
Widgets or business logic imported into the tool should be able to be run and verified immediately. This is only possible if the HMI tool has capability to simulate graphics application on the host PC. It should allow changing look and feel of the graphics without the need to recompile or rebuild the project (re-skinning).
This can be achieved by separating code and graphics data from the beginning of the project. For example, from the outset of the project, the graphics designer can churn out 2D and 3D graphics using industry standard tools such as 3D Studio Max or Maya.
Independently of the graphics design, software engineers can work on business logic to assign behavior to the graphic elements. The two elements can then be imported into the tool and verified together accordingly. If at a later stage, the look and feel needs to be changed, all that needs to be done is to re-import the new graphics assets without affecting the business logic part.
Convenient host and target environment
Microsoft Windows is undoubtedly the most common OS for both work and personal use. Having a tool based on this OS allows implementing and testing user interfaces in a convenient environment.
In addition to that, the HMI tool should allow continuous and trouble free integration on target platform. There should be as few steps as possible in going from the host to the target. One way to achieve that is by having the same execution environment engine that runs the graphics application on host and target.
HMI tool needs not only UI (User Interface) data, such as graphics assets (bit map files, 2D/3D models, asset properties, etc.), but also code that defines the application behavior. The code embodies business logic and could possibly be generated by a state machine tool or hand coded.
In the latter case, the tool should allow the standard programming languages within the software community. The HMI tool should be structured in a way that UI data is strictly separated from the code portion.
The resulting advantage is the ease of changing the look and feel of graphics without touching the code or the need to rebuild it. The old UI data in the tool output is simply replaced with the new one without rebuilding the entire application.
It’s also desirable that the tool is capable of determining hardware requirements for the intended graphics application well ahead of time.
For example, the HMI tool can help determine the minimum number of MIPS needed for graphics and provide guidance about how powerful the GPU should be to run the application.
It’s also helpful if it can give some indication about the internal bus throughputs and external graphics memory bandwidth requirements. The possible number of hardware iterations and valuable project time saved as a result of this would carry immense benefits to the initiative.
CGI Studio
CGI Studio is an HMI tool chain from Fujitsu and is targeted for automotive clusters and infotainment systems. The tool has been specifically designed from the ground up for automotive applications.
The tool chain offers a flexible licensing model covering a variety of usage scenarios, such as tool evaluation, prototyping and reference design, product development for one or more vehicle platforms, etc. As with any tool, it consists of different blocks or modules.
The HMI solution should allow the use of the same tool at all stages of the application development, including by studio artists, technical artists, and embedded engineers. This versatility results in a seamless process flow that brings many benefits to the project, including reduced cost, ease of managing the tool, common knowledge based, and others.
Each individual involved in the project can focus on his or her own role. The studio artist can design the 2D or 3D graphics content in the industry standard studio tools and then use the HMI tool for preview.
The technical artist can import that graphics content into the HMI tool and use the graphics to create scenes using appropriate lighting and composition. Finally, the embedded engineer can add his widgets to the project. This will assign appropriate behavior to graphics elements as required by the application.
Figure 1. Different roles in HMI Development
The tool chain should also support a flexible licensing model that allows its use in a variety of situations, for example, tool evaluation, rapid prototyping of a reference application, and product development. The user shouldn’t be forced to pay the same cost regardless of the immediate need.
Convenient host and target environment
Microsoft Windows is undoubtedly the most common OS for both work and personal use. Having a tool based on this OS allows implementing and testing user interfaces in a convenient environment.
In addition to that, the HMI tool should allow continuous and trouble free integration on target platform. There should be as few steps as possible in going from the host to the target. One way to achieve that is by having the same execution environment engine that runs the graphics application on host and target.
HMI tool needs not only UI (User Interface) data, such as graphics assets (bit map files, 2D/3D models, asset properties, etc.), but also code that defines the application behavior. The code embodies business logic and could possibly be generated by a state machine tool or hand coded.
In the latter case, the tool should allow the standard programming languages within the software community. The HMI tool should be structured in a way that UI data is strictly separated from the code portion.
The resulting advantage is the ease of changing the look and feel of graphics without touching the code or the need to rebuild it. The old UI data in the tool output is simply replaced with the new one without rebuilding the entire application.
It’s also desirable that the tool is capable of determining hardware requirements for the intended graphics application well ahead of time.
For example, the HMI tool can help determine the minimum number of MIPS needed for graphics and provide guidance about how powerful the GPU should be to run the application.
It’s also helpful if it can give some indication about the internal bus throughputs and external graphics memory bandwidth requirements. The possible number of hardware iterations and valuable project time saved as a result of this would carry immense benefits to the initiative.
CGI Studio
CGI Studio is an HMI tool chain from Fujitsu and is targeted for automotive clusters and infotainment systems. The tool has been specifically designed from the ground up for automotive applications.
The tool chain offers a flexible licensing model covering a variety of usage scenarios, such as tool evaluation, prototyping and reference design, product development for one or more vehicle platforms, etc. As with any tool, it consists of different blocks or modules.
Figure 2. Scene Composer IDE Snapshot
This is the part of CGI Studio that composes 2D and 3D scenes. It allows combining 2D and 3D content, thereby enabling the seamless process flow. Within a scene, it allows separating 2D and 3D content in distinct layers so that they can be catered to their respective needs.
One of the inputs to this tool is the graphics assets. They could be generated using industry standard 3D and 2D authoring tools. Some examples of graphics assets are Models and Geometry in FBX format, Textures, 2D True Type Fonts, 3D Fonts, etc.
Another key input to the scene composer is the widgets. As mentioned earlier, these are blocks of code that assign behavior to the elements of graphics. For example, a widget could control the movement of a needle for a cluster application. Other inputs to the tools are animation and shader program for the GPU.
Scene composer allows drag and drop of imported 2D/3D graphics assets and widgets. This enables easy 2D/3D scene composition and object configuration. Controls for light and camera are also available.
It is capable of creating animations and also configuration and assignment relative to 2D/3D widget properties. Scene Composer is “display controller layer and multi-display” aware. It has a mechanism to export the output of scene composer for running on either the host or the target.
Figure 3. CGI Studio Process Flow
Active scene is always rendered using a WYSIWYG approach. As a result, it is possible to visually inspect the imported graphics assets early in the development. Moreover, object properties can be dynamically changed in the active scene to get immediate results and timely evaluation.
Candera Engine
This CGI module runs the output of the Scene Composer. Candera Engine can run either on the host or on the actual target. It supports both 2D and 3D, thereby allowing one tool for both types of developments
Candera 2D engine supports dynamic scene graph. It supports 2D animations, including smooth rotation, scaling, and translation of bitmaps. In order to enable seamless integration, the 2D engine supports interaction with the 3D engine in a number of ways.
For example, it supports render to texture, can post-process 3D images, and can combine scenes with 3D, etc. In addition to that, it supports hardware layers, multiple displays, alpha blending, 2D widgets, and text rendering.
Candera 3D engine is based on OpenGL-ES 2.0 and is fully compliant with the standard specification. It has been designed to be OS, application, and hardware agnostic.
Its key functions are screen and scene management, handling 3D objects and their processing, animation framework, and asset management in the embedded system’s memory. It also optimizes render order and state management.
One of the key modules is the Player. It runs the generate application on the host as well as on the target. As a result, it is possible to see verify the application on the host even before it is deployed on the target. This is also helps in verifying the widget design.
Other Modules
CGI Studio also has some other modules that bring distinct advantages to the table. Courier is an interaction framework that handles data binding and messaging with the host system, thus allowing smooth integration of the tool.
The Analyzer helps to assess application performance and can assist in bottleneck detection and optimization. Lastly, Translator helps in context based translation for different languages.
An HMI tool chain such as CGI studio enables system developers to seamlessly implement a graphics application from concept to actual embedded implementation. This is made possible while bringing key benefits such as cost effectiveness, seamless process, having a single tool for both 2D and 3D development on a convenient host platform.
Because it is based on the Microsoft Windows platform for development as well as simulation, it allows easy integration of the application on target platform as there is no need to generate source code and merge it into the SOC software structure. Widget development can be done using industry standard programming languages, such as C, C++, or C#.
Waqar Saleem is a senior applications engineer with Fujitsu Semiconductor America, based in Detroit. He has more than a dozen years of design and applications experience with Fujitsu Semiconductor, and holds engineering degrees from San Jose State Univ. and the University of Engineering and Technology in Lahore, Pakistan.
Article by courtesy of Industrial Control DesignLine
If you enjoyed this article, you will like the following ones: don't miss them by subscribing to :
