Smart mat blurs the digital and physical worlds
Called Project Zanzibar, the physical sensing platform is able to locate, sense, and communicate with objects, as well as sense a user’s touch. It combines capacitive sensing and near-field communication (NFC) to enable multi-touch and hover gesture input to coexist with physical object manipulation and control, enabling new possibilities for areas such as games and educational tools.
The mat also offers portability in a tangible user interface by using existing devices such as tablets for a display. The flexible mat can be rolled up and carried or stored away until needed, and can be used anywhere there is a device with a screen available.
“The question driving us over a number of years was how you make interacting with a computer a very natural, invisible thing,” says Microsoft Senior Researcher Nicolas Villar. “Our fascination was with making the technology invisible so that you get the power of technology without it feeling like you are using technology.”
The company provides examples of applications that the mat makes possible. For example, says Villar, “What if you could play with toys and cards and blocks, while watching your actions come alive on screen?”
Since every physical object in “the Project Zanzibar world” has a globally unique ID, each toy or object can learn and possess its own history and the capability to tell its story. As a result, says the company, collectible cards can gain value over time, and dolls could conceivably “grow up” alongside a child, with shared experiences remembered and replayable over time.
In addition, the company envisions a Project Zanzibar movie maker scenario, in which children can tell stories by physically interacting with their toys on the Project Zanzibar mat while at the same generating an animated movie. Users can tell stories by manipulating toys and props on the mat to control corresponding graphical avatars, previewing and capturing the result as a movie on the screen of a connected device.
Another application area of interest the research team sees is in the educational space, including spelling and simple coding exercises using blocks – with or without screens. This is inspired, says the company, by the Montessori Method, which stresses the role of physical activity and multi-sensory learning in absorbing academic concepts and practical skills, as well as emphasizes the importance of self-directed activity and adapting a child’s learning environment to his or her developmental level.
The Microsoft researchers extended traditional Montessori exercises for young children with digital content and feedback using Project Zanzibar. For example, the letter plates Montessori exercise provides feedback as words are formed and tactile letter-shapes are traced. Paper overlays on the Project Zanzibar mat do not interfere with functionality and provide additional context to the exercise.
The Project Zanzibar mat prototype incorporates all sensing and processing functions. High-level interaction events are sent to connected devices via USB or Bluetooth.
The mat incorporates scalable and localized NFC coverage and has the ability to energize and communicate with multiple tags up to 30 mm above its surface. It features innovations such as capacitive object footprints and touch and hover recognition that detect gestures such as swiping across the mat or drawing shapes in the air.
For more, see the related white paper: “Project Zanzibar: A Portable and Flexible Tangible Interaction Platform.”
Related articles:
Microsoft brings AI to its HoloLens AR headset
AI wars heat up with new Microsoft research lab Microsoft to invest $5B in IoT
If you enjoyed this article, you will like the following ones: don't miss them by subscribing to :
