Tomorrow is Today

Researchers Developing Software to interpret Doodled Images

[A] new algorithm developed at Brown University and the Technical University of Berlin [is] the first computer application designed for “semantic understanding” of abstract drawings, and the research team says it could improve search applications and sketch-based interfaces…

The program can identify simple abstract sketches 56 percent of the time, compared to humans’ 73 percent average. Even those sorely lacking in verisimilitude can be detected, which is the key breakthrough here.

Computers can already recognize accurate sketches, like a police sketch of a suspect compared to photos of a face, for example. But for the type of abstract sketches we all grow up with, it’s a different challenge.

[I]f you’re asked to draw a rabbit, you would probably draw something with buck teeth, huge ears and exaggerated whiskers. Other people would easily recognize this cartoonish representation… But it doesn’t actually resemble the real thing in any meaningful way, so a computer would have no idea what it is …there are subtle tricks and meanings that a human can distinguish, but which present a tough challenge for something built in the black-and-white, ones-and-zeroes world.

[Researchers] devised a list of everyday things people might feel like doodling, settled on 250 categories and used Amazon’s Mechanical Turk crowdsourcing platform to hire some sketch artists. They took 20,000 unique sketches and fed them into existing machine-learning algorithms to train the system. The project culminated in a fun real-time computer Pictionary, where the system tries to recognize objects as the person draws them.

These are drawings of dogs, and what the computer thought they actually were.

To expand their data set, the team is thinking about gamifying this concept into something you can play on iOS or Android devices…

The goal would be improved sketch-based search, the researchers say. That could improve computer accessibility for speech, movement or literacy-impaired people — and it could work in any language, too. 

Video of the team pesenting the project at SIGGRAPH

(via Computer Learns to Recognize Badly Drawn Animals | Popular Science)

Malaysian Computer Detects Emotions, Reads Lips

Scientists in Malaysia are teaching a computer to interpret human emotions based on lip patterns.

The system could improve the way we interact with computers and perhaps allow disabled people to use computer-based communications devices, such as voice synthesizers, more effectively and more efficiently, says Karthigayan Muthukaruppan of Manipal International University.

The system uses a genetic algorithm that gets better and better with each iteration to match irregular ellipse-fitting equations to the shape of a human mouth displaying different emotions.

They have used photos of individuals from Southeast Asia and Japan to train a computer to recognize the six commonly accepted human emotions — happiness, sadness, fear, angry, disgust, surprise — and a neutral expression. The upper and lower lip is each analyzed as two separate ellipses by the algorithm.

The team’s algorithm can successfully classify the seven emotions and a neutral expression described, the scientists say.

(via Computer learning to read lips to detect emotions | KurzweilAI)

Microsoft Demos Computer Interface That Takes Input From Muscle Movements

This new technology relies on electromyography (EMG), which is basically the muscular equivalent of an EEG — electroencephalography, the basis for brain-computer interfaces. An EMG detects the electrical charges created by muscle cells after they receive a signal from the brain. By placing an armband around the muscles in your forearm — the muscles that control your finger movements — an EMG can accurately detect the movement of your fingers. With training, software can then convert specific EMG readings into gestures, which can be fed into the computer as normal keyboard, mouse, or touchscreen inputs.

(via Microsoft demos muscle-computer interface, air Guitar Hero now a reality | ExtremeTech)

Japanese Telepresence (“Telexistence”) Avatar Transmits Touch, Temperature and Vibration to Remote Operator

The TELESAR V’s hands and fingers are equipped with a number of sensors to capture and relay tactile information to its operator through special gloves.  The primary sensor inside each fingertip is a vision-based force sensor which is comprised of a wide-angle camera that looks through a gel-layer mixed with thermochromic ink.  When the gel compresses, the thermochromic ink becomes denser, which the camera interprets as force information.

Microphones underneath the robot’s fingertips convert low to mid level vibrations; when pouring marbles from one cup to another (as the robot), the operator feels the tactile sensation from doing so.  Furthermore, the operator is able to sense changes in temperature at the robot’s fingertips, thanks to thermoelectric peltier devices which reproduce warm and cold temperature inside the operator’s gloves.  Now even an object’s texture can be relayed to the operator.

(via TELESAR V Avatar Transfers Touch, Vibration, Temperature)

Mind-reading Speller Enables Conversation for Patients in Vegetative State

The first real-time brain-scanning speller will allow people in an apparent vegetative state (unable to speak or move) to communicate, according to Maastricht University scientists.

The new technology builds on earlier uses of fMRI (functional magnetic resonance imaging) brain scans by Adrian Owen and colleagues to assess consciousness by enabling patients to answer yes and no questions.

fMRI tracks brain activity by measuring blood flow. “The work led me to wonder whether it might even become possible to use fMRI, mental tasks, and appropriate experimental designs to freely encode thoughts, letter-by-letter, and therewith enable back-and-forth communication in the absence of motor behavior,” said Bettina Sorger of Maastricht University in The Netherlands.

(via Mind-reading speller allows full conversations for vegetative-state patients | KurzweilAI)

Carnegie Mellon Tracks Inventory Using Robot

Great example of the trend toward developing robots that are optimized to work with humans. Humans remain the primary, high-touch interface in the retail environment. The robot performs simple, repetitive tasks, and creates data artifacts (the map and inventory) that are usable for both bots and humans.

While making its rounds, the robot uses a combination of image-processing and machine-learning algorithms; a database of 3-D and 2-D images showing the store’s stock; and a basic map of the store’s layout—for example, where the T-shirts are stacked, and where the mugs live.

The robot has proximity sensors so that it doesn’t run into anything. None of the technologies it uses are new in themselves, says Narasimhan. It’s the combination of different types of algorithms running on a low-power system that makes the system unique.

The map generated by the robot is sent to a large touch-screen system in the store and a real-time inventory list is sent to iPad-carrying staff.

The robot uses a few different tricks to identify items. It looks for barcodes and text; and uses information about the shape, size, and color of an object to determine its identity. These are all pretty conventional computer-vision tasks, says Narasimhan. But the robot also identifies objects based on information about the structure of the store, and items belong next to each other. “If an unidentified bright orange box is near Clorox bleach, it will infer that the box is Tide detergent,” she says.

Narasimhan’s group developed the system after interviewing retailers about their needs. Stores lose money when they run low on a popular item, and when a customer puts down a jar of salsa in the detergent aisle where it won’t be found by someone who wants to buy it; or when customers ask where something is and clerks don’t know.

So far, the robotic inventory system seems to have helped increase the staff’s knowledge of where everything is. By the fall, Narasimhan expects to learn whether it has also saved the store money.

(via A Robot Takes Stock - Technology Review)