New Software Mashes Up Crowdsourced Video to Personalize Event Recordings
Cesar starts with raw video footage recorded at a specific event by different people, at different moments and from different perspectives. Next, software called a “narrative engine” uses what it knows of the relationships between people to create dynamic stories. These are tailored to an individual’s preferences, interests and social connections by automatically stitching together parts of clips into a seamless video stream.
“The stories are highly personal depending on the recipient of the story,” says Cesar.
The system works by synchronising all the video clips with a master audio track that is recorded at the event. The audio of each clip gives it a digital fingerprint allowing similar footage in different clips to be matched up. The software analyses the video content - applying facial recognition techniques, for example - and contextual information added by the film-maker. It then puts together clips or partial clips, producing a bespoke video edit for every user.
The system was tested at a school concert that was filmed with 12 cameras - some fixed, some belonging to parents in the audience - generating more than 300 raw video clips. These clips were pooled and annotated with personal details, including who was in the clip, or which musical instrument was shown. Most parents agreed that the tailored films made the viewing experience more personal.
The team presented the work at the 2012 Symposium on Document Engineering in Paris, earlier this month. “We’re living in a world of abundant content,” says Mor Naaman of Mahaya, a start-up that is developing software to find and organise social media shared from real-world events. “The real technical challenge is to do this at scale.”
(via Video mash-ups give you personalised memories - tech - 21 September 2012 - New Scientist)
![Researchers Developing Software to interpret Doodled Images
[A] new algorithm developed at Brown University and the Technical University of Berlin [is] the first computer application designed for “semantic understanding” of abstract drawings, and the research team says it could improve search applications and sketch-based interfaces…
The program can identify simple abstract sketches 56 percent of the time, compared to humans’ 73 percent average. Even those sorely lacking in verisimilitude can be detected, which is the key breakthrough here.
Computers can already recognize accurate sketches, like a police sketch of a suspect compared to photos of a face, for example. But for the type of abstract sketches we all grow up with, it’s a different challenge.
[I]f you’re asked to draw a rabbit, you would probably draw something with buck teeth, huge ears and exaggerated whiskers. Other people would easily recognize this cartoonish representation… But it doesn’t actually resemble the real thing in any meaningful way, so a computer would have no idea what it is …there are subtle tricks and meanings that a human can distinguish, but which present a tough challenge for something built in the black-and-white, ones-and-zeroes world.
[Researchers] devised a list of everyday things people might feel like doodling, settled on 250 categories and used Amazon’s Mechanical Turk crowdsourcing platform to hire some sketch artists. They took 20,000 unique sketches and fed them into existing machine-learning algorithms to train the system. The project culminated in a fun real-time computer Pictionary, where the system tries to recognize objects as the person draws them.
These are drawings of dogs, and what the computer thought they actually were.
To expand their data set, the team is thinking about gamifying this concept into something you can play on iOS or Android devices…
The goal would be improved sketch-based search, the researchers say. That could improve computer accessibility for speech, movement or literacy-impaired people — and it could work in any language, too.
Video of the team pesenting the project at SIGGRAPH
(via Computer Learns to Recognize Badly Drawn Animals | Popular Science)](http://24.media.tumblr.com/tumblr_macjbj0She1qgpcs1o1_500.jpg)
Researchers Developing Software to interpret Doodled Images
[A] new algorithm developed at Brown University and the Technical University of Berlin [is] the first computer application designed for “semantic understanding” of abstract drawings, and the research team says it could improve search applications and sketch-based interfaces…
The program can identify simple abstract sketches 56 percent of the time, compared to humans’ 73 percent average. Even those sorely lacking in verisimilitude can be detected, which is the key breakthrough here.
Computers can already recognize accurate sketches, like a police sketch of a suspect compared to photos of a face, for example. But for the type of abstract sketches we all grow up with, it’s a different challenge.
[I]f you’re asked to draw a rabbit, you would probably draw something with buck teeth, huge ears and exaggerated whiskers. Other people would easily recognize this cartoonish representation… But it doesn’t actually resemble the real thing in any meaningful way, so a computer would have no idea what it is …there are subtle tricks and meanings that a human can distinguish, but which present a tough challenge for something built in the black-and-white, ones-and-zeroes world.
[Researchers] devised a list of everyday things people might feel like doodling, settled on 250 categories and used Amazon’s Mechanical Turk crowdsourcing platform to hire some sketch artists. They took 20,000 unique sketches and fed them into existing machine-learning algorithms to train the system. The project culminated in a fun real-time computer Pictionary, where the system tries to recognize objects as the person draws them.
These are drawings of dogs, and what the computer thought they actually were.
To expand their data set, the team is thinking about gamifying this concept into something you can play on iOS or Android devices…
The goal would be improved sketch-based search, the researchers say. That could improve computer accessibility for speech, movement or literacy-impaired people — and it could work in any language, too.
Video of the team pesenting the project at SIGGRAPH
(via Computer Learns to Recognize Badly Drawn Animals | Popular Science)
Crowdsourced AI: the Low-Wage Knowledge Work of the Future
As both Artificial Intelligence and Robotics become more “real,” we have developed a more practical understanding of these technologies’ limitation. What is striking is that, contrary to what most people believe, the most “irreplaceable” human work tends to come at the lower end of the wage/status scale. Robots are better at surgery than they are at janitorial work, AI is better at legal scholarship and journalism than it is at customer service.
Enter Chorus, a crowdsourced chat platform
When people talk to the new crowd-powered chat system, called Chorus, using an instant messaging window, they get an experience practically indistinguishable from chatting with a single real person. Yet behind the scenes, each response is the result of tens of people paid a few cents to perform small tasks: including suggesting possible replies and voting for the best suggestions submitted by other workers…
Chorus does that with three simple types of task. First, any new chat updates from the human user are passed along to many crowd workers, who are asked to suggest a reply. Those suggestions are then voted on by crowd workers to determine the one that will be sent back. A final mechanism creates a kind of working memory that ensures that Chorus’s replies reflect the history of the conversation so far, crucial if it is to carry out long conversations—something that is a challenge for apps like Siri and even AI chatbots intended to showcase conversational skills.
For the working memory component, crowd members are asked to maintain a short running list of the eight most important snippets of information under discussion, to be used as a reference when workers suggest replies.
This is important, as to allow for the natural turnover of crowdsourcing workers. “A single person may not be around for the duration of the conversation—they come and go, and some may contribute more than others,” says Bigham.
Bigham says Chorus has the potential to be more than just a neat demonstration. “We definitely want to start embedding it into real systems,” he says. “Perhaps you could help someone with cognitive impairment by having a crowd as a personal assistant.” Another possibility is to combine Chorus with another system previously developed at Rochester, which has crowd workers collaborate to steer a robot. “Could you create a robot this way that can drive around and interact intelligently with humans?” asks Bigham.
(via Artificial Intelligence, Powered by Many Humans - Technology Review)
The Gamification of Synthetic Biology Continues: Creators of FoldIt Follow up With RNA Transformation Game
Meet eteRNA, your new internet addiction. Not only is it a super-fun way to procrastinate on that thing you should be doing, it also helps to advance biology’s understanding of RNA and its synthesis- in a big way.Scientists from Stanford University and Carnegie Mellon University have developed eteRNA as a successor to Foldit, a popular internet-based game that proved the pattern-matching skills of amateurs could outperform some of the best protein-folding algorithms designed by scientists.
They’re hedging their bets that eteRNA will work similarly - and are even funding the real-life synthesis of the weekly winner’s RNA molecule to see if it really does fold the same way the game predicts it should.
The scientists hope to tap the internet’s ability to harness what is described as “collective intelligence,” the collaborative potential of hundreds or thousands of human minds linked together.
Using games to harvest participation from amateurs exploits a resource which the social scientist Clay Shirky recently described as the “cognitive surplus” - the idea that together, as a collection of amateurs, we internet people make a very good algorithm because we react to information presented in a game, get better at it as we go along, and make informed decisions based on what has or hasn’t worked for us in the past.
“We’re the leading edge in asking nonexperts to do really complicated things online,” says Dr. Treuille, an assistant professor of computer science at Carnegie Mellon and one of the original masterminds behind the game. “RNA are beautiful molecules. They are very simple and they self-assemble into complex shapes. From the scientific side, there is an RNA revolution going on. The complexity of life may be due to RNA signaling.”
“This [project] is like putting a molecular chess game in people’s hands at a massive level,” he continues. “I think of this as opening up science. I think we are democratizing science.”
And, so far, the democratisation is working. Although the creators warn that game players may start to see legal and ethical issues in gameplay down the road, for now, the collective intelligence is trumping professionally designed algorithms. Significantly, not only do humans outperform their computer adversaries, but the human strategies developed during the course of the game are significantly more flexible and adaptable than those of the algorithms they’re pitted against.
9 Years After Congress Killed DARPA's Terror Futures Market, IARPA Brings It Back
Nine years ago, Congress blocked a Pentagon agency from setting up a website that would have allowed anyone with a credit card to bet on the likelihood of foreign assassinations, coups and terrorist attacks.
The idea was to take advantage of the “wisdom of crowds,” a social science maxim that contends the average of a group of forecasters, under certain circumstances, tends to be more accurate than even the most knowledgeable single forecaster…Now terrorism futures are back.
DARPA’s sister agency — the Intelligence Advanced Research Projects Activity, which funds experimental projects for the U.S. intelligence community — is running a four-year, $50-million program that pays people willing to predict major world events, including wars and terrorist strikes. Unlike the earlier scheme, participants can’t profit from their predictions.
Now in its second year, the so-called crowd-sourcing project involves competing corporate and university teams, including UC Irvine. Each team includes more than a dozen social scientists and as many as 2,000 participants, who can answer hundreds of questions each if they want.
The study, known as Aggregative Contingent Estimation, is designed to see whether the 17 agencies in the U.S. intelligence community can aggregate the judgment of its thousands of analysts — rather than rely on the expertise of just a few — to issue more accurate warnings to policy makers before and during major global events.
Human Workers, Managed by an Algorithm for $2 to $4 an Hour
Now several startups, including CrowdFlower and CrowdSource, have written software that works on top of Mechanical Turk, adding ways to test and rank workers, match them up to tasks, and organize work so it gets double- or triple-checked.
“In the past [crowdsourcing] has been more experimental than a real enterprise solution,” says Stephanie Leffler, the founder of CrowdSource. “The reality is that it’s tough to do at any kind of scale.”
MobileWorks has its own workflow software, but it’s also trying to solve the incentive problem by recruiting workers overseas, in developing nations like India, where low payments can still add up to meaningful income.
Kulkarni, who founded the company in 2010 with fellow graduate students from the University of California, Berkeley, says the value of tasks is set so that workers can reasonably earn $2 to $4 an hour; payments are on a sliding scale, with lower rates for poorer countries.
“Even though they are acting as agents of a computer program, we are creating an opportunity for them,” he says. MobileWorks charges its clients rates starting at $5 per hour for workers’ time.
Open ROV: Crowd-Funded Open Source Robotic Undersea Exploration Platform
It all started with a kid who wanted to explore a cave that was rumored to contain sunken treasure.
OpenROV is an open source underwater robot for exploration and education. We’re a community of DIY Ocean Explorers committed to developing open source technology to empower more people to explore and study underwater environments…
We want this to be a sustainable adventure. Our plan is to get user feedback from people who build and operate OpenROV’s to make the design even better and more fitted toward the community’s needs. We plan to continue selling OpenROV Kits (and assembled OpenROVs) on our website as well as payloads and accessories for specific uses. We also hope that by building a strong community of people who understand the hardware and its applications, we’ll be able to develop ways of doing better science and exploration in more remote and interesting places.
(via OpenROV - The Open Source Underwater Robot by OpenROV — Kickstarter ht BoingBoing)
Crowd-Sourcing in Near Real Time
So how quickly can a crowd be put into action? That’s the question tackled today by Michael Bernstein at the Massachusetts Institute of Technology in Cambridge and a few pals.
In the past, these guys have found ways to bring a crowd to bear in about two seconds. That’s quick. But the reaction time is limited to how quickly a worker responds to an alert. Now these guys say they’ve find a way to reduce the reaction time to 500 milliseconds—that’s effectively realtime.
A system with a half second latency could turn crowdsourcing into a very different kind of resource.
The idea that Bernstein and co have come up with is straightforward. These guys simply “precruit” a crowd and keep them on standby until a task becomes available. Effectively, they’re paying workers a retainer so that they are available immediately when needed
The difficulty is in the messy details of precruitment. How many workers do you need to keep on retainer, how do cope with drop outs and how do you keep people interested so that they are available to work at a fraction of a second’s notice?
Bernstein and co have used an idea called queuing theory to work out how to optimise the process of precruitment according to how often the task comes up, how long it takes and so on. They’ve also developed an interesting psychological trick to keep workers ready for action.
When workers are precruited, a screen opens up on their computer which downloads the task. The download occurs extremely quickly but if no task is to hand, the screen shows a “loading” bar. It turns out that the loading bar keeps workers focused on the forthcoming task for up to ten seconds, at which point their attention begins to wander. At that point, if no task materialises, the worker can be paid off.
Bernstein and co have even tested how well this works using a whack-a-mole type task which appears on workers screens after a randomly chosen period between 0 and 20 seconds. They recruited 50 workers to carry out 373 whacks and found the median length of time between the mole’s appearance and the worker moving the mouse toward the mole to click on it was 0.50 seconds.
(via How To Perfect Realtime Crowdsourcing - Technology Review)
Crowdsourcing the Genome - NIH Releases Huge Genetic Dataset Onto the Cloud:
The NIH’s initiative is part of a larger movement to manage the deluge of “big data” in science, which has become a scientific discipline in itself. Such data sets have become so massive that few researchers have the computing power to use them. The NIH has calculated that the 1000 Genomes Project is the equivalent of 16 million file cabinets filled with text, or more than 30 000 standard DVDs. Making it available on cloud is a good deal for scientists and their institutions, who won’t have to take on the costs of acquiring more bandwidth, data storage and analytical computing capacity just to access the data. “This means researchers and labs of all sizes and budgets have access to the complete 1,000 Genomes Project data and can immediately start analyzing and crunching the data without the investment it would normally require in hardware, facilities and personnel,” says Deepak Singh, a principal product manager at Amazon Web Services. “Researchers can focus on advancing science, not obtaining the resources required for their research.”
(via World’s Largest Dataset on Human Genetic Variation Goes Public - IEEE Spectrum)
Brain Analysis Follows Protein Folding into Gamification:
There are around 100 billion neurons in a human brain, forming up to 100 trillion synaptic interconnections. Neuroscientists believe that these synapses are the key to almost every one of your unique, identifiable features: Memories, mental disorders, and even your personality are encoded in the wiring of your brain.
Understandably, neuroscientists really want to investigate these neurons and synapses to work out how they play such a vital role in our human makeup. Unfortunately, these 100 trillion connections are crammed into a two-pound bag of soggy flesh, making analysis rather hard. At the moment we know that neurons trigger an electrical signal, and that hormones affect the speed at which signals cross between synapses, and that somehow this results in a mental image of a naked Kristen Bell from her Veronica Mars period, but that’s about it.
MIT wants to change all that by tasking thousands of people with analyzing a 0.3-millimeter slice of mouse retinal tissue. Using a new site called Eyewire, MIT will ask users to track a neuron’s path by coloring in each axon (tendril). In the future, MIT will roll out another “game” which challenges users to find the synapses. The end result will be the connectome (a tome of connections) of the mouse’s retina.
To perform this kind of analysis, MIT must slice this three-dimensional 0.3-millimeter piece of brain tissue into incredibly thin, “2D” slices using a diamond blade and a process called serial electron microscopy. The slices are so thin that a terabyte of images are created from a piece of brain that’s much smaller than the head of a pin. You now have some idea of how hard it will be to investigate and understand the human brain; we’re talking about hundreds of exabytes of imagery that would need to be analyzed.
Ultimately, though, if we could get our hands on the connectome of the human brain… Well, we would experience an enlightenment of unprecedented scale. We would understand exactly why we are the way that we are. There would be no stones left to turn.
(via MIT crowdsources and gamifies brain analysis | ExtremeTech)
