Monday, August 21, 2006 - Page updated at 12:00 AM

E-mail article     Print

Software takes you into world of images

Seattle Times technology reporter

Live Labs

Launched: January 2006.

Goal: Rapidly build new Internet products and services.

Staff: About 40 engineers and researchers, based at Microsoft's Redmond campus and its new office in Pioneer Square, with a focus on applied research. They collaborate with other researchers and product groups inside and outside of Microsoft.

Led by: Gary Flake, a computer scientist who founded Yahoo! Research Labs

In the works: The first project is Photosynth, three-dimensional photo organization and display software due out in a technology preview this fall. Other projects: social computing and "wisdom of the crowd"; software that extracts specific data from a Web page to allow functions, such as one-click mapping and collaborative authoring.

For a video demonstration of Photosynth:

Source: Microsoft

Microsoft is building software that can take you to Europe.

In one demonstration, the program drops you in the middle of an Italian piazza and invites you to turn your gaze from the soaring tower to the porticos lining the square. That gold painting adorning the basilica catch your eye? Smoothly zoom in for an up-close look. Click a button and a dozen images of the facade spread across the screen for even more detail. The three-dimensional photo organization and display software, called Photosynth, has been shown in recent weeks to gatherings of computer-graphics experts, Microsoft's internal sales people and analysts.

The analysts spent a day listening to presentations on every aspect of the company but clapped loudest for the Photosynth tour.

That's probably the kind of response Microsoft wanted for the first project to emerge from its new effort to innovate on the Internet. Live Labs, the unit that fostered Photosynth, is one of several ways Microsoft is competing with Google, Yahoo! and smaller companies to build and profit from the next big thing on the Web.

Microsoft launched Live Labs in January to quickly create new products from whole cloth or by combining existing technology from sources inside and outside the company.

Photosynth, for example, makes use of "photo tourism" software developed by University of Washington and Microsoft researchers; imaging products and services from elsewhere in the company; and a novel display technology developed by Seadragon Software, a Ballard startup transplanted directly into Live Labs after an acquisition late last year.

"We just pulled it all together into what felt like a coherent project," said Live Labs group manager Adam Sheppard.

Photosynth went from research to prototype in a matter of months — a limited public preview is due out in fall — much faster than the typical software-development timeline at Microsoft. Such speed has come to characterize product development on the Internet and will be a hallmark of Live Labs, Sheppard said.

"It's kind of a new method for us for developing software and we're pretty excited about the nimbleness that it will give us," Sheppard said.

The aim is to put new technology into the hands of consumers, get their feedback and quickly respond with improvements.

Roughly 40 scientists and engineers who focus on applied research will try to execute the strategy. About 100 employees of Microsoft Research — often described as an elite university computer-science department within the company — are affiliated with Live Labs.

Gary Flake, who started Yahoo! Research Labs before being recruited at Microsoft, is at the helm. (He was on vacation and unavailable for this story.)

Earlier this month, Flake and Live Labs began reporting to Chief Software Architect Ray Ozzie, the executive driving Microsoft's Internet-services strategy, which includes advertising-supported services such as Windows Live and Office Live.

"Live Labs is one of the most interesting, cutting-edge parts of the whole Live strategy," said Matt Rosoff, an analyst with Directions on Microsoft in Kirkland.

Microsoft surprise

Blaise Agüera y Arcas, Seadragon's founder and now a software architect at Live Labs, said he was surprised by what Flake had to offer at Microsoft.

"He's a very hip, Web 2.0 kind of guy and really not what I had associated with Microsoft personally," said Agüera y Arcas, a 30-year-old former Princeton researcher who has studied physics, computational neuroscience and applied mathematics. "He had some very progressive ideas about what could be done within Microsoft."

The Photosynth team, which includes former Seadragon staffers, has set up shop miles from Microsoft's Redmond campus, on the fifth floor of the iconic Smith Tower.

It says "Microsoft" on the door but the feeling inside is that of a well-funded startup. There's the modern furniture, the wood trim, the guys in black shirts.

Sitting in front of a 30-inch monitor in the corner of an open work room, Agüera y Arcas took Photosynth through its paces while explaining the "fancy stuff going on under the hood."

It starts with a collection of images of a specific place, say, St. Peter's Square in Rome or Half Dome in Yosemite National Park.

The "photo tourism" software — developed by Noah Snavely, a third-year graduate student at UW, computer-science professor Steven Seitz and Microsoft researcher Richard Szeliski — automatically organizes a photo collection.

The software matches up features the pictures have in common and uses that information to build a rudimentary three-dimensional map of the space, calculate the position from which each picture was taken and place each in the appropriate 3-D context.

The Seadragon component allows fluid navigation through the software-built world. The user can move from photo to photo and click on a specific element, a mosaic for example, to see other photos that contain the same element.

It's like viewing a nonlinear slide show: The user has the freedom to move about the space and view the pictures in any order.

Interest generated

The Photosynth demonstrations, mostly using canned photo collections taken by one photographer on one or two days, have generated considerable interest.

"I thought it was — just from a pure wow factor — the coolest thing that they showed at the [analyst] meeting," said Rosoff, the Directions analyst.

Agüera y Arcas and others at Live Labs see grander things for the software.

He showed another Photosynth environment based on pictures posted to the photo-sharing Web site.

The more than 500 images, tagged with the words "Notre Dame," were taken by many different photographers using everything from camera phones to high-end digital cameras at different times of day, throughout the year.

The resulting Photosynth collection is less uniform than the canned environments but still offers myriad details and perspectives to create a sense of being there.

"You get to see the city through the lens of people who have traveled there ahead of you," Microsoft researcher Szeliski said. "You get to follow in their footsteps."

The software also allows the addition of annotations, such as a description of Notre Dame's famous Rose Window. Instead of having to manually enter that information in each photo, Photosynth recognizes the similarities and applies the annotation to every photo that contains the window.

As more Photosynth worlds are built, they could be linked together. The software could, for example, recognize a postcard of Notre Dame in a picture of someone's refrigerator and use that image as an entry point — like a Web link — to a broader collection of images of the cathedral.

"The kind of network effects and community effects that can come out of something like this are huge," Agüera y Arcas said.

Efficient use of resources

Photosynth looks like it would require a very powerful computer to operate. But the Seadragon technology makes efficient use of computing resources by streaming in data as it's needed.

As you zoom in on a picture, for example, the software continuously loads more data to present a high-resolution image.

Szeliski said computers powerful enough to run Microsoft's forthcoming Windows Vista operating system would be "quite adequate" to run Photosynth.

He said there's no business model built around Photosynth yet, but could enhance Windows Live Local mapping software.

An aerial view could be combined with street-level pictures to give users the ability to swoop down and look at a specific storefront. Adding more photos inside a restaurant could give would-be patrons a sense of the atmosphere. It could also provide a 3-D tour of an object for sale.

Benjamin J. Romano: 206-464-2149 or

Copyright © 2006 The Seattle Times Company


Get home delivery today!