We also discard correspondences where there isn’t enough visual structure to be confident in the results of optical flow. We then downsample the correspondences for computational efficiency. This provides a smooth and dense correspondence field. Using techniques described in our PhotoScan blog post, we compute optical flow from one image to the other. The first step is to find corresponding pixel locations for each pair of images that overlap.

To simplify the task of aligning the images and to satisfy computational requirements, we’ve broken it into two steps. The approach must also be robust to varying scene geometry, lighting conditions, calibration quality, and many other conditions. This needs to be done carefully to avoid introducing new types of visual artifacts. The idea is to subtly warp each input image such that the image content lines up within regions of overlap. In order to provide more seamless Street View images, we’ve developed a new algorithm based on optical flow to help solve these challenges. Right: The same Street View panorama after optical flow seam repair. Left: The Sydney Opera House with stitching seams along its iconic shells. Overlap between adjacent cameras is shown in darker gray. Right: A visualization of the spatial coverage of each camera. Center: A close-up of the rosette, which is made up of 15 cameras.

Left: A Street View car carrying a multi-camera rosette. And while we attempt to address these issues by using approximate scene geometry to account for parallax and frequent camera re-calibration, visible seams in image overlap regions can still occur. However, many things can thwart the creation of a 'successful' panorama, such as mis-calibration of the rosette camera geometry, timing differences between adjacent cameras, and parallax. The creation of these panoramas is a complicated process, involving capturing images from a multi-camera rig called a rosette, and then using image blending techniques to carefully stitch them all together. In 2007, we introduced Google Street View, enabling you to explore the world through panoramas of neighborhoods, landmarks, museums and more, right from your browser or mobile device. Posted by Mike Krainin, Software Engineer and Ce Liu, Research Scientist, Machine Perception

Google earth 360 panorama