Before you discuss the future of video, how did we get this straight video realization
23 Apr 2016"Editor's note" the writer Wang Rui, VR senior industry practitioners.
Perhaps is this a year or two, with the surging of VR boom, "Panorama" was the word onto the table over and over again, as "virtual reality", "real 3D", "360-degree", "720" and other names, so are a lot of people as the main word for a form of virtual reality contents.Indeed, lack of VR contents is now a growing number of developers and business groups concerned, and panoramic pictures and videos, as well as infancy VR movies, will undoubtedly become a very good way to landing. It does not need too much of the ability to interactively study costs and at the same time, gives the viewer the full immersive experience, as well as offline rendering and photography get the best effect.So, panoramic view the definition and the implementation of what is, how to build a panoramic content? This article will try to tell it from several different key perspective, expects to be able to wave upon wave of creators who help.1, projection methodsPanoramic photography is not what a new concept, in fact, it can even be traced back to the 12th century of the Night Revels of Han Xizai:
Of course, this is not a truly immersive experience, even if we give this picture of long rolled up into a cylinder, then stood watch in the Center, and still find something missing, that's right, an obvious seam, as well as both the head and foot area blank. 
This problem is very simple, because the Song dynasty who did not intend to put this picture into immersive experience–this is nonsense – the real reason is, of course, picture corresponds to the physical space and the view did not reach the level of full surround, which is horizontal (latitude) 360 degrees vertically (latitude) of 180 degrees. That's right, at this point, you must think of this picture:
Like the world map on the wall may be in your home already for some years, perhaps after moving on to the University you've never even look at it, but it meets the needs of a panoramic picture all the conditions, you put it in all kinds of VR glasses to watch it, just like in the whole world around.This correctly around the physical perspective on real-world scene to a 2D image, and the ability to restore to implement immersive VR glasses to watch the mathematical process, called projection (projection).And that seemingly mundane world map photo, use a common called Equirectangular projection, it is characterized by horizontal field of view image size can get a well maintained, and the vertical angle, especially near the poles of unlimited dimensions will happen when stretched.Stretching for this form of projection in the figure below more clearly reflected, note the dome texture changes, the more near the top of the screen, the more severe the distortion. Fortunately, the VR helmet and application is the meaning of the apparent deformation of the revert to the full view of the screen content, which allows users to have a sense of immersive Surround.
Panoramic image projections, however is much more than this one, such as the recently released Ricoh Theta s Insta360 panoramic camera, using a more simple and effective alternative projection strategy:
Through its two fish-eye camera output image, each covers 180 degrees of the horizontal and vertical field of view angles, and two output "buckle" together is full of immersive Surround view.
Of course, known as Fisheye projection, generated 2D images in fact distort are more serious. By way of image reprojection process when it was displayed in a VR glasses to transform it, limited by the image sampling frequency (or generally say, limits on pixel size), restoring such distortions would exert a measure of image quality loss and, therefore, may cause the panorama the decline in the quality of the content itself.In view of this, as an important host of panoramic content matrix, projection image (or video) not only should contain complete images of all content, to avoid too much distortion projection VR glasses in order to avoid loss of quality.So, apart from two projections set out above, there are more options you can choose to do? The answer is, of course, and plenty of it!Like the Mercator projection (Mercator), along the axis of the tensile deformation smaller than Equirectangular, corresponds to the actual proportions of the scene more realistic, but expresses only the contents of about 140 degrees in the vertical direction;Another example Equisolid projection, it was also called "asteroids" or "720" panoramic views, it can even be a 360-degree vertical horizon show, but only if users don't care about huge distortion may lead to loss of quality: Jeremy Scott iPad Mini Case
So, is there a projection generated picture, is capable of covering at least 360 degrees horizontally and 180 degrees vertically and there is no distortion of the picture?The answer is: no distortion of single image projection method, does not exist. However, if the results of the projection screen is not a single image, there are:
If you happen to be a graphics or virtual reality software development practitioners, this picture should be very familiar to you, this is the Cubemap (cube).It is equivalent to a cube made up of six image mosaic box, if we assume that the observer is located in the center of the cube, each image corresponds to a surface of a cube, and equivalent in the physical space the horizontal and vertical are 90 degrees FOV. Viewers are surrounded by six pictures in the Center, eventually sight range can also achieve the same level of 360 degrees, 360 degrees vertically and are absolutely without any distortion of the frame.Is as follows:
This is a very good projection results, and if you happen to know how to use offline rendering software or plug-ins to produce and output a panoramic content, this must be the most appropriate choice. However, during the actual shooting is almost impossible for us to record this cube map, simple reason – our existing equipment to do it.2, joining and integrationIf said has six Taiwan camera, they of FOV angle was strictly qualified for level and vertical are is 90 degrees, then made a meticulous of bracket, put this six Taiwan camera firm and stable to installation to bracket Shang, ensure they of Center points strictly coincide in with, and their towards a direction–such words, output of image may can just meet cubic figure of standard, and can directly using.However, regardless of camera lens of photographic area, focal length parameter (and so calculation get of FOV depending on field angle), also is bracket of steel body structure design and making, are cannot ensure precise to reached above requirements of parameter, several mm of optical or mechanical errors seems harmless, but for fit of cubic figure image for, must will in eventually rendering of immersed type scene in the left a article or more article obviously of crack. Mention also supports vibration problem occurs during exercise, as well as the camera lens focus shift problem of aging, these seemingly small trouble hard enough for us to just build a physical model of ideal come to naught.Ideal and reality of gap so of big, fortunately we also has solution of approach–Yes, if in mosaic of place left enough big of redundant, then right recognition and processing two Taiwan camera picture coincide of regional, such not on can do six site picture of output and composition panorama content has did–and this is panorama content making of another a big magic, image of mosaic and edge fusion.
Below is a 360Heros series of panoramic cameras.It uses 6 GoPro Sports cameras and a stent to help complete filming, six cameras were headed in different directions, if we adopt the 4X3 wide angle setting, horizontal and vertical FOV angle of about 122 degrees and 94 degrees.
Panoramic video mosaic reads six camera inputs and output software streaming or video file, and set them in the physical location of information on the bracket (or direct access to a digital camera records the information). So that we have enough video content covering the entire FOV.
As we described earlier, because there is no precise alignment, so need to be on the point of the view of each camera to provide the necessary redundancy, thus obtains the video there is a certain overlapping relationship between each other, direct output when panorama, there may be significant overlapping regional or wrong side. Although currently several common of panorama video processing tool, such as VideoStitch,Kolor, has must degree of automatically edge fusion function, but many when we also is inevitably to himself manual to CD cut and adjustment these edge regional (for example Xia figure in the using PTGui to for the site picture seams of Amendment), takes picture quality more high or orthodontic change small of edge regional, and ensure picture Zhijian is strictly on align of.
Such work is time-consuming and there is an important prerequisite, is used as the input source image must be able to cover a 360-degree view and there are redundant.As the we zhiqian by calculation of, if used six a camera assembled of way, so each camera of FOV angle not should is less than 90 degrees, for GoPro Hero3 series camera for, at must used 4x3 of wide depending on domain mode, if is 16x9 of wide high than set, so vertical direction of FOV angle is may cannot reached requirements of numerical, then produced "anyway are mosaic not Shang" of problem–certainly we can through in bracket Shang adjustment all camera of towards angle, Or increase the number of cameras, to avoid this problem, but no matter from which point of view, close to the 1x1 aspect ratio wide field camera is a better choice.If only to output a panoramic picture, then the above steps usually have more than enough, don't need to think about more things. However, not moving pictures is hard to get people wearing VR helmets, Wah yelled can see flames, or ghosts haunt the dynamic scene more exciting. If you are thinking about how to make the VR movies, then there is a problem I must mention, that is-Synchronicity – in simple terms, is how all the cameras in your hands start precise guarantees, as well as in the process of recording frame rate of consistency.This does not seem to be any problem, however, if the start time of the two cameras is inconsistent, will directly affect their alignment and stitching results-even if the scene has a lot of dynamic elements or camera position in the process of change, the results may be out of alignment. Therefore, require the involvement of a large number of cameras at the same time panoramic work, Sync start synchronous recording needs become more important.Basic hardware to solve this problem, you can use "phase synchronization lock" (Genlock) technology, passing through an external device run time code to control the camera's synchronization (typical for example Red One professional movie cameras). Of course, not all cameras have dedicated Genlock interface, in that case, can also be considered traditional or look slightly "shanzhai" method of synchronization, such as unfair on the road, with a roar … …When the shooting began, actor with a snarl, or whack smack. And then the process for stitching, find each video corresponding to the roar of time nodes's place as the synchronization starts, and then the stitching of panoramic video. This method, although there is no accuracy whatsoever, but again it did not cost anything extra cost but ensures basic synchronization after start, then subtle adjustment and video slot, but no doubt considerably simplifies the difficulties of post-production.Similar methods as well as to all the cameras covered with black cloth, and began to shoot fast-draining, and so on. Anyway on the premise of hardware cannot be fully equipped, that is, eight each time.3-d and pseudo-stereoCareful you may have discovered, all discussed before shooting panoramic video ignores a vital point: regardless of the projection, generated only a 360-degree panoramic content, on the PC or Web-side to watch, of course without any problem, but if you want to enter on a VR helmet-mounted display, the result is probably not correct. In order to assign a picture to icing on the cake and presents to the human eye, the content we provide separated around eye level display mode must be used:
It seems only to be a copy of the original panorama, but careful observation, location near the border of the frame you will find around the contents of the screen there is a certain amount of offset. Because people's eyes there is a difference of perspective, eyes see certain differences in the images, and through the brain of the solution can be solid feel. Scenes from closer to the human eye, this disparity is more obvious, the scenery in the distance or relative does not have a strong sense of depth.
Any existing VR glasses required a structure designed to ensure that the wearer's left eye and can only see half of the screen, which is seen after separating the left and right eye images, respectively, so as to simulate the human eye works.
This case, panoramic view of filming equipment also corresponds to the need to make some changes, such as the original 6 cameras to 12 cameras, each direction has left two cameras are responsible for shooting; support forms are so entirely different from the original design (shown in 360 Heros3 Pro12, uses 12 GoPro sports camera).
Software for stitching and fusion, and nothing special needs to be done, just be twice six video stream is read, output two different panoramic video, one for each left and right eye images. Then through tools and applications will merge them into one picture.Of course, different way there, such as from 2011 all shook the Kickstarter to raise, but until now VR panorama fire but still no Panono issued on schedule, its design principles are uniformly distributed in the sphere of 36 cameras to capture, mosaic and the panoramic views of the left and right eye image.The designs seem to be pulling up, it is actually the same: 36 towards the different camera image, superimposed on the cover with 360-degree vertical 360 degree field of vision and scope, and will be able to cover it twice! Attitude plus their precise structure design and installation, so that you can calculate from the inside out after stitching panoramic images, and in accordance with the standard output of left and right eye images and video streams or files that can output the actual resolution is considerable.
And similar to the Bublcam (four ball super wide-angle), Nokia OZO (8 ball lens) and Jaunt in the research and development of products and so on. They all have the ability to output stereoscopic panoramic content in the form of.Of course, the bad case, we still have a choice, is created a stereo model all by himself … … Jeremy Scott iPad Mini CaseTo copy the original panorama into two halves, one offset to the left, another right, then do a mild perspective transformations (to simulate the deflection of angle of view). That it constituted a "solid" picture in many of the cases have some stereo effect of cheating, but near the scene, or the left and right eye images of the scene exist when blocking (such as analog face pasted on the door, one eye of latch blocking scenario), you will have obvious flaws. Of course, for contents at silly stages of the VR panorama enthusiasts, this is not a serious problem for the time being.60 votesStarVR virtual reality glasses\rStarVR 5K how many gimmicks it is hard to say, because of its ultra-high resolution set up, needs the support of developers, but also gamers PC configuration is good enough. StarVR will soon be publicly displayed, and the walking dead new works will be the first games to support it.\rView details of the voting »