Display effect is the facade of mobile phones and other portable devices. It will have a direct impact on the communication between users and the digital world, and even determine the purchase intention of consumers. This is also why touch display technology will promote the sales of iPhones, and even cottage mobile phone manufacturers will try their best to tilt the limited cost to display devices. In addition to the basic requirements of energy consumption, reliability, withstand voltage and display quality, for smart phones, consumers have higher and higher requirements for more real and interesting display experience. These requirements promote the application of 3D display and micro projection technology in mobile phones. The former can bring more real experience for consumers, while the latter can solve the problem that the display screen of mobile phone is too small, and also can derive a lot of interesting and practical applications. At the 4th mobile handheld display technology conference held at the same time of the portable product innovation technology exhibition, Dr. Xu Liangfeng, director of Asia innovation system engineering of Nokia Beijing Research Institute, systematically expounded his views on the application of 3D display and micro projection technology in mobile phones, as well as Nokia's latest research progress in related fields.

Adding TC layer to solve the problem of limited viewing angle and damaged resolution of 3D display

The current three-dimensional display technology can be divided into four categories: stereoscopic glasses, automatic stereoscopic display, holography and three-dimensional display. The first two of them use parallax to give people 3D images The sense of display enables the left and right eyes of the observer to see images with parallax, thus deceiving the brain and making people feel 3D; holography does not use digital means, but the interference and diffraction of light waves. It can only generate static 3D optical scenes, and it also requires the viewing angle. The amount of 3D image information formed by holography is very large, so it is suitable for mobile phones and other tapes For the application of mobile devices with limited width, 3D display using holographic technology is not suitable; at present, volume 3D display technology can be roughly divided into scanning volume display and solid-state display. Due to the constraints of reliability and performance, volume 3D display technology has not yet reached the stage of large-scale commercial use.

Comparatively speaking, the most widely used three-dimensional technology is the former two so-called "pseudo three-dimensional" technology. The first technology is the technology used in 3D movies such as avatar. In this kind of application, the observer wears special glasses and coordinates with the image on the screen through the glasses grating to produce a three-dimensional effect. In mobile phones and other portable applications, it is not convenient to wear glasses, so the grating is placed on the display screen, which is the so-called automatic split stereoscopic display technology. Comprehensive comparison shows that this technology is the most realistic choice of 3D display technology in mobile phone applications.

Nintendo, LG and HTC have all launched products with automatic split stereoscopic display technology, namely naked eye 3D display application, but there are also some problems that seriously affect the user experience. For example, you must be in front of the display screen at a certain optimal angle to view the 3D display effect. If you move the position slightly, it will be seriously affected. However, in mobile applications, on the one hand, users are often in motion, and the content of other action games (such as racing) is constantly changing. If you have to enjoy 3D effects from a specific angle, it will obviously seriously affect the user's experience. Moreover, the abnormal frequent switching of 2D display effect and 3D display effect due to the perspective problem will even make consumers feel dizzy and other discomfort. In addition, the vast majority of such three-dimensional technology needs to halve the image resolution. Although the three-dimensional film like Avatar has adopted timing control technology to solve this problem, it still exists in many mobile terminal products. If we want to ensure the 3D effect of multi-user viewing, we will encounter more problems.

Xu Liangfeng said that his team has done some exploration to solve these problems. The solution is to place a transmission unit (TC) layer made of special liquid crystal between the backlight and LCD, or directly put it on the LCD. After adding electric field to the TC layer, according to the characteristics of the liquid crystal itself, the liquid crystal cells in the TC layer will be reordered, so as to realize this special grating which can be controlled by electricity. For example, if the electric field is not applied, all light can pass through the surface of the display screen. When the electric field is applied, the liquid crystal of the TC layer will turn according to the designer's intention, and only the light that the designer wants to pass through can pass through, so as to control the viewing angle of the viewer. In a word, this technology adds a layer to the display screen, which plays a special grating role to control the viewing angle TC layer of perspective.

Using this technology, Nokia has made several beneficial attempts. First, for 2D display, TC layer is used to control the viewing angle to improve the privacy of mobile phone. The previous mobile phone display angle is 30 ° to 45 ° if users watch some messages or pictures in public that they don't want to be seen by others, they will have the psychological concern that their privacy will be seen intentionally or unintentionally. After adding the TC layer, only the vertical light can come out, and the light next to it is absorbed. The display content on the mobile phone is very clear on the front, but it becomes blurred beyond 30 degrees, which effectively eliminates the psychological discomfort of users when their privacy is peeped. Second, through the cooperation of multiple TC layers to control the light transmission direction, the image can enter the observer's left eye or right eye at a fast enough speed according to the demand, which can achieve the same 3D display effect with no loss of image resolution as the movie. In addition, the TC layer does not affect the original 2D display effect of the mobile phone display when the electric field is not applied, so that the 2D image and 3D image can be automatically switched through the on-off of the electric field on the TC layer. Third, the front camera of the mobile phone is used to track the face. According to the distance and angle between the face and the mobile phone, the arrangement of gratings on the mobile phone is calculated dynamically. In this way, when the face moves slightly, the gratings on the display will be adjusted dynamically, so as to solve the problem of limited viewing angle of 3D display.

Non contact gesture input method to improve the use experience of mobile phone micro projection

The application of micro projection in mobile phones has gradually begun. Samsung built it into mobile phones, while LG used it as peripherals. In the Indian market, consumers have a strong demand to watch TV through mobile phones. This is because India's infrastructure is poor, and power failure is a common occurrence. If consumers are watching a favorite TV play or cricket game, they can continue to watch it on their mobile phones through micro projection mobile phones to help these users solve their urgent needs.

The use of micro projection in mobile phones is affected by brightness, power consumption, heat dissipation and cost, but these aspects are gradually improving. In addition to the impact of these hard indicators, Xu Liangfeng believes that some uncomfortable user experience has seriously affected the popularity of micro projection in mobile phones. For example, users must find a more stable use position when using the micro investment mobile phone, because if they put it on the hand, the focus of the slightly moving image will change, and they need to adjust the settings again. But even if they put it on a fixed position, when the user operates the micro investment function, as long as there is contact, the vibration and position change will also bring the same problem. At this time, we can use non-contact gesture input method to solve this problem.

Specifically, it is to use the camera of mobile phone to track people's actions, and then use the processor to analyze the actions for judgment, so as to issue the corresponding instructions. There are also some difficulties to realize this function. The first is to put the micro projection module into the mobile phone, which is a challenge to the machine structure design. Second, how to solve the heating problem of micro projection module not only affects the micro projection itself, but also affects other key components of mobile phone around it. These are just the problems of realizing micro projection function, and there are some challenges in realizing gesture recognition. Gesture recognition has a high demand on the computing power of mobile phone processor. If you want to reduce the computing load of mobile phone processor, gesture recognition algorithm should be as simple and reliable as possible. In addition, the light source is also a very big problem, because the mobile phone camera is used to track people's actions, and the effect may be very different in different light environments. It's a good method in good light conditions. Another algorithm may be needed in indoor lighting, and the projection needs to be used in a dark environment to achieve better results. Effective gesture recognition in a dark environment needs to solve the problem of light source. According to Xu Liangfeng, his team uses infrared light source and infrared camera to detect people's gestures.

Now, Xu Liangfeng's team has realized some simple gesture control of projection mobile phones, such as turning pages of pictures and converting mobile TV channels by waving hands up, down, left and right. Moreover, even in a relatively dark environment, the reliability of such gesture recognition has been high.

In addition, Xu Liangfeng's team has developed a technology called "touch on the wall", which requires the following changes to the existing mobile phones: adding micro projection function, configuring front camera, and configuring two infrared light sources. When using, after the content of the mobile phone is put on the wall, the user can directly touch the displayed content on the wall, such as writing words and drawing pictures. In fact, it enlarges the touch on a small mobile phone screen to a large enough wall, which also improves the user experience of consumers and expands a new application mode and environment for mobile phones.