CN1423228A - Apparatus and method for identifying gazing direction of human eyes and its use - Google Patents
Apparatus and method for identifying gazing direction of human eyes and its use Download PDFInfo
- Publication number
- CN1423228A CN1423228A CN 02131191 CN02131191A CN1423228A CN 1423228 A CN1423228 A CN 1423228A CN 02131191 CN02131191 CN 02131191 CN 02131191 A CN02131191 A CN 02131191A CN 1423228 A CN1423228 A CN 1423228A
- Authority
- CN
- China
- Prior art keywords
- image
- people
- center
- eyes
- colour
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 210000001508 eye Anatomy 0.000 title claims abstract description 45
- 238000000034 method Methods 0.000 title claims abstract description 39
- 238000005516 engineering process Methods 0.000 claims abstract description 20
- 238000013528 artificial neural network Methods 0.000 claims description 20
- 238000012545 processing Methods 0.000 claims description 13
- 238000012549 training Methods 0.000 claims description 11
- 230000001815 facial effect Effects 0.000 claims description 10
- 210000005252 bulbus oculi Anatomy 0.000 claims description 7
- 230000000007 visual effect Effects 0.000 claims description 5
- 239000000284 extract Substances 0.000 claims description 4
- 238000003384 imaging method Methods 0.000 claims description 4
- 210000002569 neuron Anatomy 0.000 claims description 4
- 238000012360 testing method Methods 0.000 claims description 4
- 238000000605 extraction Methods 0.000 claims description 3
- 238000013461 design Methods 0.000 claims description 2
- 239000004744 fabric Substances 0.000 claims description 2
- 238000005259 measurement Methods 0.000 claims description 2
- 238000005192 partition Methods 0.000 claims description 2
- 238000001514 detection method Methods 0.000 claims 2
- 238000001914 filtration Methods 0.000 claims 2
- 238000003062 neural network model Methods 0.000 claims 1
- 230000004044 response Effects 0.000 abstract description 4
- 244000287680 Garcinia dulcis Species 0.000 description 2
- 238000004378 air conditioning Methods 0.000 description 2
- 230000004069 differentiation Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 1
- 230000008921 facial expression Effects 0.000 description 1
- 210000003128 head Anatomy 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 230000001537 neural effect Effects 0.000 description 1
- 230000009340 pathogen transmission Effects 0.000 description 1
Images
Landscapes
- Image Analysis (AREA)
Abstract
本发明涉及机器视觉与自动控制技术,特别是识别人眼注视方向的装置和方法及其在智能控制中的应用。本发明包括由摄象头、图象接口和处理器所构成的硬件设备,定时循环图像采集与人眼注视方向识别,以及将人眼注视用作控制指令或辅助信息的非接触智能控制技术。本发明可以用于家用电器、游戏机、医护仪器、智能机器人以及残疾人和某些特需者的非接触控制装置,应用注视方法开启和关闭空调等简单电器,“唤醒”处于待命状态的机器人或计算机,对驾驶员视线偏离的安全提醒,也可在以基于语音或手势的非接触智能控制技术中加入本技术,用于判别其语音或手势信号是否是用户发出的有意控制指令等,从而使机器对人的非接触控制的响应更加主动、友好和可靠。
The invention relates to machine vision and automatic control technology, in particular to a device and method for identifying the gaze direction of human eyes and its application in intelligent control. The invention includes a hardware device composed of a camera, an image interface and a processor, timing cycle image collection and human gaze direction recognition, and a non-contact intelligent control technology using human gaze as control instructions or auxiliary information. The present invention can be used in non-contact control devices of household appliances, game machines, medical instruments, intelligent robots, disabled persons and some special needs. It uses the gaze method to turn on and off simple appliances such as air conditioners, and "wakes up" robots or robots in a standby state. The computer can also add this technology to the non-contact intelligent control technology based on voice or gesture to determine whether the voice or gesture signal is an intentional control command issued by the user, etc. The response of the machine to the non-contact control of the human is more active, friendly and reliable.
Description
Affiliated technical field
The present invention relates to machine vision and automatic control technology, particularly discern the apparatus and method of gazing direction of human eyes and the application in Based Intelligent Control thereof.The present invention is a kind of new technology of differentiating gazing direction of human eyes based on video image, this technology does not need extra utility appliance except single-camera, processor and visual interface, the recognition methods simple and fast, can make response more active, the close friend and definite of machine in a lot of applications of machine vision and Based Intelligent Control to the people.
Background technology
In the reciprocal process between people and people, human to facial expression, particularly the direction of gaze to human eye has extremely sensitive discriminating power, and machine vision and people's vision is compared, and still is in very low level on the one hand at this.The identification of existing direction of gaze, the main special utility appliance that is contained on subject's head that relies on, as infrared illuminator, reflected light detector and dibit gamma camera etc.Though these devices can have higher measuring accuracy to direction of gaze, its range of application is very restricted.
In present widely used computing machine and the household electrical appliance control technology,, belong to the keying way of contact basically, proved that this is a kind of important channel of pathogen transmission as telepilot, keyboard, mouse and touch-screen etc.In addition, from the viewpoint of " people-oriented ", also there are many natures inadequately and local easily in the use in this class device.Intelligent control technology based on the vision and the sense of hearing, it is the non-contact intelligent control technology of natural interactive style between a kind of people of approaching and the people, it is the developing direction of intelligent control technology, not only convenient, nature, contactless germ infect, and to computing machine and digitized popularize also significant.
Summary of the invention
The purpose of this invention is to provide a kind of apparatus and method and application thereof of discerning gazing direction of human eyes.The present invention will whether watch attentively as the startup of machine, close or user's request information in the non-contact intelligent machine control technology based on the machine vision and the sense of hearing, make machine to people's demand response more initiatively, friendly and definite.These machines comprise the extremely great household electrical appliance of market scale, game machine, medical care instrument, driver safety companion, intelligent robot and disabled person and some person's of special procuring telechiric device etc.For example, the application method of watching attentively is started or is closed electrical equipment such as air-conditioning or electric fan, " wake " robot that is ready up or be in the computing machine of dormant state, also can in voice or gesture intelligent control technology, add the gazing direction of human eyes recognition technology, whether be intentionally steering order that user send, can improve the reliability of Based Intelligent Control greatly if being used to differentiate its voice or hand signal.
The present invention comprises that mainly imaging apparatus (stylus: gamma camera or image sensor), image storage device (storer), image recognition device and image-processing system (computing machine, single card microcomputer or DSP process chip) constitute; Described imaging apparatus is used to gather the scene image, described image storage device is used for the picture information of storage of collected scene image, described image recognition device is used for detecting the picture information that the picture information of obtaining in real time satisfies predetermined condition, described image-processing system is used for judging that automatically the scene graph image information of being gathered is steering order information or supplementary, particularly gazing direction of human eyes is as steering order or supplementary, and these steering order information or supplementary can be used for the control of non-contacting electrical equipment.
The present invention is installed in hardware device on the controlled electrical equipment, stylus points to the direction at electrical equipment user place, make stylus regularly constantly gather the scene image, and by in the process software scene image that judgement is gathered automatically whether the people being arranged, if nobody then continues circle collection scene image; If someone judges further then whether people's eyes are watching camera attentively, and is used for non-contacting electrical apparatus control system with this as steering order or supplementary.
Application the present invention check the differentiation of direction of gaze, and the result shows that identification is subjected to illumination effect little, and is convenient and reliable, and how much relevant the effect of direction of gaze identification simultaneously is with crowd's size that is identified and training sample.When using from 126 breadth portions image sample that 13 different people are gathered (comprise and watch sample attentively and non-ly watch each 63 in sample attentively) when carrying out the training of parameter extraction and neural network, in order to guarantee identification certainty to different people, the discrimination of training sample is 98.4%, test as test sample book with 127 non-training samples then, correct recognition rata can reach 84.3%.Reduce the crowd's be identified number or increase the number of training sample, correct recognition rata is improved further.Should be pointed out that in above-mentioned recognition result misjudged all is to watch sample attentively, be about to watch attentively declared and watch attentively, non-ly watch sample attentively all identification is correct into non-.The condition for identification of sample is watched in strictness attentively, be for fear of when direction of gaze is used as control information, owing to watching erroneous judgement attentively and cause maloperation, and can also in next round is handled, obtain identification, delay is arranged only slightly the erroneous judgement of watching attentively for watching attentively to differentiate with non-.At present, to discern the needed time be about 0.1 second to each direction of gaze.Processing time also can further reduce by the raising of hardware speed.
The present invention can be used for household electrical appliance, game machine, medical care instrument, driver safety companion, intelligent robot and disabled person and some person's of special procuring non-contact control device, judgement as direction of gaze can be used for simple appliances such as opening and closing air-conditioning, " wake " robot or the computing machine that are ready up, the safety that pilot's line of vision is departed from is reminded etc.Also can in voice or gesture intelligent control technology, add the gazing direction of human eyes recognition technology, whether be used to differentiate voice or hand signal is the steering order intentionally that the user sends, can improve the reliability of Based Intelligent Control greatly, make response more active, the close friend and definite of machine the people.For example present technique is applied in the gesture identification system, the implementation result of present technique is: can distinguish and have a mind to and gesture unintentionally, reduce the False Rate of gesture; Also can make the design of gesture instruction more natural, the gesture that needn't strict regulations adopts those people to be of little use or to be difficult to make.
Description of drawings
Fig. 1: image processing and direction of gaze discrimination technology process flow diagram.
Fig. 2: the geometric representation of face and plane of delineation angle Φ.A is the center of nose, B and E are respectively the left and right sides eye center on people's face plane, ABE forms people's face plane, B and C are respectively the left and right sides eye center on the image plane, ABC is an image plane, and the angle of the people's face plane and the plane of delineation is Φ, and AB is the straight line of two Plane intersects, on image plane, the angle of the line of two center B and C and nose center A is θ.
Fig. 3: judge the processing flow chart whether people is arranged in the scene.
Fig. 4: people's face location and the processing flow chart of cutting apart
Fig. 5: the positioning flow figure that windows of eyes and nose.
Fig. 6: facial parameters is extracted and direction of gaze identification processing flow chart.
Embodiment
The present invention is described in detail as follows with reference to accompanying drawing:
Shown in 1, the technological process of image acquisition of the present invention and processing comprises following 5 steps:
(1) whether people's judgement is arranged in Motion Recognition and the scene
With the identification of taking exercises of time method of difference: at first choose relatively-stationary background image, gather a width of cloth new images and background image at regular intervals and subtract each other, can judge whether have moving object in scene, to occur by the difference image.In case finding has moving object to enter scene, promptly is partitioned into the moving object image, beginning is further handled; Otherwise, continue to do the next round Motion Recognition.If there is moving object to enter scene, then adopt the feed-forward type neural network that the moving image that extracts is carried out colour of skin identification, when finding the object identical with people's colour of skin to be arranged and satisfy certain size condition, we just think that the someone enters scene, begin next step processing; Otherwise, from restarting to do the next round Motion Recognition.
The neural net method of people's colour of skin identification is adopted in above image acquisition and processing, and its structure is
4-3-1, wherein 3 in 4 of input layer nodes are respectively red, green, the blue components of image point, the 4th node is the biasing input; Hidden layer is got 3 neurons through repetition test and definite; The neuron of output layer exports+1 when being input as the colour of skin, then export-1 when the non-colour of skin is imported.This neural network adopts the BP learning algorithm to train, and training sample is taken from the actual measurement colour of skin and the non-colour of skin colour signal of the colour of skin under four kinds of situations such as daylight lamp, incandescent lamp, sun frontlighting and backlight of different people.Neural network after the training has good reliability and generalization to the identification of the colour of skin.Lighting condition change less or user number more after a little while, the number of training sample can reduce.Detailed process is seen Fig. 3.
(2) people's face is located and is cut apart
Having in the moving image identical of detecting with people's colour of skin, carry out the filling of colour of skin image and cut apart with " mountain peak algorithm ", and the geometric properties of utilization people face is from wherein going out people's face image respectively, see for details the paper that the applicant delivers [Yuan Jing and etc., " a kind of " based on the colour of skin and Facial Recognition, photoelectron laser, 2002, Vol.13, No.4, pp.394-397.], if nobody's face then returns the first step in the broca scale picture.Detailed process is seen Fig. 4.
(3) eyes location and nose location
Adopt the dynamic partition threshold value to carry out binaryzation to the facial image that is partitioned into.Because the black picture element at human eye and nose place is more, utilizes line by line and can determine the position of eyes and nose, and be partitioned into the visual window that comprises eyes and nose with the method for adding priori by column scan.Detailed process is seen Fig. 5.
(4) relevant with direction of gaze facial geometrical characteristic parameter extracts
Facial geometrical characteristic parameter comprises: the center of nose, left and right sides eye center and eyeball center on people's face image plane.Adopt the subregion dynamic thresholding method to carry out binaryzation again to the eyes window that is partitioned into, determine the center of eyes with the rectangle frame matching method, use template matching method (or mountain peak algorithm) to determine the eyeball center again, deduct eye center with the eyeball center and can determine the offset of eyeball with respect to eyes; With the subregion dynamic thresholding method nose window is carried out binaryzation equally, determine the center (being the projected position of nose on image plane) of nose with template matching method.Suppose that B and C are respectively the left and right sides eye center on the image plane, A is the center of nose, and then ABC is the image plane (see figure 2).In Fig. 2, we represent left and right sides eye center on people's face plane respectively with B and E.Suppose that the straight line that people's face plane and image plane intersect is the line AB at nose center and left eye center, ED is the perpendicular line of E to AB, and then ∠ EDC promptly is the angle Φ of the people's face plane and the plane of delineation.Here, the center E of right eye is according to geometric relationship supposition on people's face plane, and it can be determined by the angle theta of two center B on the image plane and C and they and nose center A line.Bright through theoretical reckoner, known straight line AB and the relational expression between AC and their angle theta are on angle Φ and the image plane:
(5) based on the identification of the artificial neural network direction of gaze of facial geometrical characteristic parameter
The identification of so-called direction of gaze is exactly to differentiate the eyes eyeball whether to watch camera lens attentively.Two-layer feed-forward type neural network is set up in the recognition methods of the artificial neural network direction of gaze among the present invention exactly, and its structure is
4-4-1With the position offset of two eyeballs that calculate above with respect to eyes, parameters such as the sinusoidal sin Φ of the angle between the people's face plane and the plane of delineation are as input variable, the 4th is input as biasing, by known actual of relevant people watched attentively and the non-study of watching visual sample attentively, can determine the weight parameter of this neural network, neural network after training is finished promptly can be used for differentiating the direction of gaze of human eye, and the result is differentiated in output, detailed process is seen the paper that Fig. 6 and applicant deliver [Wang Yong etc. " based on the differentiation of the gazing direction of human eyes of parameter extraction ", photoelectron laser, 2001, Vol.12, No.12, pp.1284-1287.], and this output information is used for Based Intelligent Control.
Claims (8)
1, a kind of device of discerning gazing direction of human eyes is characterized in that comprising:
Imaging apparatus is used to gather the scene image;
Image storage device is used for the picture information of storage of collected scene image,
Image recognition device is used for identifying the picture information that the picture information of obtaining in real time satisfies predetermined condition; And
Image-processing system is used for judging automatically that the scene graph image information of being gathered satisfies the gazing direction of human eyes of predetermined condition as steering order or supplementary.
2,, it is characterized in that described imaging apparatus is gamma camera or image sensor according to the device of the described identification gazing direction of human eyes of claim 1.
3,, it is characterized in that described image storage device is a storer according to the device of the described identification gazing direction of human eyes of claim 1.
4,, it is characterized in that described image recognition device and image-processing system are respectively computing machine, single card microcomputer or dsp chip according to the device of the described identification gazing direction of human eyes of claim 1.
5, a kind of method of discerning gazing direction of human eyes is characterized in that comprising the steps:
In the dynamic scene image, adopt difference algorithm and neural network colour of skin filtering method to detect the people, with filtering of the neural network colour of skin and mountain peak algorithm people's face is positioned then and cut apart, the position of orienting eyes and nose with partitioned binarization and picture element scan method, extraction is used for determining the geometric parameter of eyeball skew and people's face orientation, set up the feed-forward type neural network model, discern watching information attentively with these parameters, draw the recognition result of watching attentively or not watching attentively, and the process of finishing these technical essentials.
6, a kind of method of discerning gazing direction of human eyes is characterized in that comprising the steps:
(1) in motion and the scene whether people's judgement is arranged
With the identification of taking exercises of time method of difference: at first choose relatively-stationary background image, gathering a width of cloth new images and background image at regular intervals subtracts each other, can judge whether have moving object in scene, to occur by the difference image, there is moving object to enter scene, promptly be partitioned into the moving object image, beginning is further handled, adopt the feed-forward type neural network that the moving image that extracts is carried out Face Detection, when finding the object identical with people's colour of skin to be arranged and satisfy certain size condition, just think that the someone enters scene, begin next step processing; Otherwise, from restarting to do the next round motion detection;
Described neural network structure is 4-3-1, and wherein 3 in 4 of input layer nodes are respectively red, green, the blue components of image point, and the 4th node is the biasing input; Hidden layer is got 3 neurons through repetition test and definite; The neuron of output layer exports+1 when being input as the colour of skin, then export-1 when the non-colour of skin is imported; This neural network adopts the BP learning algorithm to train, and training sample is taken from the actual measurement colour of skin and the non-colour of skin colour signal of the colour of skin under daylight lamp, incandescent lamp, sun frontlighting and four kinds of situations of backlight of different people;
(2) people's face is located and is cut apart
Having in the moving image identical of detecting, carry out the filling of colour of skin image and cut apart with the step of mountain peak algorithm design, and the geometric properties of utilization people face is from wherein going out people's face image respectively with people's colour of skin;
(3) eyes location and nose location
Adopt the dynamic partition threshold value to carry out binaryzation to the facial image that is partitioned into; Because the black picture element at human eye and nose place is more, utilizes line by line and can determine the position of eyes and nose, and be partitioned into the visual window that comprises eyes and nose with the method for adding priori by column scan;
(4) relevant with direction of gaze facial geometrical characteristic parameter extracts
Facial geometrical characteristic parameter comprises: the center of nose, left and right sides eye center and eyeball center on people's face image plane; Adopt the subregion dynamic thresholding method to carry out binaryzation again to the eyes window that is partitioned into, determine the center of eyes with the rectangle frame matching method, determine the eyeball center with template matching method or mountain peak algorithm again, deduct eye center with the eyeball center and can determine the offset of eyeball with respect to eyes; With the subregion dynamic thresholding method nose window is carried out binaryzation equally, determine the center of nose with template matching method; Suppose that B and C are respectively the left and right sides eye center on the image plane, A is the center of nose, and then ABC is image plane; B and E represent the left and right sides eye center on people's face plane respectively.Suppose that the straight line that people's face plane and image plane intersect is the line AB at nose center and left eye center, ED is the perpendicular line of E to AB, and then ∠ EDC promptly is the angle Φ of the people's face plane and the plane of delineation; Here, the center E of right eye is according to geometric relationship supposition on people's face plane, and it can be determined by the angle theta of two center B on the image plane and C and they and nose center A line; Known straight line AB and the relational expression between AC and their angle theta are on angle Φ and the image plane:
(5) based on the identification of the artificial neural network direction of gaze of facial geometrical characteristic parameter
The identification of so-called direction of gaze is exactly to differentiate the eyes eyeball whether to watch camera lens attentively, set up two-layer feed-forward type neural network, its structure is 4-4-1, with the position offset of two eyeballs that calculate above with respect to eyes, the sinusoidal sin Φ of angle between the people's face plane and plane of delineation parameter is as input variable, the 4th is input as biasing, by known actual of relevant people watched attentively and the non-study of watching visual sample attentively, carry out the neural network colour of skin, people's face and the step of watching the discriminant software design attentively, with the BP algorithm with resultingly watch parameter attentively neural network is trained, can determine the weight parameter of this neural network, neural network after training is finished promptly can be used for differentiating the direction of gaze of human eye, and the result is differentiated in output.
7, the application of the device of the described identification gazing direction of human eyes of claim 1, as the non-contact control instruction of opening or closing general electrical equipment, be used to comprise household electrical appliance, game machine, medical care instrument, driver safety companion, intelligent robot and disabled person and some person's of special procuring non-contact control device with people's eye fixation stylus.
8, the device of the described identification gazing direction of human eyes of claim 1 is based on the application in the non-contact intelligent control technology of voice or gesture, whether add the gazing direction of human eyes recognition technology, differentiating its voice or hand signal is the steering order intentionally that the user sends.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNB021311919A CN1174337C (en) | 2002-10-17 | 2002-10-17 | Method and device for identifying whether human eyes are watching or not and application thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNB021311919A CN1174337C (en) | 2002-10-17 | 2002-10-17 | Method and device for identifying whether human eyes are watching or not and application thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN1423228A true CN1423228A (en) | 2003-06-11 |
| CN1174337C CN1174337C (en) | 2004-11-03 |
Family
ID=4746590
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CNB021311919A Expired - Fee Related CN1174337C (en) | 2002-10-17 | 2002-10-17 | Method and device for identifying whether human eyes are watching or not and application thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN1174337C (en) |
Cited By (60)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN100343867C (en) * | 2005-06-15 | 2007-10-17 | 北京中星微电子有限公司 | Method and apparatus for distinguishing direction of visual lines |
| CN100432897C (en) * | 2006-07-28 | 2008-11-12 | 上海大学 | System and method of contactless position input by hand and eye relation guiding |
| CN101815174A (en) * | 2010-01-11 | 2010-08-25 | 北京中星微电子有限公司 | Control method and control device for camera shooting |
| CN102298443A (en) * | 2011-06-24 | 2011-12-28 | 华南理工大学 | Smart home voice control system combined with video channel and control method thereof |
| CN102404584A (en) * | 2010-09-13 | 2012-04-04 | 腾讯科技(成都)有限公司 | Method and device for adjusting left camera and right camera of scene, 3D glasses and client |
| CN102446270A (en) * | 2010-10-15 | 2012-05-09 | 汉王科技股份有限公司 | Monitoring device and monitoring method based on face recognition |
| CN101383000B (en) * | 2007-09-03 | 2012-08-22 | 索尼株式会社 | Information processing apparatus, and information processing method |
| US8253776B2 (en) | 2008-01-10 | 2012-08-28 | Asustek Computer Inc. | Image rectification method and related device for a video device |
| CN103140862A (en) * | 2010-09-30 | 2013-06-05 | 法国电信公司 | User interface system and method of operation thereof |
| WO2013149466A1 (en) * | 2012-04-05 | 2013-10-10 | 广州广电运通金融电子股份有限公司 | Financial self-service equipment and anti-peeping system and method therefor |
| CN103491848A (en) * | 2011-12-26 | 2014-01-01 | 奥林巴斯医疗株式会社 | Medical endoscope system |
| CN103823560A (en) * | 2014-02-24 | 2014-05-28 | 联想(北京)有限公司 | Control method and device |
| CN103959357A (en) * | 2012-10-19 | 2014-07-30 | 儿童医院 | System, method and computer program for training for ophthalmic examinations |
| CN104253944A (en) * | 2014-09-11 | 2014-12-31 | 陈飞 | Sight connection-based voice command issuing device and method |
| CN104468010A (en) * | 2014-12-16 | 2015-03-25 | 上海斐讯数据通信技术有限公司 | Intelligent terminal and radio station searching method thereof |
| CN104714638A (en) * | 2013-12-17 | 2015-06-17 | 西门子公司 | Medical technology controller |
| CN104718507A (en) * | 2012-11-05 | 2015-06-17 | 松下知识产权经营株式会社 | Autonomous traveling device traveling-information generation device, method, and program, and autonomous traveling device |
| CN104834222A (en) * | 2015-04-30 | 2015-08-12 | 广东美的制冷设备有限公司 | Control method and apparatus for household electrical appliance |
| CN104856729A (en) * | 2015-05-25 | 2015-08-26 | 无锡海斯凯尔医学技术有限公司 | Operating control method of elasticity detection equipment and elasticity detection equipment |
| CN104879882A (en) * | 2015-04-30 | 2015-09-02 | 广东美的制冷设备有限公司 | Method and system for controlling air conditioner |
| CN105094292A (en) * | 2014-05-05 | 2015-11-25 | 索尼公司 | Method and device evaluating user attention |
| CN105700402A (en) * | 2016-03-09 | 2016-06-22 | 北京地平线机器人技术研发有限公司 | Personnel detection-based embedded control system, device and method |
| CN105874405A (en) * | 2013-12-11 | 2016-08-17 | Lg电子株式会社 | Smart home appliances, operating method of thereof, and voice recognition system using the smart home appliances |
| CN105964550A (en) * | 2016-05-20 | 2016-09-28 | 天津工业大学 | Full-automatic logistics robot |
| CN106257355A (en) * | 2015-06-18 | 2016-12-28 | 松下电器（美国）知识产权公司 | Apparatus control method and controller |
| CN106297781A (en) * | 2015-06-24 | 2017-01-04 | 松下电器（美国）知识产权公司 | Control method and controller |
| CN106548157A (en) * | 2016-11-03 | 2017-03-29 | 贺江涛 | A kind of human eye direction of bowl recognition methodss based on bank service robot |
| WO2017054599A1 (en) * | 2015-09-30 | 2017-04-06 | 宇龙计算机通信科技(深圳)有限公司 | A household appliance control method and apparatus |
| WO2017084173A1 (en) * | 2015-11-17 | 2017-05-26 | 小米科技有限责任公司 | Intelligent device control method and apparatus |
| CN107103271A (en) * | 2016-02-23 | 2017-08-29 | 芋头科技（杭州）有限公司 | A kind of method for detecting human face |
| CN107102540A (en) * | 2016-02-23 | 2017-08-29 | 芋头科技（杭州）有限公司 | A kind of method and intelligent robot for waking up intelligent robot |
| CN107238439A (en) * | 2016-03-28 | 2017-10-10 | 厦门神州鹰软件科技有限公司 | Utilize image and the method for planar stepping motor device automatic measurement human body temperature |
| CN107548483A (en) * | 2015-03-27 | 2018-01-05 | 法雷奥舒适驾驶助手公司 | Control method, control device, system and the motor vehicles for including such control device |
| CN107997933A (en) * | 2017-11-27 | 2018-05-08 | 南京大学 | A children's visual function training and rehabilitation system with real-time evaluation function |
| CN108235745A (en) * | 2017-05-08 | 2018-06-29 | 深圳前海达闼云端智能科技有限公司 | robot awakening method, device and robot |
| CN108279778A (en) * | 2018-02-12 | 2018-07-13 | 上海京颐科技股份有限公司 | User interaction approach, device and system |
| CN108510701A (en) * | 2018-03-20 | 2018-09-07 | 深圳市泰和安科技有限公司 | A kind of intelligent smog alarm method, apparatus and system |
| CN108519819A (en) * | 2018-03-30 | 2018-09-11 | 北京金山安全软件有限公司 | Intelligent device processing method and device, intelligent device and medium |
| CN108717324A (en) * | 2018-04-11 | 2018-10-30 | 努比亚技术有限公司 | Intelligent wearable device and its control method, computer readable storage medium |
| CN108803866A (en) * | 2018-03-27 | 2018-11-13 | 北京七鑫易维信息技术有限公司 | The methods, devices and systems of output information |
| CN109041379A (en) * | 2018-08-10 | 2018-12-18 | 广东工业大学 | A kind of Intelligent illuminating method, device, equipment and storage medium |
| CN109325396A (en) * | 2017-08-01 | 2019-02-12 | 欧姆龙株式会社 | Information processing unit and estimation method and learning device and learning method |
| TWI653170B (en) | 2017-11-01 | 2019-03-11 | 宏碁股份有限公司 | Driving notification method and driving notification system |
| CN109464236A (en) * | 2017-09-08 | 2019-03-15 | 拉碧斯半导体株式会社 | Goggle-type display device, line of sight detection method, and line of sight detection system |
| CN109976506A (en) * | 2017-12-28 | 2019-07-05 | 深圳市优必选科技有限公司 | Awakening method of electronic equipment, storage medium and robot |
| CN110045825A (en) * | 2018-03-27 | 2019-07-23 | 杭州凌感科技有限公司 | Gesture recognition system for vehicle interaction control |
| CN110191234A (en) * | 2019-06-21 | 2019-08-30 | 中山大学 | A smart terminal unlocking method based on gaze point analysis |
| CN110765828A (en) * | 2018-07-25 | 2020-02-07 | 卢帆 | Visual recognition method and system |
| CN111128157A (en) * | 2019-12-12 | 2020-05-08 | 珠海格力电器股份有限公司 | Wake-up-free voice recognition control method for intelligent household appliance, computer readable storage medium and air conditioner |
| CN111310630A (en) * | 2020-02-10 | 2020-06-19 | 支付宝实验室(新加坡)有限公司 | Living body detection method and device |
| CN111566582A (en) * | 2018-01-08 | 2020-08-21 | 三星电子株式会社 | Electronic device and control method thereof |
| CN112083795A (en) * | 2019-06-12 | 2020-12-15 | 北京迈格威科技有限公司 | Object control method and device, storage medium and electronic equipment |
| CN113221630A (en) * | 2021-03-22 | 2021-08-06 | 刘鸿 | Estimation method of human eye watching lens and application of estimation method in intelligent awakening |
| US11132543B2 (en) | 2016-12-28 | 2021-09-28 | Nvidia Corporation | Unconstrained appearance-based gaze estimation |
| CN113853598A (en) * | 2019-05-27 | 2021-12-28 | 索尼集团公司 | Artificial intelligence information processing device and artificial intelligence information processing method |
| CN114663864A (en) * | 2022-03-31 | 2022-06-24 | 上海商汤临港智能科技有限公司 | Vehicle window control method and device, electronic equipment and storage medium |
| CN114968166A (en) * | 2021-02-26 | 2022-08-30 | 华为技术有限公司 | Voice interaction method and electronic equipment |
| CN114995643A (en) * | 2022-05-30 | 2022-09-02 | 北京智谱华章科技有限公司 | Human-computer interaction method and device based on knowledge graph of binocular camera |
| KR102456438B1 (en) * | 2022-07-13 | 2022-10-19 | (주)인티그리트 | Visual wake-up system using artificial intelligence |
| CN115218383A (en) * | 2022-06-17 | 2022-10-21 | 青岛海尔空调器有限总公司 | Air conditioning control method, device, electronic device, and storage medium |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2006130010A (en) * | 2004-11-04 | 2006-05-25 | Denso Corp | Visual recognition support apparatus |
| CN101460832B (en) * | 2006-06-08 | 2012-03-21 | 奥林巴斯株式会社 | External appearance inspection device |
-
2002
- 2002-10-17 CN CNB021311919A patent/CN1174337C/en not_active Expired - Fee Related
Cited By (91)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN100343867C (en) * | 2005-06-15 | 2007-10-17 | 北京中星微电子有限公司 | Method and apparatus for distinguishing direction of visual lines |
| CN100432897C (en) * | 2006-07-28 | 2008-11-12 | 上海大学 | System and method of contactless position input by hand and eye relation guiding |
| CN101383000B (en) * | 2007-09-03 | 2012-08-22 | 索尼株式会社 | Information processing apparatus, and information processing method |
| US8253776B2 (en) | 2008-01-10 | 2012-08-28 | Asustek Computer Inc. | Image rectification method and related device for a video device |
| CN101815174A (en) * | 2010-01-11 | 2010-08-25 | 北京中星微电子有限公司 | Control method and control device for camera shooting |
| CN101815174B (en) * | 2010-01-11 | 2015-03-04 | 北京中星微电子有限公司 | Control method and control device for camera shooting |
| CN102404584B (en) * | 2010-09-13 | 2014-05-07 | 腾讯科技(成都)有限公司 | Method and device for adjusting scene left camera and scene right camera, three dimensional (3D) glasses and client side |
| CN102404584A (en) * | 2010-09-13 | 2012-04-04 | 腾讯科技(成都)有限公司 | Method and device for adjusting left camera and right camera of scene, 3D glasses and client |
| CN103140862A (en) * | 2010-09-30 | 2013-06-05 | 法国电信公司 | User interface system and method of operation thereof |
| CN103140862B (en) * | 2010-09-30 | 2016-11-02 | 法国电信公司 | User interface system and operational approach thereof |
| CN102446270A (en) * | 2010-10-15 | 2012-05-09 | 汉王科技股份有限公司 | Monitoring device and monitoring method based on face recognition |
| CN102298443A (en) * | 2011-06-24 | 2011-12-28 | 华南理工大学 | Smart home voice control system combined with video channel and control method thereof |
| CN102298443B (en) * | 2011-06-24 | 2013-09-25 | 华南理工大学 | Smart home voice control system combined with video channel and control method thereof |
| CN103491848A (en) * | 2011-12-26 | 2014-01-01 | 奥林巴斯医疗株式会社 | Medical endoscope system |
| CN103491848B (en) * | 2011-12-26 | 2015-10-07 | 奥林巴斯医疗株式会社 | Medical endoscope system |
| WO2013149466A1 (en) * | 2012-04-05 | 2013-10-10 | 广州广电运通金融电子股份有限公司 | Financial self-service equipment and anti-peeping system and method therefor |
| CN103959357A (en) * | 2012-10-19 | 2014-07-30 | 儿童医院 | System, method and computer program for training for ophthalmic examinations |
| CN103959357B (en) * | 2012-10-19 | 2018-02-02 | 儿童医院 | System and method for eye examination training |
| CN104718507B (en) * | 2012-11-05 | 2017-03-29 | 松下知识产权经营株式会社 | The walking information generation device of automatic walking apparatus, method and automatic walking apparatus |
| CN104718507A (en) * | 2012-11-05 | 2015-06-17 | 松下知识产权经营株式会社 | Autonomous traveling device traveling-information generation device, method, and program, and autonomous traveling device |
| US10269344B2 (en) | 2013-12-11 | 2019-04-23 | Lg Electronics Inc. | Smart home appliances, operating method of thereof, and voice recognition system using the smart home appliances |
| CN105874405A (en) * | 2013-12-11 | 2016-08-17 | Lg电子株式会社 | Smart home appliances, operating method of thereof, and voice recognition system using the smart home appliances |
| CN104714638A (en) * | 2013-12-17 | 2015-06-17 | 西门子公司 | Medical technology controller |
| CN103823560B (en) * | 2014-02-24 | 2017-01-04 | 联想(北京)有限公司 | A kind of control method and device |
| CN103823560A (en) * | 2014-02-24 | 2014-05-28 | 联想(北京)有限公司 | Control method and device |
| CN105094292A (en) * | 2014-05-05 | 2015-11-25 | 索尼公司 | Method and device evaluating user attention |
| CN104253944A (en) * | 2014-09-11 | 2014-12-31 | 陈飞 | Sight connection-based voice command issuing device and method |
| CN104468010A (en) * | 2014-12-16 | 2015-03-25 | 上海斐讯数据通信技术有限公司 | Intelligent terminal and radio station searching method thereof |
| CN107548483A (en) * | 2015-03-27 | 2018-01-05 | 法雷奥舒适驾驶助手公司 | Control method, control device, system and the motor vehicles for including such control device |
| CN104834222A (en) * | 2015-04-30 | 2015-08-12 | 广东美的制冷设备有限公司 | Control method and apparatus for household electrical appliance |
| CN104879882A (en) * | 2015-04-30 | 2015-09-02 | 广东美的制冷设备有限公司 | Method and system for controlling air conditioner |
| CN104834222B (en) * | 2015-04-30 | 2018-11-27 | 广东美的制冷设备有限公司 | The control method and device of household electrical appliance |
| CN104879882B (en) * | 2015-04-30 | 2017-09-22 | 广东美的制冷设备有限公司 | The control method and system of air conditioner |
| CN104856729A (en) * | 2015-05-25 | 2015-08-26 | 无锡海斯凯尔医学技术有限公司 | Operating control method of elasticity detection equipment and elasticity detection equipment |
| CN106257355A (en) * | 2015-06-18 | 2016-12-28 | 松下电器（美国）知识产权公司 | Apparatus control method and controller |
| CN106297781A (en) * | 2015-06-24 | 2017-01-04 | 松下电器（美国）知识产权公司 | Control method and controller |
| WO2017054599A1 (en) * | 2015-09-30 | 2017-04-06 | 宇龙计算机通信科技(深圳)有限公司 | A household appliance control method and apparatus |
| US10481569B2 (en) | 2015-09-30 | 2019-11-19 | Yulong Computer Telecommuncation Scientific (Shenzhen) Co., Ltd. | Household appliance control method and device, and central processing device |
| WO2017084173A1 (en) * | 2015-11-17 | 2017-05-26 | 小米科技有限责任公司 | Intelligent device control method and apparatus |
| RU2656690C1 (en) * | 2015-11-17 | 2018-06-06 | Сяоми Инк. | Method and device for controlling intelligent equipment |
| US9894260B2 (en) | 2015-11-17 | 2018-02-13 | Xiaomi Inc. | Method and device for controlling intelligent equipment |
| EP3422246A4 (en) * | 2016-02-23 | 2019-10-23 | Yutou Technology (Hangzhou) Co., Ltd. | Method for awakening intelligent robot, and intelligent robot |
| CN107103271A (en) * | 2016-02-23 | 2017-08-29 | 芋头科技（杭州）有限公司 | A kind of method for detecting human face |
| WO2017143948A1 (en) * | 2016-02-23 | 2017-08-31 | 芋头科技（杭州）有限公司 | Method for awakening intelligent robot, and intelligent robot |
| CN107102540A (en) * | 2016-02-23 | 2017-08-29 | 芋头科技（杭州）有限公司 | A kind of method and intelligent robot for waking up intelligent robot |
| JP2019512826A (en) * | 2016-02-23 | 2019-05-16 | 芋▲頭▼科技（杭州）有限公司 | Method for calling a smart robot and smart robot |
| US20190057247A1 (en) * | 2016-02-23 | 2019-02-21 | Yutou Technology (Hangzhou) Co., Ltd. | Method for awakening intelligent robot, and intelligent robot |
| TWI646444B (en) * | 2016-02-23 | 2019-01-01 | 芋頭科技(杭州)有限公司 | Method for waking up intelligent robot and intelligent robot |
| CN105700402B (en) * | 2016-03-09 | 2018-03-27 | 北京地平线机器人技术研发有限公司 | Embedded control system, apparatus and method based on personnel's detection |
| CN105700402A (en) * | 2016-03-09 | 2016-06-22 | 北京地平线机器人技术研发有限公司 | Personnel detection-based embedded control system, device and method |
| CN107238439A (en) * | 2016-03-28 | 2017-10-10 | 厦门神州鹰软件科技有限公司 | Utilize image and the method for planar stepping motor device automatic measurement human body temperature |
| CN105964550A (en) * | 2016-05-20 | 2016-09-28 | 天津工业大学 | Full-automatic logistics robot |
| CN106548157A (en) * | 2016-11-03 | 2017-03-29 | 贺江涛 | A kind of human eye direction of bowl recognition methodss based on bank service robot |
| US11132543B2 (en) | 2016-12-28 | 2021-09-28 | Nvidia Corporation | Unconstrained appearance-based gaze estimation |
| US11276402B2 (en) | 2017-05-08 | 2022-03-15 | Cloudminds Robotics Co., Ltd. | Method for waking up robot and robot thereof |
| WO2018205083A1 (en) * | 2017-05-08 | 2018-11-15 | 深圳前海达闼云端智能科技有限公司 | Robot wakeup method and device, and robot |
| CN108235745B (en) * | 2017-05-08 | 2021-01-08 | 深圳前海达闼云端智能科技有限公司 | Robot awakening method, device and robot |
| CN108235745A (en) * | 2017-05-08 | 2018-06-29 | 深圳前海达闼云端智能科技有限公司 | robot awakening method, device and robot |
| CN109325396A (en) * | 2017-08-01 | 2019-02-12 | 欧姆龙株式会社 | Information processing unit and estimation method and learning device and learning method |
| CN109464236A (en) * | 2017-09-08 | 2019-03-15 | 拉碧斯半导体株式会社 | Goggle-type display device, line of sight detection method, and line of sight detection system |
| CN109464236B (en) * | 2017-09-08 | 2022-08-02 | 拉碧斯半导体株式会社 | Goggle-type display device, line of sight detection method, and line of sight detection system |
| TWI653170B (en) | 2017-11-01 | 2019-03-11 | 宏碁股份有限公司 | Driving notification method and driving notification system |
| US10696224B2 (en) | 2017-11-01 | 2020-06-30 | Acer Incorporated | Driving notification method and driving notification system |
| CN107997933A (en) * | 2017-11-27 | 2018-05-08 | 南京大学 | A children's visual function training and rehabilitation system with real-time evaluation function |
| CN109976506A (en) * | 2017-12-28 | 2019-07-05 | 深圳市优必选科技有限公司 | Awakening method of electronic equipment, storage medium and robot |
| CN111566582A (en) * | 2018-01-08 | 2020-08-21 | 三星电子株式会社 | Electronic device and control method thereof |
| CN108279778A (en) * | 2018-02-12 | 2018-07-13 | 上海京颐科技股份有限公司 | User interaction approach, device and system |
| CN108510701A (en) * | 2018-03-20 | 2018-09-07 | 深圳市泰和安科技有限公司 | A kind of intelligent smog alarm method, apparatus and system |
| CN108510701B (en) * | 2018-03-20 | 2019-11-22 | 深圳市泰和安科技有限公司 | A kind of intelligent smog alarm method, apparatus and system |
| CN110045825A (en) * | 2018-03-27 | 2019-07-23 | 杭州凌感科技有限公司 | Gesture recognition system for vehicle interaction control |
| CN110045825B (en) * | 2018-03-27 | 2022-05-13 | 杭州凌感科技有限公司 | Gesture Recognition System for Vehicle Interaction Control |
| CN108803866A (en) * | 2018-03-27 | 2018-11-13 | 北京七鑫易维信息技术有限公司 | The methods, devices and systems of output information |
| CN108519819A (en) * | 2018-03-30 | 2018-09-11 | 北京金山安全软件有限公司 | Intelligent device processing method and device, intelligent device and medium |
| CN108717324A (en) * | 2018-04-11 | 2018-10-30 | 努比亚技术有限公司 | Intelligent wearable device and its control method, computer readable storage medium |
| CN110765828A (en) * | 2018-07-25 | 2020-02-07 | 卢帆 | Visual recognition method and system |
| CN109041379A (en) * | 2018-08-10 | 2018-12-18 | 广东工业大学 | A kind of Intelligent illuminating method, device, equipment and storage medium |
| CN113853598A (en) * | 2019-05-27 | 2021-12-28 | 索尼集团公司 | Artificial intelligence information processing device and artificial intelligence information processing method |
| CN112083795A (en) * | 2019-06-12 | 2020-12-15 | 北京迈格威科技有限公司 | Object control method and device, storage medium and electronic equipment |
| CN110191234A (en) * | 2019-06-21 | 2019-08-30 | 中山大学 | A smart terminal unlocking method based on gaze point analysis |
| CN110191234B (en) * | 2019-06-21 | 2021-03-26 | 中山大学 | Intelligent terminal unlocking method based on fixation point analysis |
| CN111128157A (en) * | 2019-12-12 | 2020-05-08 | 珠海格力电器股份有限公司 | Wake-up-free voice recognition control method for intelligent household appliance, computer readable storage medium and air conditioner |
| CN111128157B (en) * | 2019-12-12 | 2022-05-27 | 珠海格力电器股份有限公司 | Wake-up-free voice recognition control method for intelligent household appliance, computer readable storage medium and air conditioner |
| CN111310630A (en) * | 2020-02-10 | 2020-06-19 | 支付宝实验室(新加坡)有限公司 | Living body detection method and device |
| WO2022179269A1 (en) * | 2021-02-26 | 2022-09-01 | 华为技术有限公司 | Voice interaction method and electronic device |
| CN114968166A (en) * | 2021-02-26 | 2022-08-30 | 华为技术有限公司 | Voice interaction method and electronic equipment |
| CN113221630A (en) * | 2021-03-22 | 2021-08-06 | 刘鸿 | Estimation method of human eye watching lens and application of estimation method in intelligent awakening |
| CN114663864A (en) * | 2022-03-31 | 2022-06-24 | 上海商汤临港智能科技有限公司 | Vehicle window control method and device, electronic equipment and storage medium |
| CN114995643A (en) * | 2022-05-30 | 2022-09-02 | 北京智谱华章科技有限公司 | Human-computer interaction method and device based on knowledge graph of binocular camera |
| CN114995643B (en) * | 2022-05-30 | 2025-07-22 | 北京智谱华章科技股份有限公司 | Human-computer interaction method and device based on knowledge graph of binocular camera |
| CN115218383A (en) * | 2022-06-17 | 2022-10-21 | 青岛海尔空调器有限总公司 | Air conditioning control method, device, electronic device, and storage medium |
| KR102456438B1 (en) * | 2022-07-13 | 2022-10-19 | (주)인티그리트 | Visual wake-up system using artificial intelligence |
Also Published As
| Publication number | Publication date |
|---|---|
| CN1174337C (en) | 2004-11-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN1174337C (en) | Method and device for identifying whether human eyes are watching or not and application thereof | |
| Ji et al. | Fatigue state detection based on multi-index fusion and state recognition network | |
| CN105893980B (en) | A method and system for evaluating attention and concentration | |
| CN104392227B (en) | Living body finger print determination methods and system | |
| CN101441513B (en) | System for performing non-contact type human-machine interaction by vision | |
| CN104224204B (en) | A kind of Study in Driver Fatigue State Surveillance System based on infrared detection technology | |
| CN111462379A (en) | Access control management method, system and medium containing palm vein and face recognition | |
| CN102982316A (en) | Driver abnormal driving behavior recognition device and method thereof | |
| CN103984924A (en) | Reflection type finger vein recognition bioassay device and method | |
| CN1458006A (en) | Method for detecting fatigue driving based on multiple characteristic fusion | |
| CN106682607A (en) | Offline face recognition system and offline face recognition method based on low-power-consumption embedded and infrared triggering | |
| He et al. | A new segmentation approach for iris recognition based on hand-held capture device | |
| CN107808148A (en) | It is a kind of to gather the unit and method for referring to vein image | |
| CN109598193A (en) | A flame image recognition method based on artificial intelligence | |
| CN108375095A (en) | Range hood control method and range hood | |
| CN103105924A (en) | Man-machine interaction method and device | |
| CN110097006A (en) | A kind of contactless biometric discrimination method based on hand palm print and palm vein Fusion Features | |
| CN108388858B (en) | Iris anti-fake method and device | |
| CN109460729A (en) | A kind of embedded plurality of human faces detection method and system | |
| CN107358646A (en) | A kind of fatigue detecting system and method based on machine vision | |
| CN108363944A (en) | Recognition of face terminal is double to take the photograph method for anti-counterfeit, apparatus and system | |
| Cazzato et al. | Real-time gaze estimation via pupil center tracking | |
| CN109343694B (en) | Gesture recognition system and method for finger-guessing and punching game | |
| CN1144152C (en) | Identification method based on iris recognition | |
| Jian-Nan et al. | Key techniques of eye gaze tracking based on pupil corneal reflection |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| C19 | Lapse of patent right due to non-payment of the annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |